[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:16:59.477067 11517 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.11.63.126:40503
I20260709 13:16:59.478698 11517 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:16:59.479702 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:59.498411 11526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:59.498416 11524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:59.498497 11523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:59.499006 11517 server_base.cc:1061] running on GCE node
I20260709 13:17:00.714570 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:00.714805 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:00.714958 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603020714940 us; error 0 us; skew 500 ppm
I20260709 13:17:00.715754 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:00.721197 11517 webserver.cc:533] Webserver started at http://127.11.63.126:33177/ using document root <none> and password file <none>
I20260709 13:17:00.722110 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:00.722322 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:00.722718 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:00.728504 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603019296230-11517-0/minicluster-data/master-0-root/instance:
uuid: "8d87bdc096194fccaed6c5e7331e9b48"
format_stamp: "Formatted at 2026-07-09 13:17:00 on dist-test-slave-qv7r"
I20260709 13:17:00.736409 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.009s	sys 0.000s
I20260709 13:17:00.742045 11532 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:00.743505 11517 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 13:17:00.744066 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603019296230-11517-0/minicluster-data/master-0-root
uuid: "8d87bdc096194fccaed6c5e7331e9b48"
format_stamp: "Formatted at 2026-07-09 13:17:00 on dist-test-slave-qv7r"
I20260709 13:17:00.744570 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603019296230-11517-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603019296230-11517-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603019296230-11517-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:00.807494 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:00.809072 11517 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:17:00.809546 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:00.884068 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.126:40503
I20260709 13:17:00.884143 11583 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.126:40503 every 8 connection(s)
I20260709 13:17:00.892052 11584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:00.914000 11584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48: Bootstrap starting.
I20260709 13:17:00.920120 11584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:00.922067 11584 log.cc:826] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48: Log is configured to *not* fsync() on all Append() calls
I20260709 13:17:00.926803 11584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48: No bootstrap required, opened a new log
I20260709 13:17:00.947970 11584 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d87bdc096194fccaed6c5e7331e9b48" member_type: VOTER }
I20260709 13:17:00.948524 11584 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:00.948812 11584 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d87bdc096194fccaed6c5e7331e9b48, State: Initialized, Role: FOLLOWER
I20260709 13:17:00.949790 11584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d87bdc096194fccaed6c5e7331e9b48" member_type: VOTER }
I20260709 13:17:00.950289 11584 raft_consensus.cc:399] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:17:00.950584 11584 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:17:00.950920 11584 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:00.955093 11584 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d87bdc096194fccaed6c5e7331e9b48" member_type: VOTER }
I20260709 13:17:00.955900 11584 leader_election.cc:304] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d87bdc096194fccaed6c5e7331e9b48; no voters: 
I20260709 13:17:00.957643 11584 leader_election.cc:290] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:17:00.958009 11587 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:00.960628 11587 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48 [term 1 LEADER]: Becoming Leader. State: Replica: 8d87bdc096194fccaed6c5e7331e9b48, State: Running, Role: LEADER
I20260709 13:17:00.961580 11587 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d87bdc096194fccaed6c5e7331e9b48" member_type: VOTER }
I20260709 13:17:00.962649 11584 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:17:00.971935 11589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d87bdc096194fccaed6c5e7331e9b48. Latest consensus state: current_term: 1 leader_uuid: "8d87bdc096194fccaed6c5e7331e9b48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d87bdc096194fccaed6c5e7331e9b48" member_type: VOTER } }
I20260709 13:17:00.972487 11589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48 [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:00.972748 11588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d87bdc096194fccaed6c5e7331e9b48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d87bdc096194fccaed6c5e7331e9b48" member_type: VOTER } }
I20260709 13:17:00.973405 11588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48 [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:00.976560 11596 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:17:00.989079 11596 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:17:00.991576 11517 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:17:01.007164 11596 catalog_manager.cc:1383] Generated new cluster ID: 9d72a6e2e67c4b98b73789d45bf79938
I20260709 13:17:01.007485 11596 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:17:01.054077 11596 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:17:01.055850 11596 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:17:01.071974 11596 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48: Generated new TSK 0
I20260709 13:17:01.073076 11596 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:17:01.122666 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:01.131910 11606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:01.133003 11607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:01.138968 11517 server_base.cc:1061] running on GCE node
W20260709 13:17:01.139011 11609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:01.140255 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:01.140533 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:01.140749 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603021140728 us; error 0 us; skew 500 ppm
I20260709 13:17:01.141688 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:01.146034 11517 webserver.cc:533] Webserver started at http://127.11.63.65:45471/ using document root <none> and password file <none>
I20260709 13:17:01.146747 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:01.147006 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:01.147364 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:01.148900 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603019296230-11517-0/minicluster-data/ts-0-root/instance:
uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
format_stamp: "Formatted at 2026-07-09 13:17:01 on dist-test-slave-qv7r"
I20260709 13:17:01.156044 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.004s	sys 0.002s
I20260709 13:17:01.161481 11614 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:01.162662 11517 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 13:17:01.163177 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603019296230-11517-0/minicluster-data/ts-0-root
uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
format_stamp: "Formatted at 2026-07-09 13:17:01 on dist-test-slave-qv7r"
I20260709 13:17:01.163594 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603019296230-11517-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603019296230-11517-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603019296230-11517-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:01.183338 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:01.185155 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:01.188078 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:01.193840 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:01.194195 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:01.194552 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:01.194784 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:01.297814 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.65:45363
I20260709 13:17:01.298044 11676 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.65:45363 every 8 connection(s)
I20260709 13:17:01.306751 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:01.318701 11681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:01.322363 11682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:01.328778 11684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:01.330063 11517 server_base.cc:1061] running on GCE node
I20260709 13:17:01.331146 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:01.331434 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:01.331660 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603021331641 us; error 0 us; skew 500 ppm
I20260709 13:17:01.332520 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:01.337116 11517 webserver.cc:533] Webserver started at http://127.11.63.66:45731/ using document root <none> and password file <none>
I20260709 13:17:01.337872 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:01.338161 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:01.338052 11677 heartbeater.cc:344] Connected to a master server at 127.11.63.126:40503
I20260709 13:17:01.338563 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:01.338760 11677 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:01.339814 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603019296230-11517-0/minicluster-data/ts-1-root/instance:
uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
format_stamp: "Formatted at 2026-07-09 13:17:01 on dist-test-slave-qv7r"
I20260709 13:17:01.340269 11677 heartbeater.cc:507] Master 127.11.63.126:40503 requested a full tablet report, sending...
I20260709 13:17:01.344165 11549 ts_manager.cc:194] Registered new tserver with Master: 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363)
I20260709 13:17:01.346653 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:17:01.347404 11549 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47116
I20260709 13:17:01.352432 11689 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:01.353367 11517 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:17:01.353691 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603019296230-11517-0/minicluster-data/ts-1-root
uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
format_stamp: "Formatted at 2026-07-09 13:17:01 on dist-test-slave-qv7r"
I20260709 13:17:01.353978 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603019296230-11517-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603019296230-11517-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603019296230-11517-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:01.380589 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:01.381996 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:01.384249 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:01.387768 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:01.388013 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:01.388252 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:01.388422 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:01.484344 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.66:35361
I20260709 13:17:01.484625 11751 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.66:35361 every 8 connection(s)
I20260709 13:17:01.491278 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:01.502525 11756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:01.506506 11752 heartbeater.cc:344] Connected to a master server at 127.11.63.126:40503
I20260709 13:17:01.507093 11752 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:01.508404 11752 heartbeater.cc:507] Master 127.11.63.126:40503 requested a full tablet report, sending...
I20260709 13:17:01.511704 11549 ts_manager.cc:194] Registered new tserver with Master: 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361)
I20260709 13:17:01.514205 11549 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34182
W20260709 13:17:01.517257 11755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:01.517961 11517 server_base.cc:1061] running on GCE node
W20260709 13:17:01.522661 11758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:01.523654 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:01.523887 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:01.524065 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603021524044 us; error 0 us; skew 500 ppm
I20260709 13:17:01.524623 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:01.527503 11517 webserver.cc:533] Webserver started at http://127.11.63.67:41039/ using document root <none> and password file <none>
I20260709 13:17:01.527971 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:01.528178 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:01.528443 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:01.529451 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603019296230-11517-0/minicluster-data/ts-2-root/instance:
uuid: "8ea1829e48e84553bac8093876f65de9"
format_stamp: "Formatted at 2026-07-09 13:17:01 on dist-test-slave-qv7r"
I20260709 13:17:01.547423 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.017s	user 0.006s	sys 0.005s
I20260709 13:17:01.552718 11763 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:01.553746 11517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:17:01.554083 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603019296230-11517-0/minicluster-data/ts-2-root
uuid: "8ea1829e48e84553bac8093876f65de9"
format_stamp: "Formatted at 2026-07-09 13:17:01 on dist-test-slave-qv7r"
I20260709 13:17:01.554361 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603019296230-11517-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603019296230-11517-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603019296230-11517-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:01.587782 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:01.589026 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:01.590600 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:01.592741 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:01.592921 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:01.593195 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:01.593351 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:01.695240 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.67:46543
I20260709 13:17:01.695919 11825 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.67:46543 every 8 connection(s)
I20260709 13:17:01.714790 11826 heartbeater.cc:344] Connected to a master server at 127.11.63.126:40503
I20260709 13:17:01.715286 11826 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:01.716354 11826 heartbeater.cc:507] Master 127.11.63.126:40503 requested a full tablet report, sending...
I20260709 13:17:01.718670 11549 ts_manager.cc:194] Registered new tserver with Master: 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:01.719446 11517 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017891629s
I20260709 13:17:01.720386 11549 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34188
I20260709 13:17:01.777822 11549 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34192:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:01.781450 11549 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:01.885960 11789 tablet_service.cc:1511] Processing CreateTablet for tablet 85e1a267c28d4122b8e8cd93d34e9c50 (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:01.887785 11789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85e1a267c28d4122b8e8cd93d34e9c50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:01.892416 11790 tablet_service.cc:1511] Processing CreateTablet for tablet 112100a2ecf0402f8a4f2fc0571a6a21 (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:01.894981 11791 tablet_service.cc:1511] Processing CreateTablet for tablet 0749b17931544ef6938fd8a33768d265 (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:01.896606 11788 tablet_service.cc:1511] Processing CreateTablet for tablet e41561d6cff44c26a93018d0f521697a (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:01.898137 11790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 112100a2ecf0402f8a4f2fc0571a6a21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:01.891810 11785 tablet_service.cc:1511] Processing CreateTablet for tablet 90d25fd4fab44cd4a8e27fc6ce41eb61 (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:01.903393 11791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0749b17931544ef6938fd8a33768d265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:01.899024 11786 tablet_service.cc:1511] Processing CreateTablet for tablet 15aec8d8a49042f9adfc877e482c4907 (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:01.900202 11787 tablet_service.cc:1511] Processing CreateTablet for tablet 5dd080bfdd96482486e355711e04fa83 (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:01.904336 11785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90d25fd4fab44cd4a8e27fc6ce41eb61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:01.902058 11784 tablet_service.cc:1511] Processing CreateTablet for tablet ebbcce463d0e4eefb40d760fff058275 (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:01.906925 11787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dd080bfdd96482486e355711e04fa83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:01.908583 11784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebbcce463d0e4eefb40d760fff058275. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:01.910751 11786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15aec8d8a49042f9adfc877e482c4907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:01.928200 11788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e41561d6cff44c26a93018d0f521697a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:01.936043 11783 tablet_service.cc:1511] Processing CreateTablet for tablet e1e3276493b1468aab90033dfbf0ceeb (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:01.937465 11783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1e3276493b1468aab90033dfbf0ceeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:01.967149 11640 tablet_service.cc:1511] Processing CreateTablet for tablet 85e1a267c28d4122b8e8cd93d34e9c50 (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:01.968753 11640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85e1a267c28d4122b8e8cd93d34e9c50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:01.979408 11638 tablet_service.cc:1511] Processing CreateTablet for tablet 5dd080bfdd96482486e355711e04fa83 (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:01.984635 11639 tablet_service.cc:1511] Processing CreateTablet for tablet e41561d6cff44c26a93018d0f521697a (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:01.986634 11639 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e41561d6cff44c26a93018d0f521697a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:01.991516 11634 tablet_service.cc:1511] Processing CreateTablet for tablet e1e3276493b1468aab90033dfbf0ceeb (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:01.993522 11634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1e3276493b1468aab90033dfbf0ceeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:01.998098 11641 tablet_service.cc:1511] Processing CreateTablet for tablet 112100a2ecf0402f8a4f2fc0571a6a21 (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:01.968811 11642 tablet_service.cc:1511] Processing CreateTablet for tablet 0749b17931544ef6938fd8a33768d265 (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION VALUES < 238609294
W20260709 13:17:01.996230 11597 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:02.005012 11641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 112100a2ecf0402f8a4f2fc0571a6a21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:02.008142 11637 tablet_service.cc:1511] Processing CreateTablet for tablet 15aec8d8a49042f9adfc877e482c4907 (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:02.009276 11636 tablet_service.cc:1511] Processing CreateTablet for tablet 90d25fd4fab44cd4a8e27fc6ce41eb61 (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:02.015594 11638 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dd080bfdd96482486e355711e04fa83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:02.019286 11635 tablet_service.cc:1511] Processing CreateTablet for tablet ebbcce463d0e4eefb40d760fff058275 (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:02.020892 11637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15aec8d8a49042f9adfc877e482c4907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:02.028831 11844 tablet_bootstrap.cc:492] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9: Bootstrap starting.
I20260709 13:17:02.030462 11642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0749b17931544ef6938fd8a33768d265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:02.039289 11717 tablet_service.cc:1511] Processing CreateTablet for tablet 0749b17931544ef6938fd8a33768d265 (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:02.040527 11635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebbcce463d0e4eefb40d760fff058275. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:02.044034 11717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0749b17931544ef6938fd8a33768d265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:02.042665 11716 tablet_service.cc:1511] Processing CreateTablet for tablet 112100a2ecf0402f8a4f2fc0571a6a21 (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:02.047884 11715 tablet_service.cc:1511] Processing CreateTablet for tablet 85e1a267c28d4122b8e8cd93d34e9c50 (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:02.054805 11713 tablet_service.cc:1511] Processing CreateTablet for tablet 5dd080bfdd96482486e355711e04fa83 (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:02.057849 11715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85e1a267c28d4122b8e8cd93d34e9c50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:02.058499 11636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90d25fd4fab44cd4a8e27fc6ce41eb61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:02.072127 11713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dd080bfdd96482486e355711e04fa83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:02.081696 11716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 112100a2ecf0402f8a4f2fc0571a6a21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:02.069111 11714 tablet_service.cc:1511] Processing CreateTablet for tablet e41561d6cff44c26a93018d0f521697a (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:02.087349 11710 tablet_service.cc:1511] Processing CreateTablet for tablet ebbcce463d0e4eefb40d760fff058275 (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:02.071287 11712 tablet_service.cc:1511] Processing CreateTablet for tablet 15aec8d8a49042f9adfc877e482c4907 (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:02.095103 11709 tablet_service.cc:1511] Processing CreateTablet for tablet e1e3276493b1468aab90033dfbf0ceeb (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:02.097198 11709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1e3276493b1468aab90033dfbf0ceeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:02.087503 11711 tablet_service.cc:1511] Processing CreateTablet for tablet 90d25fd4fab44cd4a8e27fc6ce41eb61 (DEFAULT_TABLE table=test-workload [id=4a3fa0286d9c4796af9cb9fa3f39a299]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:02.103338 11710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebbcce463d0e4eefb40d760fff058275. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:02.106135 11714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e41561d6cff44c26a93018d0f521697a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:02.110169 11712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15aec8d8a49042f9adfc877e482c4907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:02.120028 11711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90d25fd4fab44cd4a8e27fc6ce41eb61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:02.158550 11844 tablet_bootstrap.cc:654] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.215631 11844 tablet_bootstrap.cc:492] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9: No bootstrap required, opened a new log
I20260709 13:17:02.216676 11844 ts_tablet_manager.cc:1403] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9: Time spent bootstrapping tablet: real 0.198s	user 0.019s	sys 0.012s
I20260709 13:17:02.214272 11845 tablet_bootstrap.cc:492] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32: Bootstrap starting.
I20260709 13:17:02.228883 11847 tablet_bootstrap.cc:492] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e: Bootstrap starting.
I20260709 13:17:02.243777 11844 raft_consensus.cc:359] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.244583 11844 raft_consensus.cc:385] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.244894 11844 raft_consensus.cc:740] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ea1829e48e84553bac8093876f65de9, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.245746 11844 consensus_queue.cc:260] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.246382 11845 tablet_bootstrap.cc:654] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.248271 11826 heartbeater.cc:499] Master 127.11.63.126:40503 was elected leader, sending a full tablet report...
I20260709 13:17:02.248670 11844 ts_tablet_manager.cc:1434] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9: Time spent starting tablet: real 0.032s	user 0.004s	sys 0.004s
I20260709 13:17:02.249641 11844 tablet_bootstrap.cc:492] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9: Bootstrap starting.
I20260709 13:17:02.253819 11847 tablet_bootstrap.cc:654] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.270432 11844 tablet_bootstrap.cc:654] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.270599 11847 tablet_bootstrap.cc:492] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e: No bootstrap required, opened a new log
I20260709 13:17:02.271288 11847 ts_tablet_manager.cc:1403] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e: Time spent bootstrapping tablet: real 0.049s	user 0.019s	sys 0.005s
I20260709 13:17:02.274207 11845 tablet_bootstrap.cc:492] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32: No bootstrap required, opened a new log
I20260709 13:17:02.274339 11847 raft_consensus.cc:359] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } }
I20260709 13:17:02.275297 11847 raft_consensus.cc:385] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.275668 11847 raft_consensus.cc:740] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.274888 11845 ts_tablet_manager.cc:1403] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32: Time spent bootstrapping tablet: real 0.070s	user 0.014s	sys 0.017s
I20260709 13:17:02.276520 11847 consensus_queue.cc:260] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } }
I20260709 13:17:02.283720 11844 tablet_bootstrap.cc:492] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9: No bootstrap required, opened a new log
I20260709 13:17:02.284332 11844 ts_tablet_manager.cc:1403] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9: Time spent bootstrapping tablet: real 0.035s	user 0.004s	sys 0.013s
I20260709 13:17:02.285666 11845 raft_consensus.cc:359] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.287078 11845 raft_consensus.cc:385] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.287492 11845 raft_consensus.cc:740] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.288229 11844 raft_consensus.cc:359] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } }
I20260709 13:17:02.289000 11844 raft_consensus.cc:385] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.289314 11844 raft_consensus.cc:740] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ea1829e48e84553bac8093876f65de9, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.290089 11844 consensus_queue.cc:260] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } }
I20260709 13:17:02.288257 11845 consensus_queue.cc:260] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.294204 11752 heartbeater.cc:499] Master 127.11.63.126:40503 was elected leader, sending a full tablet report...
I20260709 13:17:02.294625 11847 ts_tablet_manager.cc:1434] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.011s
I20260709 13:17:02.295603 11847 tablet_bootstrap.cc:492] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e: Bootstrap starting.
I20260709 13:17:02.300846 11844 ts_tablet_manager.cc:1434] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.004s
I20260709 13:17:02.301977 11844 tablet_bootstrap.cc:492] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9: Bootstrap starting.
I20260709 13:17:02.305337 11847 tablet_bootstrap.cc:654] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.309183 11844 tablet_bootstrap.cc:654] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.321471 11844 tablet_bootstrap.cc:492] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9: No bootstrap required, opened a new log
I20260709 13:17:02.322088 11844 ts_tablet_manager.cc:1403] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.001s
I20260709 13:17:02.326752 11844 raft_consensus.cc:359] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.327731 11844 raft_consensus.cc:385] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.328186 11844 raft_consensus.cc:740] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ea1829e48e84553bac8093876f65de9, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.329183 11844 consensus_queue.cc:260] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.330453 11845 ts_tablet_manager.cc:1434] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32: Time spent starting tablet: real 0.053s	user 0.015s	sys 0.000s
I20260709 13:17:02.332123 11844 ts_tablet_manager.cc:1434] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:17:02.333307 11844 tablet_bootstrap.cc:492] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9: Bootstrap starting.
I20260709 13:17:02.341629 11844 tablet_bootstrap.cc:654] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.344108 11677 heartbeater.cc:499] Master 127.11.63.126:40503 was elected leader, sending a full tablet report...
I20260709 13:17:02.350271 11845 tablet_bootstrap.cc:492] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32: Bootstrap starting.
I20260709 13:17:02.357738 11844 tablet_bootstrap.cc:492] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9: No bootstrap required, opened a new log
I20260709 13:17:02.358130 11845 tablet_bootstrap.cc:654] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.358453 11844 ts_tablet_manager.cc:1403] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.005s
I20260709 13:17:02.359736 11853 raft_consensus.cc:493] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.360232 11853 raft_consensus.cc:515] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.361759 11844 raft_consensus.cc:359] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.363559 11853 leader_election.cc:290] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:02.370886 11852 raft_consensus.cc:493] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.372059 11847 tablet_bootstrap.cc:492] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e: No bootstrap required, opened a new log
I20260709 13:17:02.371608 11852 raft_consensus.cc:515] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } }
I20260709 13:17:02.372586 11847 ts_tablet_manager.cc:1403] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e: Time spent bootstrapping tablet: real 0.077s	user 0.030s	sys 0.028s
I20260709 13:17:02.375530 11847 raft_consensus.cc:359] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.376238 11847 raft_consensus.cc:385] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.376506 11847 raft_consensus.cc:740] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.381448 11845 tablet_bootstrap.cc:492] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32: No bootstrap required, opened a new log
I20260709 13:17:02.382088 11845 ts_tablet_manager.cc:1403] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.009s
I20260709 13:17:02.377229 11847 consensus_queue.cc:260] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.386466 11852 leader_election.cc:290] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543), 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363)
I20260709 13:17:02.386647 11845 raft_consensus.cc:359] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.396940 11845 raft_consensus.cc:385] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.397359 11845 raft_consensus.cc:740] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.398240 11845 consensus_queue.cc:260] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.400906 11847 ts_tablet_manager.cc:1434] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.003s
I20260709 13:17:02.401206 11845 ts_tablet_manager.cc:1434] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:17:02.401908 11847 tablet_bootstrap.cc:492] T e1e3276493b1468aab90033dfbf0ceeb P 928a1fe46e5944c8801f6ff56e1d5a3e: Bootstrap starting.
I20260709 13:17:02.402401 11845 tablet_bootstrap.cc:492] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32: Bootstrap starting.
I20260709 13:17:02.371605 11844 raft_consensus.cc:385] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.410362 11844 raft_consensus.cc:740] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ea1829e48e84553bac8093876f65de9, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.411372 11845 tablet_bootstrap.cc:654] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.411676 11801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d25fd4fab44cd4a8e27fc6ce41eb61" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea1829e48e84553bac8093876f65de9" is_pre_election: true
I20260709 13:17:02.412515 11801 raft_consensus.cc:2468] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 0.
I20260709 13:17:02.414101 11615 leader_election.cc:304] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32, 8ea1829e48e84553bac8093876f65de9; no voters: 
I20260709 13:17:02.414929 11853 raft_consensus.cc:2804] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:02.415418 11853 raft_consensus.cc:493] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.415820 11853 raft_consensus.cc:3060] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:02.417692 11847 tablet_bootstrap.cc:654] T e1e3276493b1468aab90033dfbf0ceeb P 928a1fe46e5944c8801f6ff56e1d5a3e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.411394 11844 consensus_queue.cc:260] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.427392 11844 ts_tablet_manager.cc:1434] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9: Time spent starting tablet: real 0.069s	user 0.006s	sys 0.000s
I20260709 13:17:02.428481 11844 tablet_bootstrap.cc:492] T e41561d6cff44c26a93018d0f521697a P 8ea1829e48e84553bac8093876f65de9: Bootstrap starting.
I20260709 13:17:02.431865 11847 tablet_bootstrap.cc:492] T e1e3276493b1468aab90033dfbf0ceeb P 928a1fe46e5944c8801f6ff56e1d5a3e: No bootstrap required, opened a new log
I20260709 13:17:02.432403 11847 ts_tablet_manager.cc:1403] T e1e3276493b1468aab90033dfbf0ceeb P 928a1fe46e5944c8801f6ff56e1d5a3e: Time spent bootstrapping tablet: real 0.031s	user 0.004s	sys 0.011s
I20260709 13:17:02.435474 11847 raft_consensus.cc:359] T e1e3276493b1468aab90033dfbf0ceeb P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.437436 11847 raft_consensus.cc:385] T e1e3276493b1468aab90033dfbf0ceeb P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.437810 11847 raft_consensus.cc:740] T e1e3276493b1468aab90033dfbf0ceeb P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.438520 11847 consensus_queue.cc:260] T e1e3276493b1468aab90033dfbf0ceeb P 928a1fe46e5944c8801f6ff56e1d5a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.439183 11853 raft_consensus.cc:515] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.441506 11853 leader_election.cc:290] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 election: Requested vote from peers 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:02.442672 11801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d25fd4fab44cd4a8e27fc6ce41eb61" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea1829e48e84553bac8093876f65de9"
I20260709 13:17:02.443361 11801 raft_consensus.cc:3060] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:02.447556 11727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d25fd4fab44cd4a8e27fc6ce41eb61" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" is_pre_election: true
I20260709 13:17:02.447908 11726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d25fd4fab44cd4a8e27fc6ce41eb61" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
I20260709 13:17:02.450976 11801 raft_consensus.cc:2468] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 1.
I20260709 13:17:02.452373 11615 leader_election.cc:304] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32, 8ea1829e48e84553bac8093876f65de9; no voters: 
W20260709 13:17:02.452844 11827 tablet.cc:2406] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:17:02.457468 11845 tablet_bootstrap.cc:492] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32: No bootstrap required, opened a new log
I20260709 13:17:02.458056 11845 ts_tablet_manager.cc:1403] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32: Time spent bootstrapping tablet: real 0.056s	user 0.012s	sys 0.004s
I20260709 13:17:02.456488 11847 ts_tablet_manager.cc:1434] T e1e3276493b1468aab90033dfbf0ceeb P 928a1fe46e5944c8801f6ff56e1d5a3e: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:17:02.461819 11847 tablet_bootstrap.cc:492] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e: Bootstrap starting.
I20260709 13:17:02.466704 11853 raft_consensus.cc:2804] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:17:02.467775 11616 leader_election.cc:343] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:02.466102 11844 tablet_bootstrap.cc:654] T e41561d6cff44c26a93018d0f521697a P 8ea1829e48e84553bac8093876f65de9: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.468776 11652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0749b17931544ef6938fd8a33768d265" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" is_pre_election: true
W20260709 13:17:02.471135 11691 leader_election.cc:343] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:02.473248 11801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0749b17931544ef6938fd8a33768d265" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea1829e48e84553bac8093876f65de9" is_pre_election: true
I20260709 13:17:02.473951 11801 raft_consensus.cc:2468] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 928a1fe46e5944c8801f6ff56e1d5a3e in term 0.
I20260709 13:17:02.475159 11847 tablet_bootstrap.cc:654] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.475647 11690 leader_election.cc:304] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ea1829e48e84553bac8093876f65de9, 928a1fe46e5944c8801f6ff56e1d5a3e; no voters: 8134a93015124c3ba3a2a7bf5ae8af32
I20260709 13:17:02.476866 11863 raft_consensus.cc:2804] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:02.477239 11863 raft_consensus.cc:493] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.477684 11863 raft_consensus.cc:3060] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:02.477304 11616 leader_election.cc:343] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:02.481580 11853 raft_consensus.cc:697] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Becoming Leader. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Running, Role: LEADER
I20260709 13:17:02.482005 11845 raft_consensus.cc:359] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.482885 11845 raft_consensus.cc:385] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.482502 11853 consensus_queue.cc:237] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.483222 11845 raft_consensus.cc:740] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.484016 11845 consensus_queue.cc:260] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.487378 11863 raft_consensus.cc:515] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } }
I20260709 13:17:02.491092 11801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0749b17931544ef6938fd8a33768d265" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea1829e48e84553bac8093876f65de9"
I20260709 13:17:02.491693 11863 leader_election.cc:290] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 1 election: Requested vote from peers 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543), 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363)
I20260709 13:17:02.491191 11652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0749b17931544ef6938fd8a33768d265" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
W20260709 13:17:02.494787 11691 leader_election.cc:343] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:02.491788 11801 raft_consensus.cc:3060] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:02.495692 11847 tablet_bootstrap.cc:492] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e: No bootstrap required, opened a new log
I20260709 13:17:02.496230 11847 ts_tablet_manager.cc:1403] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e: Time spent bootstrapping tablet: real 0.035s	user 0.009s	sys 0.005s
I20260709 13:17:02.499043 11845 ts_tablet_manager.cc:1434] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32: Time spent starting tablet: real 0.041s	user 0.001s	sys 0.010s
I20260709 13:17:02.499300 11847 raft_consensus.cc:359] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.500030 11847 raft_consensus.cc:385] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.500339 11847 raft_consensus.cc:740] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.501909 11844 tablet_bootstrap.cc:492] T e41561d6cff44c26a93018d0f521697a P 8ea1829e48e84553bac8093876f65de9: No bootstrap required, opened a new log
I20260709 13:17:02.502256 11801 raft_consensus.cc:2468] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 928a1fe46e5944c8801f6ff56e1d5a3e in term 1.
I20260709 13:17:02.502470 11844 ts_tablet_manager.cc:1403] T e41561d6cff44c26a93018d0f521697a P 8ea1829e48e84553bac8093876f65de9: Time spent bootstrapping tablet: real 0.074s	user 0.013s	sys 0.001s
I20260709 13:17:02.503666 11690 leader_election.cc:304] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ea1829e48e84553bac8093876f65de9, 928a1fe46e5944c8801f6ff56e1d5a3e; no voters: 8134a93015124c3ba3a2a7bf5ae8af32
I20260709 13:17:02.505787 11863 raft_consensus.cc:2804] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:02.506294 11863 raft_consensus.cc:697] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 LEADER]: Becoming Leader. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Running, Role: LEADER
I20260709 13:17:02.506966 11863 consensus_queue.cc:237] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } }
I20260709 13:17:02.508549 11548 catalog_manager.cc:5697] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:02.505282 11844 raft_consensus.cc:359] T e41561d6cff44c26a93018d0f521697a P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.511051 11844 raft_consensus.cc:385] T e41561d6cff44c26a93018d0f521697a P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.511579 11844 raft_consensus.cc:740] T e41561d6cff44c26a93018d0f521697a P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ea1829e48e84553bac8093876f65de9, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.512816 11844 consensus_queue.cc:260] T e41561d6cff44c26a93018d0f521697a P 8ea1829e48e84553bac8093876f65de9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.515592 11844 ts_tablet_manager.cc:1434] T e41561d6cff44c26a93018d0f521697a P 8ea1829e48e84553bac8093876f65de9: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:17:02.517326 11844 tablet_bootstrap.cc:492] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9: Bootstrap starting.
I20260709 13:17:02.517935 11847 consensus_queue.cc:260] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.520192 11847 ts_tablet_manager.cc:1434] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 13:17:02.516266 11845 tablet_bootstrap.cc:492] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32: Bootstrap starting.
I20260709 13:17:02.521123 11847 tablet_bootstrap.cc:492] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e: Bootstrap starting.
I20260709 13:17:02.531239 11845 tablet_bootstrap.cc:654] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.545544 11844 tablet_bootstrap.cc:654] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.554783 11547 catalog_manager.cc:5697] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e reported cstate change: term changed from 0 to 1, leader changed from <none> to 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66). New cstate: current_term: 1 leader_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:02.563725 11847 tablet_bootstrap.cc:654] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.563283 11848 raft_consensus.cc:493] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.565517 11848 raft_consensus.cc:515] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.574121 11848 leader_election.cc:290] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363), 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361)
I20260709 13:17:02.591559 11845 tablet_bootstrap.cc:492] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32: No bootstrap required, opened a new log
I20260709 13:17:02.592353 11845 ts_tablet_manager.cc:1403] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32: Time spent bootstrapping tablet: real 0.076s	user 0.031s	sys 0.021s
I20260709 13:17:02.611364 11845 raft_consensus.cc:359] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.611941 11847 tablet_bootstrap.cc:492] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e: No bootstrap required, opened a new log
I20260709 13:17:02.612162 11845 raft_consensus.cc:385] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.612421 11847 ts_tablet_manager.cc:1403] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e: Time spent bootstrapping tablet: real 0.091s	user 0.011s	sys 0.021s
I20260709 13:17:02.612458 11845 raft_consensus.cc:740] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.613747 11845 consensus_queue.cc:260] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.616602 11845 ts_tablet_manager.cc:1434] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:17:02.617705 11845 tablet_bootstrap.cc:492] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32: Bootstrap starting.
I20260709 13:17:02.618283 11844 tablet_bootstrap.cc:492] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9: No bootstrap required, opened a new log
I20260709 13:17:02.621289 11844 ts_tablet_manager.cc:1403] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9: Time spent bootstrapping tablet: real 0.104s	user 0.010s	sys 0.010s
I20260709 13:17:02.624596 11844 raft_consensus.cc:359] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.626019 11853 raft_consensus.cc:493] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.618798 11847 raft_consensus.cc:359] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.629878 11844 raft_consensus.cc:385] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.631645 11853 raft_consensus.cc:515] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.638749 11727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85e1a267c28d4122b8e8cd93d34e9c50" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" is_pre_election: true
I20260709 13:17:02.638847 11853 leader_election.cc:290] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:02.639454 11727 raft_consensus.cc:2468] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 0.
I20260709 13:17:02.632182 11847 raft_consensus.cc:385] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.639952 11801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85e1a267c28d4122b8e8cd93d34e9c50" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea1829e48e84553bac8093876f65de9" is_pre_election: true
I20260709 13:17:02.640246 11844 raft_consensus.cc:740] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ea1829e48e84553bac8093876f65de9, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.641307 11616 leader_election.cc:304] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32, 928a1fe46e5944c8801f6ff56e1d5a3e; no voters: 
I20260709 13:17:02.641068 11801 raft_consensus.cc:2468] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 0.
I20260709 13:17:02.652410 11853 raft_consensus.cc:2804] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:02.652311 11652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85e1a267c28d4122b8e8cd93d34e9c50" candidate_uuid: "8ea1829e48e84553bac8093876f65de9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" is_pre_election: true
I20260709 13:17:02.652856 11853 raft_consensus.cc:493] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.653268 11853 raft_consensus.cc:3060] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:02.640549 11847 raft_consensus.cc:740] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.658607 11847 consensus_queue.cc:260] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.659439 11853 raft_consensus.cc:515] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.660864 11652 raft_consensus.cc:2393] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ea1829e48e84553bac8093876f65de9 in current term 1: Already voted for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in this term.
I20260709 13:17:02.661911 11853 leader_election.cc:290] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 election: Requested vote from peers 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:02.651825 11844 consensus_queue.cc:260] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.662643 11727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85e1a267c28d4122b8e8cd93d34e9c50" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
I20260709 13:17:02.663188 11727 raft_consensus.cc:3060] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:02.665551 11844 ts_tablet_manager.cc:1434] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9: Time spent starting tablet: real 0.044s	user 0.010s	sys 0.000s
I20260709 13:17:02.666496 11847 ts_tablet_manager.cc:1434] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e: Time spent starting tablet: real 0.054s	user 0.004s	sys 0.007s
I20260709 13:17:02.666702 11844 tablet_bootstrap.cc:492] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8ea1829e48e84553bac8093876f65de9: Bootstrap starting.
I20260709 13:17:02.667573 11847 tablet_bootstrap.cc:492] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e: Bootstrap starting.
I20260709 13:17:02.672380 11727 raft_consensus.cc:2468] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 1.
I20260709 13:17:02.673758 11616 leader_election.cc:304] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32, 928a1fe46e5944c8801f6ff56e1d5a3e; no voters: 
I20260709 13:17:02.674576 11853 raft_consensus.cc:2804] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:02.675156 11853 raft_consensus.cc:697] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Becoming Leader. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Running, Role: LEADER
I20260709 13:17:02.676694 11845 tablet_bootstrap.cc:654] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.677101 11848 raft_consensus.cc:493] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.677639 11848 raft_consensus.cc:515] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.676121 11853 consensus_queue.cc:237] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.680357 11801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85e1a267c28d4122b8e8cd93d34e9c50" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea1829e48e84553bac8093876f65de9"
I20260709 13:17:02.681033 11848 leader_election.cc:290] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363), 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361)
I20260709 13:17:02.674312 11844 tablet_bootstrap.cc:654] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8ea1829e48e84553bac8093876f65de9: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.683260 11847 tablet_bootstrap.cc:654] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.681104 11801 raft_consensus.cc:3060] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:02.688560 11652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15aec8d8a49042f9adfc877e482c4907" candidate_uuid: "8ea1829e48e84553bac8093876f65de9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" is_pre_election: true
W20260709 13:17:02.694852 11765 leader_election.cc:343] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:02.695691 11848 raft_consensus.cc:493] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.696379 11848 raft_consensus.cc:515] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.699499 11848 leader_election.cc:290] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363), 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361)
I20260709 13:17:02.700562 11652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e3276493b1468aab90033dfbf0ceeb" candidate_uuid: "8ea1829e48e84553bac8093876f65de9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" is_pre_election: true
W20260709 13:17:02.701988 11765 leader_election.cc:343] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:02.708544 11801 raft_consensus.cc:2468] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 1.
I20260709 13:17:02.714470 11725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e3276493b1468aab90033dfbf0ceeb" candidate_uuid: "8ea1829e48e84553bac8093876f65de9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" is_pre_election: true
I20260709 13:17:02.714479 11847 tablet_bootstrap.cc:492] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e: No bootstrap required, opened a new log
I20260709 13:17:02.715602 11725 raft_consensus.cc:2468] T e1e3276493b1468aab90033dfbf0ceeb P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ea1829e48e84553bac8093876f65de9 in term 0.
I20260709 13:17:02.715618 11726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15aec8d8a49042f9adfc877e482c4907" candidate_uuid: "8ea1829e48e84553bac8093876f65de9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" is_pre_election: true
I20260709 13:17:02.714808 11727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85e1a267c28d4122b8e8cd93d34e9c50" candidate_uuid: "8ea1829e48e84553bac8093876f65de9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" is_pre_election: true
I20260709 13:17:02.716461 11726 raft_consensus.cc:2468] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ea1829e48e84553bac8093876f65de9 in term 0.
I20260709 13:17:02.717089 11727 raft_consensus.cc:2393] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ea1829e48e84553bac8093876f65de9 in current term 1: Already voted for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in this term.
I20260709 13:17:02.717113 11765 leader_election.cc:304] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ea1829e48e84553bac8093876f65de9, 928a1fe46e5944c8801f6ff56e1d5a3e; no voters: 8134a93015124c3ba3a2a7bf5ae8af32
I20260709 13:17:02.717895 11845 tablet_bootstrap.cc:492] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32: No bootstrap required, opened a new log
I20260709 13:17:02.718269 11848 raft_consensus.cc:2804] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:02.715920 11847 ts_tablet_manager.cc:1403] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e: Time spent bootstrapping tablet: real 0.049s	user 0.018s	sys 0.004s
I20260709 13:17:02.718581 11765 leader_election.cc:304] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ea1829e48e84553bac8093876f65de9, 928a1fe46e5944c8801f6ff56e1d5a3e; no voters: 8134a93015124c3ba3a2a7bf5ae8af32
I20260709 13:17:02.718822 11848 raft_consensus.cc:493] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.719465 11848 raft_consensus.cc:3060] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:02.722133 11847 raft_consensus.cc:359] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.723001 11847 raft_consensus.cc:385] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.723335 11847 raft_consensus.cc:740] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.724189 11847 consensus_queue.cc:260] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.718432 11845 ts_tablet_manager.cc:1403] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32: Time spent bootstrapping tablet: real 0.101s	user 0.014s	sys 0.012s
I20260709 13:17:02.728658 11845 raft_consensus.cc:359] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.729470 11845 raft_consensus.cc:385] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.729827 11845 raft_consensus.cc:740] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.730633 11845 consensus_queue.cc:260] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.730998 11848 raft_consensus.cc:515] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.733507 11845 ts_tablet_manager.cc:1434] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:17:02.734362 11844 tablet_bootstrap.cc:492] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8ea1829e48e84553bac8093876f65de9: No bootstrap required, opened a new log
I20260709 13:17:02.735206 11844 ts_tablet_manager.cc:1403] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8ea1829e48e84553bac8093876f65de9: Time spent bootstrapping tablet: real 0.069s	user 0.006s	sys 0.013s
I20260709 13:17:02.731635 11885 raft_consensus.cc:2804] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:02.738799 11885 raft_consensus.cc:493] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.739336 11885 raft_consensus.cc:3060] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:02.733831 11765 leader_election.cc:304] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ea1829e48e84553bac8093876f65de9; no voters: 8134a93015124c3ba3a2a7bf5ae8af32, 928a1fe46e5944c8801f6ff56e1d5a3e
I20260709 13:17:02.745923 11869 raft_consensus.cc:493] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.746618 11869 raft_consensus.cc:515] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.748863 11848 leader_election.cc:290] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 [CANDIDATE]: Term 1 election: Requested vote from peers 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363), 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361)
I20260709 13:17:02.750240 11848 raft_consensus.cc:2749] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:02.753428 11869 leader_election.cc:290] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:02.754245 11727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41561d6cff44c26a93018d0f521697a" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" is_pre_election: true
I20260709 13:17:02.754681 11847 ts_tablet_manager.cc:1434] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.003s
I20260709 13:17:02.754206 11845 tablet_bootstrap.cc:492] T e1e3276493b1468aab90033dfbf0ceeb P 8134a93015124c3ba3a2a7bf5ae8af32: Bootstrap starting.
I20260709 13:17:02.754822 11801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41561d6cff44c26a93018d0f521697a" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea1829e48e84553bac8093876f65de9" is_pre_election: true
I20260709 13:17:02.755587 11801 raft_consensus.cc:2468] T e41561d6cff44c26a93018d0f521697a P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 0.
I20260709 13:17:02.756129 11547 catalog_manager.cc:5697] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:02.757066 11615 leader_election.cc:304] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32, 8ea1829e48e84553bac8093876f65de9; no voters: 
I20260709 13:17:02.754910 11727 raft_consensus.cc:2468] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 0.
I20260709 13:17:02.757896 11869 raft_consensus.cc:2804] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:02.758478 11869 raft_consensus.cc:493] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.758882 11869 raft_consensus.cc:3060] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:02.759996 11844 raft_consensus.cc:359] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.761096 11844 raft_consensus.cc:385] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.761472 11844 raft_consensus.cc:740] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ea1829e48e84553bac8093876f65de9, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.762329 11844 consensus_queue.cc:260] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8ea1829e48e84553bac8093876f65de9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.765285 11844 ts_tablet_manager.cc:1434] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8ea1829e48e84553bac8093876f65de9: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.005s
I20260709 13:17:02.766402 11844 tablet_bootstrap.cc:492] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9: Bootstrap starting.
I20260709 13:17:02.766676 11869 raft_consensus.cc:515] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.770690 11727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41561d6cff44c26a93018d0f521697a" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
I20260709 13:17:02.771407 11727 raft_consensus.cc:3060] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:02.773737 11844 tablet_bootstrap.cc:654] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.780475 11727 raft_consensus.cc:2468] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 1.
I20260709 13:17:02.781872 11616 leader_election.cc:304] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32, 928a1fe46e5944c8801f6ff56e1d5a3e; no voters: 
I20260709 13:17:02.782838 11853 raft_consensus.cc:2804] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:02.783174 11847 tablet_bootstrap.cc:492] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e: Bootstrap starting.
I20260709 13:17:02.783344 11853 raft_consensus.cc:697] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Becoming Leader. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Running, Role: LEADER
I20260709 13:17:02.784197 11853 consensus_queue.cc:237] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.790002 11869 leader_election.cc:290] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 election: Requested vote from peers 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:02.790912 11845 tablet_bootstrap.cc:654] T e1e3276493b1468aab90033dfbf0ceeb P 8134a93015124c3ba3a2a7bf5ae8af32: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.792992 11885 raft_consensus.cc:515] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.794628 11801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41561d6cff44c26a93018d0f521697a" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea1829e48e84553bac8093876f65de9"
I20260709 13:17:02.795284 11801 raft_consensus.cc:3060] T e41561d6cff44c26a93018d0f521697a P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:02.795436 11885 leader_election.cc:290] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [CANDIDATE]: Term 1 election: Requested vote from peers 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363), 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361)
I20260709 13:17:02.795903 11847 tablet_bootstrap.cc:654] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.800877 11844 tablet_bootstrap.cc:492] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9: No bootstrap required, opened a new log
I20260709 13:17:02.801743 11844 ts_tablet_manager.cc:1403] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9: Time spent bootstrapping tablet: real 0.036s	user 0.007s	sys 0.016s
I20260709 13:17:02.803874 11801 raft_consensus.cc:2468] T e41561d6cff44c26a93018d0f521697a P 8ea1829e48e84553bac8093876f65de9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 1.
I20260709 13:17:02.804774 11844 raft_consensus.cc:359] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.805540 11844 raft_consensus.cc:385] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.805891 11844 raft_consensus.cc:740] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ea1829e48e84553bac8093876f65de9, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.806615 11844 consensus_queue.cc:260] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
W20260709 13:17:02.806810 11765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:35361, user_credentials={real_user=slave}} blocked reactor thread for 75073.8us
I20260709 13:17:02.807053 11845 tablet_bootstrap.cc:492] T e1e3276493b1468aab90033dfbf0ceeb P 8134a93015124c3ba3a2a7bf5ae8af32: No bootstrap required, opened a new log
I20260709 13:17:02.808257 11845 ts_tablet_manager.cc:1403] T e1e3276493b1468aab90033dfbf0ceeb P 8134a93015124c3ba3a2a7bf5ae8af32: Time spent bootstrapping tablet: real 0.064s	user 0.011s	sys 0.002s
I20260709 13:17:02.807300 11546 catalog_manager.cc:5697] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:02.808717 11652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e3276493b1468aab90033dfbf0ceeb" candidate_uuid: "8ea1829e48e84553bac8093876f65de9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
I20260709 13:17:02.809078 11844 ts_tablet_manager.cc:1434] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 13:17:02.810268 11844 tablet_bootstrap.cc:492] T ebbcce463d0e4eefb40d760fff058275 P 8ea1829e48e84553bac8093876f65de9: Bootstrap starting.
I20260709 13:17:02.811307 11845 raft_consensus.cc:359] T e1e3276493b1468aab90033dfbf0ceeb P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.812008 11845 raft_consensus.cc:385] T e1e3276493b1468aab90033dfbf0ceeb P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.812305 11845 raft_consensus.cc:740] T e1e3276493b1468aab90033dfbf0ceeb P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.813050 11845 consensus_queue.cc:260] T e1e3276493b1468aab90033dfbf0ceeb P 8134a93015124c3ba3a2a7bf5ae8af32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.814033 11652 raft_consensus.cc:3060] T e1e3276493b1468aab90033dfbf0ceeb P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:02.815415 11845 ts_tablet_manager.cc:1434] T e1e3276493b1468aab90033dfbf0ceeb P 8134a93015124c3ba3a2a7bf5ae8af32: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:17:02.816417 11845 tablet_bootstrap.cc:492] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32: Bootstrap starting.
I20260709 13:17:02.817684 11844 tablet_bootstrap.cc:654] T ebbcce463d0e4eefb40d760fff058275 P 8ea1829e48e84553bac8093876f65de9: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.819792 11847 tablet_bootstrap.cc:492] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e: No bootstrap required, opened a new log
I20260709 13:17:02.820278 11847 ts_tablet_manager.cc:1403] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e: Time spent bootstrapping tablet: real 0.037s	user 0.012s	sys 0.000s
I20260709 13:17:02.824182 11847 raft_consensus.cc:359] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.825304 11847 raft_consensus.cc:385] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.825758 11847 raft_consensus.cc:740] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.826993 11847 consensus_queue.cc:260] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.830385 11886 raft_consensus.cc:493] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.830926 11886 raft_consensus.cc:515] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.833267 11886 leader_election.cc:290] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363), 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361)
I20260709 13:17:02.833878 11853 raft_consensus.cc:493] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.834374 11853 raft_consensus.cc:515] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.835300 11845 tablet_bootstrap.cc:654] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.836591 11853 leader_election.cc:290] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:02.839066 11801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "112100a2ecf0402f8a4f2fc0571a6a21" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea1829e48e84553bac8093876f65de9" is_pre_election: true
I20260709 13:17:02.839748 11801 raft_consensus.cc:2468] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 0.
I20260709 13:17:02.841034 11615 leader_election.cc:304] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32, 8ea1829e48e84553bac8093876f65de9; no voters: 
I20260709 13:17:02.841841 11853 raft_consensus.cc:2804] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:02.842017 11727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "112100a2ecf0402f8a4f2fc0571a6a21" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" is_pre_election: true
I20260709 13:17:02.842231 11853 raft_consensus.cc:493] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.842727 11853 raft_consensus.cc:3060] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:02.843825 11726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1e3276493b1468aab90033dfbf0ceeb" candidate_uuid: "8ea1829e48e84553bac8093876f65de9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
I20260709 13:17:02.844388 11726 raft_consensus.cc:3060] T e1e3276493b1468aab90033dfbf0ceeb P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:02.847352 11847 ts_tablet_manager.cc:1434] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e: Time spent starting tablet: real 0.027s	user 0.010s	sys 0.000s
I20260709 13:17:02.849861 11853 raft_consensus.cc:515] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.852155 11853 leader_election.cc:290] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 election: Requested vote from peers 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:02.852893 11725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "112100a2ecf0402f8a4f2fc0571a6a21" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
I20260709 13:17:02.854858 11844 tablet_bootstrap.cc:492] T ebbcce463d0e4eefb40d760fff058275 P 8ea1829e48e84553bac8093876f65de9: No bootstrap required, opened a new log
I20260709 13:17:02.856923 11847 tablet_bootstrap.cc:492] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e: Bootstrap starting.
I20260709 13:17:02.856958 11800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "112100a2ecf0402f8a4f2fc0571a6a21" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea1829e48e84553bac8093876f65de9"
I20260709 13:17:02.857762 11800 raft_consensus.cc:3060] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:02.858470 11853 raft_consensus.cc:493] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.858929 11853 raft_consensus.cc:515] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.861115 11853 leader_election.cc:290] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:02.861770 11844 ts_tablet_manager.cc:1403] T ebbcce463d0e4eefb40d760fff058275 P 8ea1829e48e84553bac8093876f65de9: Time spent bootstrapping tablet: real 0.052s	user 0.014s	sys 0.014s
I20260709 13:17:02.863449 11725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd080bfdd96482486e355711e04fa83" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" is_pre_election: true
W20260709 13:17:02.865015 11616 leader_election.cc:343] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:02.864941 11844 raft_consensus.cc:359] T ebbcce463d0e4eefb40d760fff058275 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.865592 11847 tablet_bootstrap.cc:654] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.865803 11844 raft_consensus.cc:385] T ebbcce463d0e4eefb40d760fff058275 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.866144 11844 raft_consensus.cc:740] T ebbcce463d0e4eefb40d760fff058275 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ea1829e48e84553bac8093876f65de9, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.866554 11800 raft_consensus.cc:2468] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8ea1829e48e84553bac8093876f65de9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 1.
I20260709 13:17:02.851619 11726 raft_consensus.cc:2468] T e1e3276493b1468aab90033dfbf0ceeb P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ea1829e48e84553bac8093876f65de9 in term 1.
I20260709 13:17:02.867718 11799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd080bfdd96482486e355711e04fa83" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea1829e48e84553bac8093876f65de9" is_pre_election: true
I20260709 13:17:02.869057 11799 raft_consensus.cc:2468] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 0.
I20260709 13:17:02.869302 11725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15aec8d8a49042f9adfc877e482c4907" candidate_uuid: "8ea1829e48e84553bac8093876f65de9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
I20260709 13:17:02.870260 11725 raft_consensus.cc:3060] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:02.869902 11844 consensus_queue.cc:260] T ebbcce463d0e4eefb40d760fff058275 P 8ea1829e48e84553bac8093876f65de9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.842741 11727 raft_consensus.cc:2468] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 0.
I20260709 13:17:02.873752 11651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15aec8d8a49042f9adfc877e482c4907" candidate_uuid: "8ea1829e48e84553bac8093876f65de9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
I20260709 13:17:02.868721 11652 raft_consensus.cc:2468] T e1e3276493b1468aab90033dfbf0ceeb P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ea1829e48e84553bac8093876f65de9 in term 1.
I20260709 13:17:02.876281 11844 ts_tablet_manager.cc:1434] T ebbcce463d0e4eefb40d760fff058275 P 8ea1829e48e84553bac8093876f65de9: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:17:02.870996 11615 leader_election.cc:304] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32, 8ea1829e48e84553bac8093876f65de9; no voters: 928a1fe46e5944c8801f6ff56e1d5a3e
I20260709 13:17:02.880584 11615 leader_election.cc:304] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32, 8ea1829e48e84553bac8093876f65de9; no voters: 928a1fe46e5944c8801f6ff56e1d5a3e
I20260709 13:17:02.881341 11869 raft_consensus.cc:2804] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:02.881645 11853 raft_consensus.cc:2804] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:02.881834 11869 raft_consensus.cc:697] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Becoming Leader. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Running, Role: LEADER
I20260709 13:17:02.882004 11853 raft_consensus.cc:493] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.882392 11853 raft_consensus.cc:3060] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:02.882731 11869 consensus_queue.cc:237] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.885076 11651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd080bfdd96482486e355711e04fa83" candidate_uuid: "8ea1829e48e84553bac8093876f65de9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" is_pre_election: true
I20260709 13:17:02.890612 11845 tablet_bootstrap.cc:492] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32: No bootstrap required, opened a new log
I20260709 13:17:02.890758 11726 raft_consensus.cc:3060] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:02.890554 11853 raft_consensus.cc:515] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.890988 11725 raft_consensus.cc:2468] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ea1829e48e84553bac8093876f65de9 in term 1.
I20260709 13:17:02.891196 11845 ts_tablet_manager.cc:1403] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32: Time spent bootstrapping tablet: real 0.075s	user 0.015s	sys 0.007s
I20260709 13:17:02.894548 11853 leader_election.cc:290] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 election: Requested vote from peers 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:02.896164 11725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd080bfdd96482486e355711e04fa83" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
W20260709 13:17:02.897490 11616 leader_election.cc:343] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:02.896222 11727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd080bfdd96482486e355711e04fa83" candidate_uuid: "8ea1829e48e84553bac8093876f65de9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" is_pre_election: true
I20260709 13:17:02.900916 11800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd080bfdd96482486e355711e04fa83" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea1829e48e84553bac8093876f65de9"
I20260709 13:17:02.901501 11800 raft_consensus.cc:3060] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:02.901930 11845 raft_consensus.cc:359] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.906250 11651 raft_consensus.cc:2393] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8ea1829e48e84553bac8093876f65de9 in current term 1: Already voted for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in this term.
I20260709 13:17:02.906178 11765 leader_election.cc:304] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ea1829e48e84553bac8093876f65de9, 928a1fe46e5944c8801f6ff56e1d5a3e; no voters: 
W20260709 13:17:02.907612 11765 leader_election.cc:343] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:02.908762 11765 leader_election.cc:304] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8ea1829e48e84553bac8093876f65de9, 928a1fe46e5944c8801f6ff56e1d5a3e; no voters: 8134a93015124c3ba3a2a7bf5ae8af32
I20260709 13:17:02.908833 11546 catalog_manager.cc:5697] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:02.910148 11885 raft_consensus.cc:2804] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:02.910670 11885 raft_consensus.cc:697] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [term 1 LEADER]: Becoming Leader. State: Replica: 8ea1829e48e84553bac8093876f65de9, State: Running, Role: LEADER
I20260709 13:17:02.910974 11845 raft_consensus.cc:385] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.909721 11886 raft_consensus.cc:2804] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:02.911471 11845 raft_consensus.cc:740] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.912706 11886 raft_consensus.cc:697] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 [term 1 LEADER]: Becoming Leader. State: Replica: 8ea1829e48e84553bac8093876f65de9, State: Running, Role: LEADER
I20260709 13:17:02.913328 11845 consensus_queue.cc:260] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.913753 11886 consensus_queue.cc:237] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.917042 11845 ts_tablet_manager.cc:1434] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:17:02.918505 11845 tablet_bootstrap.cc:492] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32: Bootstrap starting.
I20260709 13:17:02.911491 11885 consensus_queue.cc:237] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
W20260709 13:17:02.920290 11765 leader_election.cc:343] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:02.921434 11765 leader_election.cc:304] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8ea1829e48e84553bac8093876f65de9; no voters: 8134a93015124c3ba3a2a7bf5ae8af32, 928a1fe46e5944c8801f6ff56e1d5a3e
I20260709 13:17:02.931497 11845 tablet_bootstrap.cc:654] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.933846 11800 raft_consensus.cc:2468] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 1.
I20260709 13:17:02.934386 11848 raft_consensus.cc:2749] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:02.938437 11869 consensus_queue.cc:1048] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:02.940485 11615 leader_election.cc:304] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32, 8ea1829e48e84553bac8093876f65de9; no voters: 928a1fe46e5944c8801f6ff56e1d5a3e
I20260709 13:17:02.941371 11869 raft_consensus.cc:2804] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:02.942031 11869 raft_consensus.cc:697] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Becoming Leader. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Running, Role: LEADER
I20260709 13:17:02.942906 11869 consensus_queue.cc:237] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.943466 11866 raft_consensus.cc:493] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.943951 11866 raft_consensus.cc:515] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.946421 11866 leader_election.cc:290] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:02.947234 11727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebbcce463d0e4eefb40d760fff058275" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" is_pre_election: true
I20260709 13:17:02.947542 11800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebbcce463d0e4eefb40d760fff058275" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea1829e48e84553bac8093876f65de9" is_pre_election: true
I20260709 13:17:02.948216 11800 raft_consensus.cc:2468] T ebbcce463d0e4eefb40d760fff058275 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 0.
W20260709 13:17:02.948740 11616 leader_election.cc:343] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:02.954526 11615 leader_election.cc:304] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32, 8ea1829e48e84553bac8093876f65de9; no voters: 928a1fe46e5944c8801f6ff56e1d5a3e
I20260709 13:17:02.955379 11869 raft_consensus.cc:2804] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:02.955828 11869 raft_consensus.cc:493] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.956724 11869 raft_consensus.cc:3060] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:02.956449 11548 catalog_manager.cc:5697] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:02.959479 11845 tablet_bootstrap.cc:492] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32: No bootstrap required, opened a new log
I20260709 13:17:02.959949 11845 ts_tablet_manager.cc:1403] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32: Time spent bootstrapping tablet: real 0.042s	user 0.011s	sys 0.013s
I20260709 13:17:02.962586 11845 raft_consensus.cc:359] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } }
I20260709 13:17:02.963238 11845 raft_consensus.cc:385] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.963513 11845 raft_consensus.cc:740] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.963495 11869 raft_consensus.cc:515] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.964311 11845 consensus_queue.cc:260] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } }
I20260709 13:17:02.965654 11869 leader_election.cc:290] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 election: Requested vote from peers 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:02.966672 11845 ts_tablet_manager.cc:1434] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:17:02.966825 11800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebbcce463d0e4eefb40d760fff058275" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea1829e48e84553bac8093876f65de9"
I20260709 13:17:02.967352 11845 tablet_bootstrap.cc:492] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32: Bootstrap starting.
I20260709 13:17:02.967401 11800 raft_consensus.cc:3060] T ebbcce463d0e4eefb40d760fff058275 P 8ea1829e48e84553bac8093876f65de9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:02.970291 11727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebbcce463d0e4eefb40d760fff058275" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
I20260709 13:17:02.971872 11847 tablet_bootstrap.cc:492] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e: No bootstrap required, opened a new log
W20260709 13:17:02.972083 11616 leader_election.cc:343] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:02.972307 11847 ts_tablet_manager.cc:1403] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e: Time spent bootstrapping tablet: real 0.116s	user 0.020s	sys 0.034s
I20260709 13:17:02.973510 11845 tablet_bootstrap.cc:654] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:02.974339 11800 raft_consensus.cc:2468] T ebbcce463d0e4eefb40d760fff058275 P 8ea1829e48e84553bac8093876f65de9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 1.
I20260709 13:17:02.975005 11847 raft_consensus.cc:359] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.975634 11847 raft_consensus.cc:385] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.975445 11615 leader_election.cc:304] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32, 8ea1829e48e84553bac8093876f65de9; no voters: 928a1fe46e5944c8801f6ff56e1d5a3e
I20260709 13:17:02.975894 11847 raft_consensus.cc:740] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.976224 11853 raft_consensus.cc:2804] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:02.976552 11847 consensus_queue.cc:260] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.978667 11847 ts_tablet_manager.cc:1434] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:17:02.979620 11847 tablet_bootstrap.cc:492] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e: Bootstrap starting.
I20260709 13:17:02.986016 11845 tablet_bootstrap.cc:492] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32: No bootstrap required, opened a new log
I20260709 13:17:02.986596 11845 ts_tablet_manager.cc:1403] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.004s
I20260709 13:17:02.989745 11845 raft_consensus.cc:359] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.990469 11845 raft_consensus.cc:385] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:02.990772 11845 raft_consensus.cc:740] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.991683 11845 consensus_queue.cc:260] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:02.994791 11546 catalog_manager.cc:5697] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ea1829e48e84553bac8093876f65de9 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "8ea1829e48e84553bac8093876f65de9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:03.001947 11845 ts_tablet_manager.cc:1434] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32: Time spent starting tablet: real 0.015s	user 0.015s	sys 0.000s
I20260709 13:17:03.004097 11847 tablet_bootstrap.cc:654] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:03.012117 11548 catalog_manager.cc:5697] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ea1829e48e84553bac8093876f65de9 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "8ea1829e48e84553bac8093876f65de9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:03.013278 11597 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:03.016316 11853 raft_consensus.cc:697] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Becoming Leader. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Running, Role: LEADER
I20260709 13:17:03.017237 11853 consensus_queue.cc:237] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:03.025609 11853 consensus_queue.cc:1048] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:03.030373 11847 tablet_bootstrap.cc:492] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e: No bootstrap required, opened a new log
I20260709 13:17:03.030840 11847 ts_tablet_manager.cc:1403] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e: Time spent bootstrapping tablet: real 0.051s	user 0.024s	sys 0.000s
I20260709 13:17:03.033427 11847 raft_consensus.cc:359] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:03.034152 11847 raft_consensus.cc:385] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:03.034446 11847 raft_consensus.cc:740] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Initialized, Role: FOLLOWER
I20260709 13:17:03.035183 11847 consensus_queue.cc:260] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:03.035005 11546 catalog_manager.cc:5697] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:03.037534 11847 ts_tablet_manager.cc:1434] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:17:03.047039 11863 consensus_queue.cc:1048] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:03.051887 11853 raft_consensus.cc:493] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.052419 11853 raft_consensus.cc:515] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } }
I20260709 13:17:03.054600 11853 leader_election.cc:290] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543), 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361)
I20260709 13:17:03.055927 11727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0749b17931544ef6938fd8a33768d265" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" is_pre_election: true
I20260709 13:17:03.057943 11800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0749b17931544ef6938fd8a33768d265" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea1829e48e84553bac8093876f65de9" is_pre_election: true
I20260709 13:17:03.059887 11615 leader_election.cc:304] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32; no voters: 8ea1829e48e84553bac8093876f65de9, 928a1fe46e5944c8801f6ff56e1d5a3e
I20260709 13:17:03.060897 11853 raft_consensus.cc:2749] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:03.074631 11866 consensus_queue.cc:1048] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:03.081311 11866 consensus_queue.cc:1048] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:03.106887 11651 raft_consensus.cc:3060] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:03.137728 11910 consensus_queue.cc:1048] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:03.148067 11726 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0749b17931544ef6938fd8a33768d265"
dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
mode: GRACEFUL
new_leader_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
 from {username='slave'} at 127.0.0.1:37974
I20260709 13:17:03.158099 11726 raft_consensus.cc:606] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 LEADER]: Received request to transfer leadership to TS 8134a93015124c3ba3a2a7bf5ae8af32
I20260709 13:17:03.201422 11651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5dd080bfdd96482486e355711e04fa83"
dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
mode: GRACEFUL
new_leader_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
 from {username='slave'} at 127.0.0.1:46398
I20260709 13:17:03.202097 11651 raft_consensus.cc:606] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Received request to transfer leadership to TS 928a1fe46e5944c8801f6ff56e1d5a3e
I20260709 13:17:03.217514 11863 raft_consensus.cc:493] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.218427 11799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15aec8d8a49042f9adfc877e482c4907"
dest_uuid: "8ea1829e48e84553bac8093876f65de9"
mode: GRACEFUL
new_leader_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
 from {username='slave'} at 127.0.0.1:46258
I20260709 13:17:03.218724 11863 raft_consensus.cc:515] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:03.219090 11799 raft_consensus.cc:606] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [term 1 LEADER]: Received request to transfer leadership to TS 928a1fe46e5944c8801f6ff56e1d5a3e
I20260709 13:17:03.220970 11863 leader_election.cc:290] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:03.222020 11651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd080bfdd96482486e355711e04fa83" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" is_pre_election: true
I20260709 13:17:03.222673 11799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd080bfdd96482486e355711e04fa83" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea1829e48e84553bac8093876f65de9" is_pre_election: true
I20260709 13:17:03.223521 11799 raft_consensus.cc:2393] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 928a1fe46e5944c8801f6ff56e1d5a3e in current term 1: Already voted for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in this term.
I20260709 13:17:03.224603 11690 leader_election.cc:304] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 928a1fe46e5944c8801f6ff56e1d5a3e; no voters: 8134a93015124c3ba3a2a7bf5ae8af32, 8ea1829e48e84553bac8093876f65de9
I20260709 13:17:03.225446 11863 raft_consensus.cc:3060] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:03.225440 11651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90d25fd4fab44cd4a8e27fc6ce41eb61"
dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
mode: GRACEFUL
new_leader_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
 from {username='slave'} at 127.0.0.1:46398
I20260709 13:17:03.226349 11651 raft_consensus.cc:606] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Received request to transfer leadership to TS 928a1fe46e5944c8801f6ff56e1d5a3e
I20260709 13:17:03.231773 11651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ebbcce463d0e4eefb40d760fff058275"
dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
mode: GRACEFUL
new_leader_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
 from {username='slave'} at 127.0.0.1:46398
I20260709 13:17:03.232288 11651 raft_consensus.cc:606] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Received request to transfer leadership to TS 928a1fe46e5944c8801f6ff56e1d5a3e
I20260709 13:17:03.232327 11863 raft_consensus.cc:2749] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:03.292418 11866 consensus_queue.cc:1048] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:03.293716 11904 consensus_queue.cc:1048] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:03.321460 11904 raft_consensus.cc:493] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.321970 11904 raft_consensus.cc:515] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:03.324244 11904 leader_election.cc:290] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:03.324589 11726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15aec8d8a49042f9adfc877e482c4907" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" is_pre_election: true
I20260709 13:17:03.324890 11799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15aec8d8a49042f9adfc877e482c4907" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea1829e48e84553bac8093876f65de9" is_pre_election: true
I20260709 13:17:03.325405 11726 raft_consensus.cc:2393] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8134a93015124c3ba3a2a7bf5ae8af32 in current term 1: Already voted for candidate 8ea1829e48e84553bac8093876f65de9 in this term.
I20260709 13:17:03.326355 11616 leader_election.cc:304] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32; no voters: 8ea1829e48e84553bac8093876f65de9, 928a1fe46e5944c8801f6ff56e1d5a3e
I20260709 13:17:03.327016 11904 raft_consensus.cc:3060] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:03.331252 11863 raft_consensus.cc:493] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:03.331732 11863 raft_consensus.cc:515] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:03.332125 11904 raft_consensus.cc:2749] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:03.333617 11863 leader_election.cc:290] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:03.334463 11651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebbcce463d0e4eefb40d760fff058275" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" is_pre_election: true
I20260709 13:17:03.334700 11799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebbcce463d0e4eefb40d760fff058275" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ea1829e48e84553bac8093876f65de9" is_pre_election: true
I20260709 13:17:03.335426 11799 raft_consensus.cc:2393] T ebbcce463d0e4eefb40d760fff058275 P 8ea1829e48e84553bac8093876f65de9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 928a1fe46e5944c8801f6ff56e1d5a3e in current term 1: Already voted for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in this term.
I20260709 13:17:03.336310 11690 leader_election.cc:304] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 928a1fe46e5944c8801f6ff56e1d5a3e; no voters: 8134a93015124c3ba3a2a7bf5ae8af32, 8ea1829e48e84553bac8093876f65de9
I20260709 13:17:03.336946 11863 raft_consensus.cc:3060] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:03.343452 11863 raft_consensus.cc:2749] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:03.349242 11885 consensus_queue.cc:1048] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:03.354128 11885 consensus_queue.cc:1048] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:03.354187 11904 consensus_queue.cc:1048] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:03.385737 11726 raft_consensus.cc:3060] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:03.394021 11904 consensus_queue.cc:1048] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:03.396669 11904 consensus_queue.cc:1048] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:03.404699 11885 consensus_queue.cc:1048] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:03.420727 11866 consensus_queue.cc:1048] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:03.434604 11848 raft_consensus.cc:993] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9: : Instructing follower 928a1fe46e5944c8801f6ff56e1d5a3e to start an election
I20260709 13:17:03.435065 11848 raft_consensus.cc:1081] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [term 1 LEADER]: Signalling peer 928a1fe46e5944c8801f6ff56e1d5a3e to start an election
I20260709 13:17:03.438264 11727 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "15aec8d8a49042f9adfc877e482c4907"
dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
 from {username='slave'} at 127.0.0.1:37970
I20260709 13:17:03.438903 11727 raft_consensus.cc:493] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:03.439361 11727 raft_consensus.cc:3060] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:03.440176 11886 consensus_queue.cc:1048] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:03.446832 11727 raft_consensus.cc:515] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:03.449582 11726 raft_consensus.cc:1240] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Rejecting Update request from peer 8ea1829e48e84553bac8093876f65de9 for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:03.451624 11866 raft_consensus.cc:993] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32: : Instructing follower 928a1fe46e5944c8801f6ff56e1d5a3e to start an election
I20260709 13:17:03.452065 11866 raft_consensus.cc:1081] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Signalling peer 928a1fe46e5944c8801f6ff56e1d5a3e to start an election
I20260709 13:17:03.454747 11726 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5dd080bfdd96482486e355711e04fa83"
dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
 from {username='slave'} at 127.0.0.1:37962
I20260709 13:17:03.455334 11726 raft_consensus.cc:493] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:03.455751 11726 raft_consensus.cc:3060] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:03.458873 11885 consensus_queue.cc:1059] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:03.459928 11885 raft_consensus.cc:3055] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:03.460361 11885 raft_consensus.cc:740] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8ea1829e48e84553bac8093876f65de9, State: Running, Role: LEADER
I20260709 13:17:03.465930 11727 leader_election.cc:290] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 2 election: Requested vote from peers 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:03.468588 11799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15aec8d8a49042f9adfc877e482c4907" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8ea1829e48e84553bac8093876f65de9"
I20260709 13:17:03.468588 11651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15aec8d8a49042f9adfc877e482c4907" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
I20260709 13:17:03.469426 11885 consensus_queue.cc:260] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:03.469363 11651 raft_consensus.cc:3060] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:03.469728 11869 consensus_queue.cc:1048] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:03.473903 11726 raft_consensus.cc:515] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:03.477366 11726 leader_election.cc:290] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 2 election: Requested vote from peers 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:03.477491 11801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd080bfdd96482486e355711e04fa83" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8ea1829e48e84553bac8093876f65de9"
I20260709 13:17:03.478139 11801 raft_consensus.cc:3060] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:03.479144 11652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd080bfdd96482486e355711e04fa83" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
I20260709 13:17:03.479720 11652 raft_consensus.cc:3055] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:03.480275 11652 raft_consensus.cc:740] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Running, Role: LEADER
I20260709 13:17:03.480846 11652 consensus_queue.cc:260] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:03.482221 11652 raft_consensus.cc:3060] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:03.487183 11651 raft_consensus.cc:2468] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 928a1fe46e5944c8801f6ff56e1d5a3e in term 2.
I20260709 13:17:03.488411 11866 consensus_queue.cc:1048] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:03.488619 11691 leader_election.cc:304] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32, 928a1fe46e5944c8801f6ff56e1d5a3e; no voters: 
I20260709 13:17:03.489460 11863 raft_consensus.cc:2804] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:03.489962 11863 raft_consensus.cc:697] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Becoming Leader. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Running, Role: LEADER
I20260709 13:17:03.490768 11863 consensus_queue.cc:237] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:03.491525 11652 raft_consensus.cc:2468] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 928a1fe46e5944c8801f6ff56e1d5a3e in term 2.
I20260709 13:17:03.492776 11691 leader_election.cc:304] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32, 928a1fe46e5944c8801f6ff56e1d5a3e; no voters: 
I20260709 13:17:03.493731 11915 raft_consensus.cc:2804] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:03.494199 11915 raft_consensus.cc:697] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Becoming Leader. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Running, Role: LEADER
I20260709 13:17:03.495237 11801 raft_consensus.cc:2468] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 928a1fe46e5944c8801f6ff56e1d5a3e in term 2.
I20260709 13:17:03.495106 11915 consensus_queue.cc:237] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:03.501144 11885 raft_consensus.cc:3060] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:03.514199 11549 catalog_manager.cc:5697] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e reported cstate change: term changed from 1 to 2, leader changed from 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65) to 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66). New cstate: current_term: 2 leader_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:03.515869 11549 catalog_manager.cc:5697] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e reported cstate change: term changed from 1 to 2, leader changed from 8ea1829e48e84553bac8093876f65de9 (127.11.63.67) to 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66). New cstate: current_term: 2 leader_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:03.521695 11869 raft_consensus.cc:993] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32: : Instructing follower 928a1fe46e5944c8801f6ff56e1d5a3e to start an election
I20260709 13:17:03.522156 11869 raft_consensus.cc:1081] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Signalling peer 928a1fe46e5944c8801f6ff56e1d5a3e to start an election
I20260709 13:17:03.523900 11727 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ebbcce463d0e4eefb40d760fff058275"
dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
 from {username='slave'} at 127.0.0.1:37962
I20260709 13:17:03.524477 11727 raft_consensus.cc:493] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:03.524863 11727 raft_consensus.cc:3060] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:03.532446 11727 raft_consensus.cc:515] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:03.535089 11799 raft_consensus.cc:2468] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 928a1fe46e5944c8801f6ff56e1d5a3e in term 2.
I20260709 13:17:03.539077 11727 leader_election.cc:290] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 2 election: Requested vote from peers 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:03.539886 11652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebbcce463d0e4eefb40d760fff058275" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
I20260709 13:17:03.540663 11652 raft_consensus.cc:3055] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:03.540957 11652 raft_consensus.cc:740] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Running, Role: LEADER
I20260709 13:17:03.540067 11799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebbcce463d0e4eefb40d760fff058275" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8ea1829e48e84553bac8093876f65de9"
I20260709 13:17:03.541836 11799 raft_consensus.cc:3060] T ebbcce463d0e4eefb40d760fff058275 P 8ea1829e48e84553bac8093876f65de9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:03.541673 11652 consensus_queue.cc:260] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:03.543220 11652 raft_consensus.cc:3060] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:03.549714 11799 raft_consensus.cc:2468] T ebbcce463d0e4eefb40d760fff058275 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 928a1fe46e5944c8801f6ff56e1d5a3e in term 2.
I20260709 13:17:03.549706 11652 raft_consensus.cc:2468] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 928a1fe46e5944c8801f6ff56e1d5a3e in term 2.
I20260709 13:17:03.550962 11690 leader_election.cc:304] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ea1829e48e84553bac8093876f65de9, 928a1fe46e5944c8801f6ff56e1d5a3e; no voters: 
I20260709 13:17:03.551854 11915 raft_consensus.cc:2804] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:03.552330 11915 raft_consensus.cc:697] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Becoming Leader. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Running, Role: LEADER
I20260709 13:17:03.553121 11915 consensus_queue.cc:237] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:03.562356 11546 catalog_manager.cc:5697] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e reported cstate change: term changed from 1 to 2, leader changed from 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65) to 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66). New cstate: current_term: 2 leader_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:03.587387 11902 raft_consensus.cc:993] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32: : Instructing follower 928a1fe46e5944c8801f6ff56e1d5a3e to start an election
I20260709 13:17:03.587929 11869 raft_consensus.cc:1081] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Signalling peer 928a1fe46e5944c8801f6ff56e1d5a3e to start an election
I20260709 13:17:03.589495 11727 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90d25fd4fab44cd4a8e27fc6ce41eb61"
dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
 from {username='slave'} at 127.0.0.1:37962
I20260709 13:17:03.590194 11727 raft_consensus.cc:493] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:03.590801 11727 raft_consensus.cc:3060] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:03.598372 11727 raft_consensus.cc:515] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:03.600732 11727 leader_election.cc:290] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 2 election: Requested vote from peers 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:03.601475 11652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d25fd4fab44cd4a8e27fc6ce41eb61" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
I20260709 13:17:03.602210 11652 raft_consensus.cc:3055] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:03.602111 11801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d25fd4fab44cd4a8e27fc6ce41eb61" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8ea1829e48e84553bac8093876f65de9"
I20260709 13:17:03.602692 11652 raft_consensus.cc:740] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Running, Role: LEADER
I20260709 13:17:03.602890 11801 raft_consensus.cc:3060] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:03.603725 11652 consensus_queue.cc:260] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:03.604862 11652 raft_consensus.cc:3060] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:03.609720 11652 raft_consensus.cc:2468] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 928a1fe46e5944c8801f6ff56e1d5a3e in term 2.
I20260709 13:17:03.610215 11801 raft_consensus.cc:2468] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 928a1fe46e5944c8801f6ff56e1d5a3e in term 2.
I20260709 13:17:03.610972 11691 leader_election.cc:304] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32, 928a1fe46e5944c8801f6ff56e1d5a3e; no voters: 
I20260709 13:17:03.611833 11915 raft_consensus.cc:2804] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:03.612370 11915 raft_consensus.cc:697] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Becoming Leader. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Running, Role: LEADER
I20260709 13:17:03.613161 11915 consensus_queue.cc:237] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:03.619779 11546 catalog_manager.cc:5697] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e reported cstate change: term changed from 1 to 2, leader changed from 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65) to 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66). New cstate: current_term: 2 leader_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:03.762621 11910 raft_consensus.cc:993] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e: : Instructing follower 8134a93015124c3ba3a2a7bf5ae8af32 to start an election
I20260709 13:17:03.763109 11910 raft_consensus.cc:1081] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 LEADER]: Signalling peer 8134a93015124c3ba3a2a7bf5ae8af32 to start an election
I20260709 13:17:03.764935 11652 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0749b17931544ef6938fd8a33768d265"
dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:17:03.765515 11652 raft_consensus.cc:493] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:03.765904 11652 raft_consensus.cc:3060] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:03.772145 11652 raft_consensus.cc:515] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } }
I20260709 13:17:03.773975 11652 leader_election.cc:290] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 2 election: Requested vote from peers 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543), 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361)
I20260709 13:17:03.775147 11727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0749b17931544ef6938fd8a33768d265" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
I20260709 13:17:03.775207 11801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0749b17931544ef6938fd8a33768d265" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8ea1829e48e84553bac8093876f65de9"
I20260709 13:17:03.775909 11727 raft_consensus.cc:3055] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:03.775965 11801 raft_consensus.cc:3060] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:03.776290 11727 raft_consensus.cc:740] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Running, Role: LEADER
I20260709 13:17:03.777284 11727 consensus_queue.cc:260] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } }
I20260709 13:17:03.778357 11727 raft_consensus.cc:3060] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:03.783558 11801 raft_consensus.cc:2468] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 2.
I20260709 13:17:03.784722 11727 raft_consensus.cc:2468] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 2.
I20260709 13:17:03.784788 11615 leader_election.cc:304] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32, 8ea1829e48e84553bac8093876f65de9; no voters: 
I20260709 13:17:03.785768 11869 raft_consensus.cc:2804] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:03.786262 11869 raft_consensus.cc:697] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 LEADER]: Becoming Leader. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Running, Role: LEADER
I20260709 13:17:03.787091 11869 consensus_queue.cc:237] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } }
I20260709 13:17:03.794670 11546 catalog_manager.cc:5697] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 reported cstate change: term changed from 1 to 2, leader changed from 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66) to 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65). New cstate: current_term: 2 leader_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:03.927179 11652 raft_consensus.cc:1275] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Refusing update from remote peer 928a1fe46e5944c8801f6ff56e1d5a3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:03.929159 11915 consensus_queue.cc:1048] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:03.941471 11801 raft_consensus.cc:1275] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Refusing update from remote peer 928a1fe46e5944c8801f6ff56e1d5a3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:03.944154 11863 consensus_queue.cc:1048] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:03.954360 11652 raft_consensus.cc:1275] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Refusing update from remote peer 928a1fe46e5944c8801f6ff56e1d5a3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:03.955869 11863 consensus_queue.cc:1048] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:03.959306 11651 raft_consensus.cc:1275] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Refusing update from remote peer 928a1fe46e5944c8801f6ff56e1d5a3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:03.960856 11910 consensus_queue.cc:1048] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:03.971500 11801 raft_consensus.cc:1275] T ebbcce463d0e4eefb40d760fff058275 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Refusing update from remote peer 928a1fe46e5944c8801f6ff56e1d5a3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:03.973428 11915 consensus_queue.cc:1048] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:03.976040 11799 raft_consensus.cc:1275] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Refusing update from remote peer 928a1fe46e5944c8801f6ff56e1d5a3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:03.977766 11931 consensus_queue.cc:1048] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:04.006810 11799 raft_consensus.cc:1275] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Refusing update from remote peer 928a1fe46e5944c8801f6ff56e1d5a3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:04.008155 11863 consensus_queue.cc:1048] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:04.021087 11651 raft_consensus.cc:1275] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Refusing update from remote peer 928a1fe46e5944c8801f6ff56e1d5a3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:04.022965 11931 consensus_queue.cc:1048] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:04.183333 11799 raft_consensus.cc:1275] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Refusing update from remote peer 8134a93015124c3ba3a2a7bf5ae8af32: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:04.184643 11869 consensus_queue.cc:1048] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:04.193506 11727 raft_consensus.cc:1275] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Refusing update from remote peer 8134a93015124c3ba3a2a7bf5ae8af32: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:04.195117 11902 consensus_queue.cc:1048] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:06.270815 11517 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:17:06.271612 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:06.289384 11651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0749b17931544ef6938fd8a33768d265"
dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
mode: GRACEFUL
new_leader_uuid: "8ea1829e48e84553bac8093876f65de9"
 from {username='slave'} at 127.0.0.1:46410
I20260709 13:17:06.290005 11651 raft_consensus.cc:606] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 LEADER]: Received request to transfer leadership to TS 8ea1829e48e84553bac8093876f65de9
I20260709 13:17:06.302386 11727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90d25fd4fab44cd4a8e27fc6ce41eb61"
dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
mode: GRACEFUL
new_leader_uuid: "8ea1829e48e84553bac8093876f65de9"
 from {username='slave'} at 127.0.0.1:37976
I20260709 13:17:06.303118 11727 raft_consensus.cc:606] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Received request to transfer leadership to TS 8ea1829e48e84553bac8093876f65de9
I20260709 13:17:06.304330 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:06.304879 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:06.329908 11939 raft_consensus.cc:993] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e: : Instructing follower 8ea1829e48e84553bac8093876f65de9 to start an election
I20260709 13:17:06.330956 11943 raft_consensus.cc:1081] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Signalling peer 8ea1829e48e84553bac8093876f65de9 to start an election
I20260709 13:17:06.332746 11800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90d25fd4fab44cd4a8e27fc6ce41eb61"
dest_uuid: "8ea1829e48e84553bac8093876f65de9"
 from {username='slave'} at 127.0.0.1:46242
I20260709 13:17:06.333403 11800 raft_consensus.cc:493] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:06.333873 11800 raft_consensus.cc:3060] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:06.339524 11800 raft_consensus.cc:515] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:06.341249 11800 leader_election.cc:290] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [CANDIDATE]: Term 3 election: Requested vote from peers 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363), 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361)
I20260709 13:17:06.342262 11651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d25fd4fab44cd4a8e27fc6ce41eb61" candidate_uuid: "8ea1829e48e84553bac8093876f65de9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
I20260709 13:17:06.342926 11651 raft_consensus.cc:3060] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:06.342723 11727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d25fd4fab44cd4a8e27fc6ce41eb61" candidate_uuid: "8ea1829e48e84553bac8093876f65de9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
I20260709 13:17:06.343675 11727 raft_consensus.cc:3055] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:06.344059 11727 raft_consensus.cc:740] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Running, Role: LEADER
I20260709 13:17:06.344861 11727 consensus_queue.cc:260] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:06.345870 11727 raft_consensus.cc:3060] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:06.348272 11651 raft_consensus.cc:2468] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ea1829e48e84553bac8093876f65de9 in term 3.
I20260709 13:17:06.349843 11765 leader_election.cc:304] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32, 8ea1829e48e84553bac8093876f65de9; no voters: 
I20260709 13:17:06.350677 11938 raft_consensus.cc:2804] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:06.350792 11727 raft_consensus.cc:2468] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ea1829e48e84553bac8093876f65de9 in term 3.
I20260709 13:17:06.351922 11938 raft_consensus.cc:697] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 3 LEADER]: Becoming Leader. State: Replica: 8ea1829e48e84553bac8093876f65de9, State: Running, Role: LEADER
I20260709 13:17:06.352680 11938 consensus_queue.cc:237] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:06.360947 11546 catalog_manager.cc:5697] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 reported cstate change: term changed from 2 to 3, leader changed from 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66) to 8ea1829e48e84553bac8093876f65de9 (127.11.63.67). New cstate: current_term: 3 leader_uuid: "8ea1829e48e84553bac8093876f65de9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:06.502735 11937 raft_consensus.cc:993] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32: : Instructing follower 8ea1829e48e84553bac8093876f65de9 to start an election
I20260709 13:17:06.503252 11937 raft_consensus.cc:1081] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 LEADER]: Signalling peer 8ea1829e48e84553bac8093876f65de9 to start an election
I20260709 13:17:06.504908 11800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0749b17931544ef6938fd8a33768d265"
dest_uuid: "8ea1829e48e84553bac8093876f65de9"
 from {username='slave'} at 127.0.0.1:46236
I20260709 13:17:06.505465 11800 raft_consensus.cc:493] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:06.505802 11800 raft_consensus.cc:3060] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:06.510756 11800 raft_consensus.cc:515] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } }
I20260709 13:17:06.512271 11800 leader_election.cc:290] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [CANDIDATE]: Term 3 election: Requested vote from peers 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361), 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363)
I20260709 13:17:06.513257 11727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0749b17931544ef6938fd8a33768d265" candidate_uuid: "8ea1829e48e84553bac8093876f65de9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
I20260709 13:17:06.513587 11651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0749b17931544ef6938fd8a33768d265" candidate_uuid: "8ea1829e48e84553bac8093876f65de9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
I20260709 13:17:06.513942 11727 raft_consensus.cc:3060] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:06.514142 11651 raft_consensus.cc:3055] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:06.514532 11651 raft_consensus.cc:740] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Running, Role: LEADER
I20260709 13:17:06.515416 11651 consensus_queue.cc:260] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } }
I20260709 13:17:06.516776 11651 raft_consensus.cc:3060] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:06.521334 11727 raft_consensus.cc:2468] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ea1829e48e84553bac8093876f65de9 in term 3.
I20260709 13:17:06.522635 11765 leader_election.cc:304] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ea1829e48e84553bac8093876f65de9, 928a1fe46e5944c8801f6ff56e1d5a3e; no voters: 
I20260709 13:17:06.523511 11938 raft_consensus.cc:2804] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:06.523905 11938 raft_consensus.cc:697] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 3 LEADER]: Becoming Leader. State: Replica: 8ea1829e48e84553bac8093876f65de9, State: Running, Role: LEADER
I20260709 13:17:06.524076 11651 raft_consensus.cc:2468] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ea1829e48e84553bac8093876f65de9 in term 3.
I20260709 13:17:06.524670 11938 consensus_queue.cc:237] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } }
I20260709 13:17:06.531939 11546 catalog_manager.cc:5697] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 reported cstate change: term changed from 2 to 3, leader changed from 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65) to 8ea1829e48e84553bac8093876f65de9 (127.11.63.67). New cstate: current_term: 3 leader_uuid: "8ea1829e48e84553bac8093876f65de9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:06.908983 11727 raft_consensus.cc:1275] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 3 FOLLOWER]: Refusing update from remote peer 8ea1829e48e84553bac8093876f65de9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:06.910470 11938 consensus_queue.cc:1048] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:06.921211 11651 raft_consensus.cc:1275] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 3 FOLLOWER]: Refusing update from remote peer 8ea1829e48e84553bac8093876f65de9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:06.923055 11945 consensus_queue.cc:1048] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:06.940828 11651 raft_consensus.cc:1275] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 3 FOLLOWER]: Refusing update from remote peer 8ea1829e48e84553bac8093876f65de9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:06.942310 11938 consensus_queue.cc:1048] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:06.948482 11727 raft_consensus.cc:1275] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 3 FOLLOWER]: Refusing update from remote peer 8ea1829e48e84553bac8093876f65de9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:06.950024 11945 consensus_queue.cc:1048] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:07.305742 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:07.316398 11799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0749b17931544ef6938fd8a33768d265"
dest_uuid: "8ea1829e48e84553bac8093876f65de9"
mode: GRACEFUL
new_leader_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
 from {username='slave'} at 127.0.0.1:46258
I20260709 13:17:07.316897 11799 raft_consensus.cc:606] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 3 LEADER]: Received request to transfer leadership to TS 928a1fe46e5944c8801f6ff56e1d5a3e
I20260709 13:17:07.322475 11651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "112100a2ecf0402f8a4f2fc0571a6a21"
dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
mode: GRACEFUL
new_leader_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
 from {username='slave'} at 127.0.0.1:46398
I20260709 13:17:07.323004 11651 raft_consensus.cc:606] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Received request to transfer leadership to TS 928a1fe46e5944c8801f6ff56e1d5a3e
I20260709 13:17:07.328487 11651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85e1a267c28d4122b8e8cd93d34e9c50"
dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
mode: GRACEFUL
new_leader_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
 from {username='slave'} at 127.0.0.1:46398
I20260709 13:17:07.329002 11651 raft_consensus.cc:606] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Received request to transfer leadership to TS 928a1fe46e5944c8801f6ff56e1d5a3e
I20260709 13:17:07.334751 11651 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e41561d6cff44c26a93018d0f521697a"
dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
mode: GRACEFUL
new_leader_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
 from {username='slave'} at 127.0.0.1:46398
I20260709 13:17:07.335434 11651 raft_consensus.cc:606] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Received request to transfer leadership to TS 928a1fe46e5944c8801f6ff56e1d5a3e
I20260709 13:17:07.347229 11799 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90d25fd4fab44cd4a8e27fc6ce41eb61"
dest_uuid: "8ea1829e48e84553bac8093876f65de9"
mode: GRACEFUL
new_leader_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
 from {username='slave'} at 127.0.0.1:46258
I20260709 13:17:07.347779 11799 raft_consensus.cc:606] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 3 LEADER]: Received request to transfer leadership to TS 928a1fe46e5944c8801f6ff56e1d5a3e
I20260709 13:17:07.356120 11945 raft_consensus.cc:993] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9: : Instructing follower 928a1fe46e5944c8801f6ff56e1d5a3e to start an election
I20260709 13:17:07.356544 11955 raft_consensus.cc:1081] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 3 LEADER]: Signalling peer 928a1fe46e5944c8801f6ff56e1d5a3e to start an election
I20260709 13:17:07.358073 11727 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90d25fd4fab44cd4a8e27fc6ce41eb61"
dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
 from {username='slave'} at 127.0.0.1:37970
I20260709 13:17:07.358525 11727 raft_consensus.cc:493] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:07.358819 11727 raft_consensus.cc:3060] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:07.364619 11727 raft_consensus.cc:515] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:07.366207 11727 leader_election.cc:290] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 4 election: Requested vote from peers 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:07.367385 11799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d25fd4fab44cd4a8e27fc6ce41eb61" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8ea1829e48e84553bac8093876f65de9"
I20260709 13:17:07.367378 11651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d25fd4fab44cd4a8e27fc6ce41eb61" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
I20260709 13:17:07.367960 11799 raft_consensus.cc:3055] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:07.368137 11651 raft_consensus.cc:3060] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:07.368269 11799 raft_consensus.cc:740] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8ea1829e48e84553bac8093876f65de9, State: Running, Role: LEADER
I20260709 13:17:07.369067 11799 consensus_queue.cc:260] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:07.370194 11799 raft_consensus.cc:3060] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:07.373013 11651 raft_consensus.cc:2468] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 928a1fe46e5944c8801f6ff56e1d5a3e in term 4.
I20260709 13:17:07.374359 11691 leader_election.cc:304] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32, 928a1fe46e5944c8801f6ff56e1d5a3e; no voters: 
I20260709 13:17:07.375092 11943 raft_consensus.cc:2804] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:07.375217 11799 raft_consensus.cc:2468] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 928a1fe46e5944c8801f6ff56e1d5a3e in term 4.
I20260709 13:17:07.376223 11943 raft_consensus.cc:697] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 4 LEADER]: Becoming Leader. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Running, Role: LEADER
I20260709 13:17:07.376989 11943 consensus_queue.cc:237] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:07.383435 11548 catalog_manager.cc:5697] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e reported cstate change: term changed from 3 to 4, leader changed from 8ea1829e48e84553bac8093876f65de9 (127.11.63.67) to 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66). New cstate: current_term: 4 leader_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:07.424297 11938 raft_consensus.cc:993] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9: : Instructing follower 928a1fe46e5944c8801f6ff56e1d5a3e to start an election
I20260709 13:17:07.424787 11955 raft_consensus.cc:1081] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 3 LEADER]: Signalling peer 928a1fe46e5944c8801f6ff56e1d5a3e to start an election
I20260709 13:17:07.426247 11727 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0749b17931544ef6938fd8a33768d265"
dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
 from {username='slave'} at 127.0.0.1:37970
I20260709 13:17:07.426713 11727 raft_consensus.cc:493] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:07.427135 11727 raft_consensus.cc:3060] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:07.431216 11727 raft_consensus.cc:515] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } }
I20260709 13:17:07.432727 11727 leader_election.cc:290] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 4 election: Requested vote from peers 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543), 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363)
I20260709 13:17:07.433887 11651 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0749b17931544ef6938fd8a33768d265" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
I20260709 13:17:07.433897 11799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0749b17931544ef6938fd8a33768d265" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8ea1829e48e84553bac8093876f65de9"
I20260709 13:17:07.434454 11651 raft_consensus.cc:3060] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:07.434659 11799 raft_consensus.cc:3055] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:07.435027 11799 raft_consensus.cc:740] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8ea1829e48e84553bac8093876f65de9, State: Running, Role: LEADER
I20260709 13:17:07.435760 11799 consensus_queue.cc:260] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } }
I20260709 13:17:07.436739 11799 raft_consensus.cc:3060] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:07.439178 11651 raft_consensus.cc:2468] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 928a1fe46e5944c8801f6ff56e1d5a3e in term 4.
I20260709 13:17:07.440208 11691 leader_election.cc:304] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32, 928a1fe46e5944c8801f6ff56e1d5a3e; no voters: 
I20260709 13:17:07.441082 11943 raft_consensus.cc:2804] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:07.441517 11943 raft_consensus.cc:697] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 4 LEADER]: Becoming Leader. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Running, Role: LEADER
I20260709 13:17:07.441813 11799 raft_consensus.cc:2468] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 928a1fe46e5944c8801f6ff56e1d5a3e in term 4.
I20260709 13:17:07.442323 11943 consensus_queue.cc:237] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } }
I20260709 13:17:07.446149 11965 raft_consensus.cc:993] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32: : Instructing follower 928a1fe46e5944c8801f6ff56e1d5a3e to start an election
I20260709 13:17:07.446553 11951 raft_consensus.cc:1081] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Signalling peer 928a1fe46e5944c8801f6ff56e1d5a3e to start an election
I20260709 13:17:07.448879 11727 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "85e1a267c28d4122b8e8cd93d34e9c50"
dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
 from {username='slave'} at 127.0.0.1:37962
I20260709 13:17:07.449378 11727 raft_consensus.cc:493] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:07.449810 11727 raft_consensus.cc:3060] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:07.450206 11548 catalog_manager.cc:5697] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e reported cstate change: term changed from 3 to 4, leader changed from 8ea1829e48e84553bac8093876f65de9 (127.11.63.67) to 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66). New cstate: current_term: 4 leader_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:07.455940 11727 raft_consensus.cc:515] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:07.459184 11652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85e1a267c28d4122b8e8cd93d34e9c50" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
I20260709 13:17:07.459575 11727 leader_election.cc:290] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 2 election: Requested vote from peers 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:07.460835 11799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85e1a267c28d4122b8e8cd93d34e9c50" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8ea1829e48e84553bac8093876f65de9"
I20260709 13:17:07.460776 11652 raft_consensus.cc:3055] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:07.461503 11799 raft_consensus.cc:3060] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:07.461627 11652 raft_consensus.cc:740] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Running, Role: LEADER
I20260709 13:17:07.462935 11652 consensus_queue.cc:260] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:07.464314 11652 raft_consensus.cc:3060] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:07.467908 11799 raft_consensus.cc:2468] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 928a1fe46e5944c8801f6ff56e1d5a3e in term 2.
I20260709 13:17:07.468911 11690 leader_election.cc:304] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ea1829e48e84553bac8093876f65de9, 928a1fe46e5944c8801f6ff56e1d5a3e; no voters: 
I20260709 13:17:07.469800 11943 raft_consensus.cc:2804] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:07.470474 11943 raft_consensus.cc:697] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Becoming Leader. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Running, Role: LEADER
I20260709 13:17:07.470604 11652 raft_consensus.cc:2468] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 928a1fe46e5944c8801f6ff56e1d5a3e in term 2.
I20260709 13:17:07.471302 11943 consensus_queue.cc:237] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:07.482640 11548 catalog_manager.cc:5697] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e reported cstate change: term changed from 1 to 2, leader changed from 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65) to 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66). New cstate: current_term: 2 leader_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:07.540856 11965 raft_consensus.cc:993] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32: : Instructing follower 928a1fe46e5944c8801f6ff56e1d5a3e to start an election
I20260709 13:17:07.541296 11951 raft_consensus.cc:1081] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Signalling peer 928a1fe46e5944c8801f6ff56e1d5a3e to start an election
I20260709 13:17:07.542766 11727 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "112100a2ecf0402f8a4f2fc0571a6a21"
dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
 from {username='slave'} at 127.0.0.1:37962
I20260709 13:17:07.543229 11727 raft_consensus.cc:493] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:07.543576 11727 raft_consensus.cc:3060] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:07.547533 11727 raft_consensus.cc:515] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:07.549088 11727 leader_election.cc:290] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 2 election: Requested vote from peers 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:07.549933 11652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "112100a2ecf0402f8a4f2fc0571a6a21" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
I20260709 13:17:07.550271 11800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "112100a2ecf0402f8a4f2fc0571a6a21" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8ea1829e48e84553bac8093876f65de9"
I20260709 13:17:07.550554 11652 raft_consensus.cc:3055] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:07.550922 11800 raft_consensus.cc:3060] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8ea1829e48e84553bac8093876f65de9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:07.550951 11652 raft_consensus.cc:740] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Running, Role: LEADER
I20260709 13:17:07.551846 11652 consensus_queue.cc:260] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:07.552668 11652 raft_consensus.cc:3060] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:07.556524 11800 raft_consensus.cc:2468] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 928a1fe46e5944c8801f6ff56e1d5a3e in term 2.
I20260709 13:17:07.557686 11652 raft_consensus.cc:2468] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 928a1fe46e5944c8801f6ff56e1d5a3e in term 2.
I20260709 13:17:07.557652 11690 leader_election.cc:304] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ea1829e48e84553bac8093876f65de9, 928a1fe46e5944c8801f6ff56e1d5a3e; no voters: 
I20260709 13:17:07.558655 11943 raft_consensus.cc:2804] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:07.559056 11943 raft_consensus.cc:697] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Becoming Leader. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Running, Role: LEADER
I20260709 13:17:07.559777 11943 consensus_queue.cc:237] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:07.566025 11548 catalog_manager.cc:5697] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e reported cstate change: term changed from 1 to 2, leader changed from 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65) to 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66). New cstate: current_term: 2 leader_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:07.669961 11965 raft_consensus.cc:993] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32: : Instructing follower 928a1fe46e5944c8801f6ff56e1d5a3e to start an election
I20260709 13:17:07.670325 11965 raft_consensus.cc:1081] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Signalling peer 928a1fe46e5944c8801f6ff56e1d5a3e to start an election
I20260709 13:17:07.671947 11727 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e41561d6cff44c26a93018d0f521697a"
dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
 from {username='slave'} at 127.0.0.1:37962
I20260709 13:17:07.672444 11727 raft_consensus.cc:493] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:07.672709 11727 raft_consensus.cc:3060] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:07.676877 11727 raft_consensus.cc:515] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:07.678344 11727 leader_election.cc:290] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 2 election: Requested vote from peers 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:07.679296 11652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41561d6cff44c26a93018d0f521697a" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
I20260709 13:17:07.679486 11800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e41561d6cff44c26a93018d0f521697a" candidate_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8ea1829e48e84553bac8093876f65de9"
I20260709 13:17:07.679908 11652 raft_consensus.cc:3055] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:07.680013 11800 raft_consensus.cc:3060] T e41561d6cff44c26a93018d0f521697a P 8ea1829e48e84553bac8093876f65de9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:07.680222 11652 raft_consensus.cc:740] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Running, Role: LEADER
I20260709 13:17:07.680943 11652 consensus_queue.cc:260] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:07.681865 11652 raft_consensus.cc:3060] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:07.685278 11800 raft_consensus.cc:2468] T e41561d6cff44c26a93018d0f521697a P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 928a1fe46e5944c8801f6ff56e1d5a3e in term 2.
I20260709 13:17:07.686338 11690 leader_election.cc:304] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ea1829e48e84553bac8093876f65de9, 928a1fe46e5944c8801f6ff56e1d5a3e; no voters: 
I20260709 13:17:07.687093 11943 raft_consensus.cc:2804] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:07.686846 11652 raft_consensus.cc:2468] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 928a1fe46e5944c8801f6ff56e1d5a3e in term 2.
I20260709 13:17:07.687743 11943 raft_consensus.cc:697] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Becoming Leader. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Running, Role: LEADER
I20260709 13:17:07.688766 11943 consensus_queue.cc:237] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:07.696264 11548 catalog_manager.cc:5697] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e reported cstate change: term changed from 1 to 2, leader changed from 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65) to 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66). New cstate: current_term: 2 leader_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:07.826489 11800 raft_consensus.cc:1275] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 4 FOLLOWER]: Refusing update from remote peer 928a1fe46e5944c8801f6ff56e1d5a3e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:07.827948 11943 consensus_queue.cc:1048] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:07.838130 11651 raft_consensus.cc:1275] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 4 FOLLOWER]: Refusing update from remote peer 928a1fe46e5944c8801f6ff56e1d5a3e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:07.839987 11971 consensus_queue.cc:1048] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:07.943675 11800 raft_consensus.cc:1275] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Refusing update from remote peer 928a1fe46e5944c8801f6ff56e1d5a3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:07.945007 11971 consensus_queue.cc:1048] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:07.953753 11651 raft_consensus.cc:1275] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Refusing update from remote peer 928a1fe46e5944c8801f6ff56e1d5a3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:07.955845 11961 consensus_queue.cc:1048] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:07.967159 11651 raft_consensus.cc:1275] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 4 FOLLOWER]: Refusing update from remote peer 928a1fe46e5944c8801f6ff56e1d5a3e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:07.968525 11961 consensus_queue.cc:1048] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:07.977046 11800 raft_consensus.cc:1275] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 4 FOLLOWER]: Refusing update from remote peer 928a1fe46e5944c8801f6ff56e1d5a3e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:07.978467 11961 consensus_queue.cc:1048] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:08.080134 11651 raft_consensus.cc:1275] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Refusing update from remote peer 928a1fe46e5944c8801f6ff56e1d5a3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:08.080686 11652 raft_consensus.cc:1275] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Refusing update from remote peer 928a1fe46e5944c8801f6ff56e1d5a3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:08.081688 11961 consensus_queue.cc:1048] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:08.082151 11943 consensus_queue.cc:1048] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:08.090659 11800 raft_consensus.cc:1275] T e41561d6cff44c26a93018d0f521697a P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Refusing update from remote peer 928a1fe46e5944c8801f6ff56e1d5a3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:08.096457 11979 consensus_queue.cc:1048] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:08.098898 11800 raft_consensus.cc:1275] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Refusing update from remote peer 928a1fe46e5944c8801f6ff56e1d5a3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:08.101094 11943 consensus_queue.cc:1048] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:10.355695 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:10.363646 11727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0749b17931544ef6938fd8a33768d265"
dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
mode: GRACEFUL
new_leader_uuid: "8ea1829e48e84553bac8093876f65de9"
 from {username='slave'} at 127.0.0.1:37976
I20260709 13:17:10.364473 11727 raft_consensus.cc:606] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 4 LEADER]: Received request to transfer leadership to TS 8ea1829e48e84553bac8093876f65de9
I20260709 13:17:10.367681 11727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "112100a2ecf0402f8a4f2fc0571a6a21"
dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
mode: GRACEFUL
new_leader_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
 from {username='slave'} at 127.0.0.1:37976
I20260709 13:17:10.368165 11727 raft_consensus.cc:606] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Received request to transfer leadership to TS 8134a93015124c3ba3a2a7bf5ae8af32
I20260709 13:17:10.370849 11727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15aec8d8a49042f9adfc877e482c4907"
dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
mode: GRACEFUL
new_leader_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
 from {username='slave'} at 127.0.0.1:37976
I20260709 13:17:10.371325 11727 raft_consensus.cc:606] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Received request to transfer leadership to TS 8134a93015124c3ba3a2a7bf5ae8af32
I20260709 13:17:10.373651 11727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85e1a267c28d4122b8e8cd93d34e9c50"
dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
mode: GRACEFUL
new_leader_uuid: "8ea1829e48e84553bac8093876f65de9"
 from {username='slave'} at 127.0.0.1:37976
I20260709 13:17:10.374063 11727 raft_consensus.cc:606] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Received request to transfer leadership to TS 8ea1829e48e84553bac8093876f65de9
I20260709 13:17:10.376422 11727 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90d25fd4fab44cd4a8e27fc6ce41eb61"
dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
mode: GRACEFUL
new_leader_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
 from {username='slave'} at 127.0.0.1:37976
I20260709 13:17:10.376914 11727 raft_consensus.cc:606] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 4 LEADER]: Received request to transfer leadership to TS 8134a93015124c3ba3a2a7bf5ae8af32
I20260709 13:17:10.378000 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:17:10.378427 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:10.423826 11943 raft_consensus.cc:993] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e: : Instructing follower 8134a93015124c3ba3a2a7bf5ae8af32 to start an election
I20260709 13:17:10.424335 11985 raft_consensus.cc:1081] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 4 LEADER]: Signalling peer 8134a93015124c3ba3a2a7bf5ae8af32 to start an election
I20260709 13:17:10.425813 11652 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90d25fd4fab44cd4a8e27fc6ce41eb61"
dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:17:10.426316 11652 raft_consensus.cc:493] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:10.426681 11652 raft_consensus.cc:3060] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:10.432307 11652 raft_consensus.cc:515] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:10.434343 11652 leader_election.cc:290] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 5 election: Requested vote from peers 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:10.435899 11800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d25fd4fab44cd4a8e27fc6ce41eb61" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "8ea1829e48e84553bac8093876f65de9"
I20260709 13:17:10.436136 11727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d25fd4fab44cd4a8e27fc6ce41eb61" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
I20260709 13:17:10.436713 11800 raft_consensus.cc:3060] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:10.436872 11727 raft_consensus.cc:3055] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:10.437206 11727 raft_consensus.cc:740] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Running, Role: LEADER
I20260709 13:17:10.438766 11727 consensus_queue.cc:260] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:10.439745 11727 raft_consensus.cc:3060] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:10.442160 11800 raft_consensus.cc:2468] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 5.
I20260709 13:17:10.443396 11615 leader_election.cc:304] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32, 8ea1829e48e84553bac8093876f65de9; no voters: 
I20260709 13:17:10.445138 11986 raft_consensus.cc:2804] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:10.445331 11727 raft_consensus.cc:2468] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 5.
I20260709 13:17:10.446234 11986 raft_consensus.cc:697] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 5 LEADER]: Becoming Leader. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Running, Role: LEADER
I20260709 13:17:10.447292 11986 consensus_queue.cc:237] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:10.454294 11548 catalog_manager.cc:5697] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 reported cstate change: term changed from 4 to 5, leader changed from 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66) to 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65). New cstate: current_term: 5 leader_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:10.456974 11985 raft_consensus.cc:993] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e: : Instructing follower 8134a93015124c3ba3a2a7bf5ae8af32 to start an election
I20260709 13:17:10.457468 11943 raft_consensus.cc:1081] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Signalling peer 8134a93015124c3ba3a2a7bf5ae8af32 to start an election
I20260709 13:17:10.459371 11652 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "112100a2ecf0402f8a4f2fc0571a6a21"
dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:17:10.459964 11652 raft_consensus.cc:493] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:10.460325 11652 raft_consensus.cc:3060] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:10.461863 11943 raft_consensus.cc:993] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e: : Instructing follower 8ea1829e48e84553bac8093876f65de9 to start an election
I20260709 13:17:10.462298 11943 raft_consensus.cc:1081] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 4 LEADER]: Signalling peer 8ea1829e48e84553bac8093876f65de9 to start an election
I20260709 13:17:10.464188 11800 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0749b17931544ef6938fd8a33768d265"
dest_uuid: "8ea1829e48e84553bac8093876f65de9"
 from {username='slave'} at 127.0.0.1:46242
I20260709 13:17:10.464731 11800 raft_consensus.cc:493] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:10.465044 11800 raft_consensus.cc:3060] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:10.467514 11652 raft_consensus.cc:515] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:10.469457 11652 leader_election.cc:290] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 3 election: Requested vote from peers 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:10.470219 11727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "112100a2ecf0402f8a4f2fc0571a6a21" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
I20260709 13:17:10.470589 11799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "112100a2ecf0402f8a4f2fc0571a6a21" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8ea1829e48e84553bac8093876f65de9"
I20260709 13:17:10.470752 11727 raft_consensus.cc:3055] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:10.471215 11727 raft_consensus.cc:740] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Running, Role: LEADER
I20260709 13:17:10.471225 11799 raft_consensus.cc:3060] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:10.471092 11800 raft_consensus.cc:515] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } }
I20260709 13:17:10.472141 11727 consensus_queue.cc:260] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:10.473155 11727 raft_consensus.cc:3060] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:10.474241 11800 leader_election.cc:290] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [CANDIDATE]: Term 5 election: Requested vote from peers 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361), 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363)
I20260709 13:17:10.474171 11726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0749b17931544ef6938fd8a33768d265" candidate_uuid: "8ea1829e48e84553bac8093876f65de9" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
I20260709 13:17:10.475744 11652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0749b17931544ef6938fd8a33768d265" candidate_uuid: "8ea1829e48e84553bac8093876f65de9" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
I20260709 13:17:10.475792 11726 raft_consensus.cc:3055] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:10.476259 11726 raft_consensus.cc:740] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Running, Role: LEADER
I20260709 13:17:10.477131 11726 consensus_queue.cc:260] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } }
I20260709 13:17:10.478164 11799 raft_consensus.cc:2468] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8ea1829e48e84553bac8093876f65de9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 3.
I20260709 13:17:10.478377 11726 raft_consensus.cc:3060] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:10.479110 11615 leader_election.cc:304] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32, 8ea1829e48e84553bac8093876f65de9; no voters: 
I20260709 13:17:10.479599 11727 raft_consensus.cc:2468] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 3.
I20260709 13:17:10.479786 11986 raft_consensus.cc:2804] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:10.480590 11986 raft_consensus.cc:697] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 3 LEADER]: Becoming Leader. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Running, Role: LEADER
I20260709 13:17:10.480628 11652 raft_consensus.cc:3060] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:10.481616 11986 consensus_queue.cc:237] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:10.485467 11726 raft_consensus.cc:2468] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ea1829e48e84553bac8093876f65de9 in term 5.
I20260709 13:17:10.486663 11765 leader_election.cc:304] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ea1829e48e84553bac8093876f65de9, 928a1fe46e5944c8801f6ff56e1d5a3e; no voters: 
I20260709 13:17:10.487208 11652 raft_consensus.cc:2468] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ea1829e48e84553bac8093876f65de9 in term 5.
I20260709 13:17:10.488296 11983 raft_consensus.cc:2804] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:10.489681 11983 raft_consensus.cc:697] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 5 LEADER]: Becoming Leader. State: Replica: 8ea1829e48e84553bac8093876f65de9, State: Running, Role: LEADER
I20260709 13:17:10.489831 11548 catalog_manager.cc:5697] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 reported cstate change: term changed from 2 to 3, leader changed from 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66) to 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65). New cstate: current_term: 3 leader_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:10.490518 11983 consensus_queue.cc:237] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } }
I20260709 13:17:10.499529 11549 catalog_manager.cc:5697] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 reported cstate change: term changed from 4 to 5, leader changed from 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66) to 8ea1829e48e84553bac8093876f65de9 (127.11.63.67). New cstate: current_term: 5 leader_uuid: "8ea1829e48e84553bac8093876f65de9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:10.597364 11943 raft_consensus.cc:993] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e: : Instructing follower 8ea1829e48e84553bac8093876f65de9 to start an election
I20260709 13:17:10.597972 11943 raft_consensus.cc:1081] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Signalling peer 8ea1829e48e84553bac8093876f65de9 to start an election
I20260709 13:17:10.599468 11799 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "85e1a267c28d4122b8e8cd93d34e9c50"
dest_uuid: "8ea1829e48e84553bac8093876f65de9"
 from {username='slave'} at 127.0.0.1:46242
I20260709 13:17:10.600080 11799 raft_consensus.cc:493] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:10.600438 11799 raft_consensus.cc:3060] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:10.605698 11799 raft_consensus.cc:515] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:10.607587 11799 leader_election.cc:290] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [CANDIDATE]: Term 3 election: Requested vote from peers 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363), 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361)
I20260709 13:17:10.608392 11652 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85e1a267c28d4122b8e8cd93d34e9c50" candidate_uuid: "8ea1829e48e84553bac8093876f65de9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
I20260709 13:17:10.608814 11726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85e1a267c28d4122b8e8cd93d34e9c50" candidate_uuid: "8ea1829e48e84553bac8093876f65de9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
I20260709 13:17:10.608999 11652 raft_consensus.cc:3060] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:10.609414 11726 raft_consensus.cc:3055] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:10.609798 11726 raft_consensus.cc:740] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Running, Role: LEADER
I20260709 13:17:10.610519 11726 consensus_queue.cc:260] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:10.611483 11726 raft_consensus.cc:3060] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:10.613449 11652 raft_consensus.cc:2468] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ea1829e48e84553bac8093876f65de9 in term 3.
I20260709 13:17:10.614589 11765 leader_election.cc:304] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32, 8ea1829e48e84553bac8093876f65de9; no voters: 
I20260709 13:17:10.615453 11983 raft_consensus.cc:2804] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:10.615911 11983 raft_consensus.cc:697] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [term 3 LEADER]: Becoming Leader. State: Replica: 8ea1829e48e84553bac8093876f65de9, State: Running, Role: LEADER
I20260709 13:17:10.616653 11983 consensus_queue.cc:237] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:10.617063 11726 raft_consensus.cc:2468] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ea1829e48e84553bac8093876f65de9 in term 3.
I20260709 13:17:10.623417 11549 catalog_manager.cc:5697] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 reported cstate change: term changed from 2 to 3, leader changed from 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66) to 8ea1829e48e84553bac8093876f65de9 (127.11.63.67). New cstate: current_term: 3 leader_uuid: "8ea1829e48e84553bac8093876f65de9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:10.708369 11985 raft_consensus.cc:993] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e: : Instructing follower 8134a93015124c3ba3a2a7bf5ae8af32 to start an election
I20260709 13:17:10.708882 11943 raft_consensus.cc:1081] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Signalling peer 8134a93015124c3ba3a2a7bf5ae8af32 to start an election
I20260709 13:17:10.710431 11652 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "15aec8d8a49042f9adfc877e482c4907"
dest_uuid: "8134a93015124c3ba3a2a7bf5ae8af32"
 from {username='slave'} at 127.0.0.1:46374
I20260709 13:17:10.710850 11652 raft_consensus.cc:493] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:10.711097 11652 raft_consensus.cc:3060] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:10.715116 11652 raft_consensus.cc:515] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:10.716657 11652 leader_election.cc:290] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 3 election: Requested vote from peers 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361), 8ea1829e48e84553bac8093876f65de9 (127.11.63.67:46543)
I20260709 13:17:10.717463 11726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15aec8d8a49042f9adfc877e482c4907" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e"
I20260709 13:17:10.717751 11799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15aec8d8a49042f9adfc877e482c4907" candidate_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8ea1829e48e84553bac8093876f65de9"
I20260709 13:17:10.718065 11726 raft_consensus.cc:3055] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:10.718230 11799 raft_consensus.cc:3060] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:10.718400 11726 raft_consensus.cc:740] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 928a1fe46e5944c8801f6ff56e1d5a3e, State: Running, Role: LEADER
I20260709 13:17:10.719199 11726 consensus_queue.cc:260] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:10.720085 11726 raft_consensus.cc:3060] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:10.722435 11799 raft_consensus.cc:2468] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 3.
I20260709 13:17:10.723418 11615 leader_election.cc:304] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8134a93015124c3ba3a2a7bf5ae8af32, 8ea1829e48e84553bac8093876f65de9; no voters: 
I20260709 13:17:10.724202 11986 raft_consensus.cc:2804] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:10.724673 11986 raft_consensus.cc:697] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 3 LEADER]: Becoming Leader. State: Replica: 8134a93015124c3ba3a2a7bf5ae8af32, State: Running, Role: LEADER
I20260709 13:17:10.725102 11726 raft_consensus.cc:2468] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8134a93015124c3ba3a2a7bf5ae8af32 in term 3.
I20260709 13:17:10.725493 11986 consensus_queue.cc:237] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } }
I20260709 13:17:10.732888 11549 catalog_manager.cc:5697] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 reported cstate change: term changed from 2 to 3, leader changed from 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66) to 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65). New cstate: current_term: 3 leader_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:10.843760 11799 raft_consensus.cc:1275] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 5 FOLLOWER]: Refusing update from remote peer 8134a93015124c3ba3a2a7bf5ae8af32: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:17:10.845239 11986 consensus_queue.cc:1048] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:10.857190 11726 raft_consensus.cc:1275] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 5 FOLLOWER]: Refusing update from remote peer 8134a93015124c3ba3a2a7bf5ae8af32: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:17:10.859643 11987 consensus_queue.cc:1048] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:10.873265 11799 raft_consensus.cc:1275] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8ea1829e48e84553bac8093876f65de9 [term 3 FOLLOWER]: Refusing update from remote peer 8134a93015124c3ba3a2a7bf5ae8af32: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:10.874439 11987 consensus_queue.cc:1048] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:10.883581 11726 raft_consensus.cc:1275] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 3 FOLLOWER]: Refusing update from remote peer 8134a93015124c3ba3a2a7bf5ae8af32: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:10.884797 11652 raft_consensus.cc:1275] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 5 FOLLOWER]: Refusing update from remote peer 8ea1829e48e84553bac8093876f65de9: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:17:10.884992 11986 consensus_queue.cc:1048] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:10.886584 11983 consensus_queue.cc:1048] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:10.904227 11726 raft_consensus.cc:1275] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 5 FOLLOWER]: Refusing update from remote peer 8ea1829e48e84553bac8093876f65de9: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:17:10.905753 12007 consensus_queue.cc:1048] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:11.032737 11726 raft_consensus.cc:1275] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 3 FOLLOWER]: Refusing update from remote peer 8ea1829e48e84553bac8093876f65de9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:11.034197 12007 consensus_queue.cc:1048] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:11.045293 11652 raft_consensus.cc:1275] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 3 FOLLOWER]: Refusing update from remote peer 8ea1829e48e84553bac8093876f65de9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:11.046885 11983 consensus_queue.cc:1048] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8134a93015124c3ba3a2a7bf5ae8af32" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 45363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:11.139063 11799 raft_consensus.cc:1275] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [term 3 FOLLOWER]: Refusing update from remote peer 8134a93015124c3ba3a2a7bf5ae8af32: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:11.140445 12001 consensus_queue.cc:1048] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ea1829e48e84553bac8093876f65de9" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:11.150609 11726 raft_consensus.cc:1275] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 3 FOLLOWER]: Refusing update from remote peer 8134a93015124c3ba3a2a7bf5ae8af32: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:11.151970 11987 consensus_queue.cc:1048] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "928a1fe46e5944c8801f6ff56e1d5a3e" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 35361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:11.379338 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:11.385018 11517 tablet_server.cc:179] TabletServer@127.11.63.65:0 shutting down...
I20260709 13:17:11.426750 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:11.427609 11517 tablet_replica.cc:333] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32: stopping tablet replica
I20260709 13:17:11.428292 11517 raft_consensus.cc:2243] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:11.428885 11517 raft_consensus.cc:2272] T ebbcce463d0e4eefb40d760fff058275 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.431190 11517 tablet_replica.cc:333] T e1e3276493b1468aab90033dfbf0ceeb P 8134a93015124c3ba3a2a7bf5ae8af32: stopping tablet replica
I20260709 13:17:11.431710 11517 raft_consensus.cc:2243] T e1e3276493b1468aab90033dfbf0ceeb P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:11.432068 11517 raft_consensus.cc:2272] T e1e3276493b1468aab90033dfbf0ceeb P 8134a93015124c3ba3a2a7bf5ae8af32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.433919 11517 tablet_replica.cc:333] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32: stopping tablet replica
I20260709 13:17:11.434417 11517 raft_consensus.cc:2243] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:11.435119 11517 raft_consensus.cc:2272] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.437717 11517 tablet_replica.cc:333] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32: stopping tablet replica
I20260709 13:17:11.438254 11517 raft_consensus.cc:2243] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:11.438668 11517 raft_consensus.cc:2272] T e41561d6cff44c26a93018d0f521697a P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.440171 11517 tablet_replica.cc:333] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32: stopping tablet replica
I20260709 13:17:11.440630 11517 raft_consensus.cc:2243] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:11.441051 11517 raft_consensus.cc:2272] T 0749b17931544ef6938fd8a33768d265 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.443675 11517 tablet_replica.cc:333] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32: stopping tablet replica
I20260709 13:17:11.444231 11517 raft_consensus.cc:2243] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:11.444761 11517 raft_consensus.cc:2272] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.447078 11517 tablet_replica.cc:333] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32: stopping tablet replica
W20260709 13:17:11.447166 11691 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:11.447827 11517 raft_consensus.cc:2243] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:11.448479 11517 raft_consensus.cc:2272] T 5dd080bfdd96482486e355711e04fa83 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:11.450488 11691 consensus_peers.cc:597] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e -> Peer 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363): Couldn't send request to peer 8134a93015124c3ba3a2a7bf5ae8af32. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:11.450743 11517 tablet_replica.cc:333] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32: stopping tablet replica
I20260709 13:17:11.451228 11517 raft_consensus.cc:2243] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:11.451876 11517 raft_consensus.cc:2272] T 15aec8d8a49042f9adfc877e482c4907 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.453634 11517 tablet_replica.cc:333] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32: stopping tablet replica
I20260709 13:17:11.454027 11517 raft_consensus.cc:2243] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:17:11.454607 11517 raft_consensus.cc:2272] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8134a93015124c3ba3a2a7bf5ae8af32 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.594969 11517 tablet_server.cc:196] TabletServer@127.11.63.65:0 shutdown complete.
W20260709 13:17:11.594906 11765 consensus_peers.cc:597] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 -> Peer 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363): Couldn't send request to peer 8134a93015124c3ba3a2a7bf5ae8af32. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:45363: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:11.595621 11765 consensus_peers.cc:597] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 -> Peer 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363): Couldn't send request to peer 8134a93015124c3ba3a2a7bf5ae8af32. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:45363: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:11.596097 11765 consensus_peers.cc:597] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 -> Peer 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363): Couldn't send request to peer 8134a93015124c3ba3a2a7bf5ae8af32. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:45363: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:11.612527 11691 consensus_peers.cc:597] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e -> Peer 8134a93015124c3ba3a2a7bf5ae8af32 (127.11.63.65:45363): Couldn't send request to peer 8134a93015124c3ba3a2a7bf5ae8af32. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:45363: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:11.619083 11517 tablet_server.cc:179] TabletServer@127.11.63.66:0 shutting down...
W20260709 13:17:11.635541 11765 consensus_peers.cc:597] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 -> Peer 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361): Couldn't send request to peer 928a1fe46e5944c8801f6ff56e1d5a3e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:11.650758 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:11.651546 11517 tablet_replica.cc:333] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e: stopping tablet replica
I20260709 13:17:11.652204 11517 raft_consensus.cc:2243] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:11.652988 11517 raft_consensus.cc:2272] T 5dd080bfdd96482486e355711e04fa83 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.655083 11517 tablet_replica.cc:333] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e: stopping tablet replica
I20260709 13:17:11.655659 11517 raft_consensus.cc:2243] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:11.656464 11517 raft_consensus.cc:2272] T ebbcce463d0e4eefb40d760fff058275 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.658273 11517 tablet_replica.cc:333] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e: stopping tablet replica
I20260709 13:17:11.658806 11517 raft_consensus.cc:2243] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:11.659211 11517 raft_consensus.cc:2272] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.661128 11517 tablet_replica.cc:333] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e: stopping tablet replica
I20260709 13:17:11.661715 11517 raft_consensus.cc:2243] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:11.662155 11517 raft_consensus.cc:2272] T 15aec8d8a49042f9adfc877e482c4907 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.664286 11517 tablet_replica.cc:333] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e: stopping tablet replica
I20260709 13:17:11.664886 11517 raft_consensus.cc:2243] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:11.665302 11517 raft_consensus.cc:2272] T 0749b17931544ef6938fd8a33768d265 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.667585 11517 tablet_replica.cc:333] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e: stopping tablet replica
I20260709 13:17:11.668118 11517 raft_consensus.cc:2243] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:11.668536 11517 raft_consensus.cc:2272] T 85e1a267c28d4122b8e8cd93d34e9c50 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.670735 11517 tablet_replica.cc:333] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e: stopping tablet replica
I20260709 13:17:11.671262 11517 raft_consensus.cc:2243] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:11.671671 11517 raft_consensus.cc:2272] T 112100a2ecf0402f8a4f2fc0571a6a21 P 928a1fe46e5944c8801f6ff56e1d5a3e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.673702 11517 tablet_replica.cc:333] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e: stopping tablet replica
I20260709 13:17:11.674178 11517 raft_consensus.cc:2243] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:11.674932 11517 raft_consensus.cc:2272] T e41561d6cff44c26a93018d0f521697a P 928a1fe46e5944c8801f6ff56e1d5a3e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.676667 11517 tablet_replica.cc:333] T e1e3276493b1468aab90033dfbf0ceeb P 928a1fe46e5944c8801f6ff56e1d5a3e: stopping tablet replica
I20260709 13:17:11.677099 11517 raft_consensus.cc:2243] T e1e3276493b1468aab90033dfbf0ceeb P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:11.677537 11517 raft_consensus.cc:2272] T e1e3276493b1468aab90033dfbf0ceeb P 928a1fe46e5944c8801f6ff56e1d5a3e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:11.684034 11765 consensus_peers.cc:597] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 -> Peer 928a1fe46e5944c8801f6ff56e1d5a3e (127.11.63.66:35361): Couldn't send request to peer 928a1fe46e5944c8801f6ff56e1d5a3e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:11.707891 11517 tablet_server.cc:196] TabletServer@127.11.63.66:0 shutdown complete.
I20260709 13:17:11.724017 11517 tablet_server.cc:179] TabletServer@127.11.63.67:0 shutting down...
I20260709 13:17:11.748340 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:11.749114 11517 tablet_replica.cc:333] T ebbcce463d0e4eefb40d760fff058275 P 8ea1829e48e84553bac8093876f65de9: stopping tablet replica
I20260709 13:17:11.749795 11517 raft_consensus.cc:2243] T ebbcce463d0e4eefb40d760fff058275 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:11.750262 11517 raft_consensus.cc:2272] T ebbcce463d0e4eefb40d760fff058275 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.751977 11517 tablet_replica.cc:333] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9: stopping tablet replica
I20260709 13:17:11.752480 11517 raft_consensus.cc:2243] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:11.752862 11517 raft_consensus.cc:2272] T 5dd080bfdd96482486e355711e04fa83 P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.754597 11517 tablet_replica.cc:333] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8ea1829e48e84553bac8093876f65de9: stopping tablet replica
I20260709 13:17:11.755079 11517 raft_consensus.cc:2243] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8ea1829e48e84553bac8093876f65de9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:11.755478 11517 raft_consensus.cc:2272] T 112100a2ecf0402f8a4f2fc0571a6a21 P 8ea1829e48e84553bac8093876f65de9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.757942 11517 tablet_replica.cc:333] T e41561d6cff44c26a93018d0f521697a P 8ea1829e48e84553bac8093876f65de9: stopping tablet replica
I20260709 13:17:11.758442 11517 raft_consensus.cc:2243] T e41561d6cff44c26a93018d0f521697a P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:11.758841 11517 raft_consensus.cc:2272] T e41561d6cff44c26a93018d0f521697a P 8ea1829e48e84553bac8093876f65de9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.760478 11517 tablet_replica.cc:333] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9: stopping tablet replica
I20260709 13:17:11.761058 11517 raft_consensus.cc:2243] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:11.761853 11517 raft_consensus.cc:2272] T e1e3276493b1468aab90033dfbf0ceeb P 8ea1829e48e84553bac8093876f65de9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.763458 11517 tablet_replica.cc:333] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9: stopping tablet replica
I20260709 13:17:11.764021 11517 raft_consensus.cc:2243] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:17:11.764729 11517 raft_consensus.cc:2272] T 0749b17931544ef6938fd8a33768d265 P 8ea1829e48e84553bac8093876f65de9 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.766799 11517 tablet_replica.cc:333] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9: stopping tablet replica
I20260709 13:17:11.767277 11517 raft_consensus.cc:2243] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:11.767978 11517 raft_consensus.cc:2272] T 85e1a267c28d4122b8e8cd93d34e9c50 P 8ea1829e48e84553bac8093876f65de9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.769912 11517 tablet_replica.cc:333] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9: stopping tablet replica
I20260709 13:17:11.770378 11517 raft_consensus.cc:2243] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:11.770797 11517 raft_consensus.cc:2272] T 15aec8d8a49042f9adfc877e482c4907 P 8ea1829e48e84553bac8093876f65de9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.773036 11517 tablet_replica.cc:333] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9: stopping tablet replica
I20260709 13:17:11.773485 11517 raft_consensus.cc:2243] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:11.773974 11517 raft_consensus.cc:2272] T 90d25fd4fab44cd4a8e27fc6ce41eb61 P 8ea1829e48e84553bac8093876f65de9 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:11.801550 11517 tablet_server.cc:196] TabletServer@127.11.63.67:0 shutdown complete.
I20260709 13:17:11.819264 11517 master.cc:562] Master@127.11.63.126:40503 shutting down...
I20260709 13:17:11.945448 11604 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Closing
I20260709 13:17:12.021683 11517 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:12.022185 11517 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:12.022462 11517 tablet_replica.cc:333] T 00000000000000000000000000000000 P 8d87bdc096194fccaed6c5e7331e9b48: stopping tablet replica
I20260709 13:17:12.041826 11517 master.cc:584] Master@127.11.63.126:40503 shutdown complete.
8134a93015124c3ba3a2a7bf5ae8af32  4
8ea1829e48e84553bac8093876f65de9  1
928a1fe46e5944c8801f6ff56e1d5a3e  4
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (12608 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:17:12.082652 11517 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.11.63.126:43933
I20260709 13:17:12.083752 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:12.089526 12018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:12.089742 12019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:12.090704 12021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:12.096268 11517 server_base.cc:1061] running on GCE node
I20260709 13:17:12.097257 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:12.097448 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:12.097694 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603032097679 us; error 0 us; skew 500 ppm
I20260709 13:17:12.098284 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:12.101202 11517 webserver.cc:533] Webserver started at http://127.11.63.126:38053/ using document root <none> and password file <none>
I20260709 13:17:12.101864 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:12.102088 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:12.102414 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:12.103665 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603019296230-11517-0/minicluster-data/master-0-root/instance:
uuid: "0ffef970145944f9be864f3ba5eb1068"
format_stamp: "Formatted at 2026-07-09 13:17:12 on dist-test-slave-qv7r"
I20260709 13:17:12.108474 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:17:12.112406 12026 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:12.113217 11517 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:17:12.113544 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603019296230-11517-0/minicluster-data/master-0-root
uuid: "0ffef970145944f9be864f3ba5eb1068"
format_stamp: "Formatted at 2026-07-09 13:17:12 on dist-test-slave-qv7r"
I20260709 13:17:12.113991 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603019296230-11517-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603019296230-11517-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603019296230-11517-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:12.131253 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:12.132535 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:12.172686 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.126:43933
I20260709 13:17:12.172789 12077 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.126:43933 every 8 connection(s)
I20260709 13:17:12.176750 12078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.187397 12078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068: Bootstrap starting.
I20260709 13:17:12.192013 12078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:12.196368 12078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068: No bootstrap required, opened a new log
I20260709 13:17:12.198530 12078 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ffef970145944f9be864f3ba5eb1068" member_type: VOTER }
I20260709 13:17:12.198921 12078 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:12.199100 12078 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ffef970145944f9be864f3ba5eb1068, State: Initialized, Role: FOLLOWER
I20260709 13:17:12.199664 12078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ffef970145944f9be864f3ba5eb1068" member_type: VOTER }
I20260709 13:17:12.200076 12078 raft_consensus.cc:399] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:17:12.200290 12078 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:17:12.200502 12078 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:12.205055 12078 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ffef970145944f9be864f3ba5eb1068" member_type: VOTER }
I20260709 13:17:12.205575 12078 leader_election.cc:304] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ffef970145944f9be864f3ba5eb1068; no voters: 
I20260709 13:17:12.206651 12078 leader_election.cc:290] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:17:12.206992 12081 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:12.208389 12081 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068 [term 1 LEADER]: Becoming Leader. State: Replica: 0ffef970145944f9be864f3ba5eb1068, State: Running, Role: LEADER
I20260709 13:17:12.209084 12081 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ffef970145944f9be864f3ba5eb1068" member_type: VOTER }
I20260709 13:17:12.209659 12078 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:17:12.213840 12083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ffef970145944f9be864f3ba5eb1068. Latest consensus state: current_term: 1 leader_uuid: "0ffef970145944f9be864f3ba5eb1068" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ffef970145944f9be864f3ba5eb1068" member_type: VOTER } }
I20260709 13:17:12.214540 12083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068 [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:12.216756 12082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ffef970145944f9be864f3ba5eb1068" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ffef970145944f9be864f3ba5eb1068" member_type: VOTER } }
I20260709 13:17:12.217262 12088 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:17:12.217525 12082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068 [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:12.225764 12088 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:17:12.226737 11517 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:17:12.234438 12088 catalog_manager.cc:1383] Generated new cluster ID: ebe2cbe642104bd7a41f6575e6039ca3
I20260709 13:17:12.234741 12088 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:17:12.250751 12088 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:17:12.251961 12088 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:17:12.263880 12088 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068: Generated new TSK 0
I20260709 13:17:12.264528 12088 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:17:12.293341 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:12.300141 12099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:12.300660 12100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:12.302536 12102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:12.302812 11517 server_base.cc:1061] running on GCE node
I20260709 13:17:12.303699 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:12.303913 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:12.304087 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603032304071 us; error 0 us; skew 500 ppm
I20260709 13:17:12.304631 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:12.307147 11517 webserver.cc:533] Webserver started at http://127.11.63.65:35385/ using document root <none> and password file <none>
I20260709 13:17:12.307641 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:12.307842 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:12.308109 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:12.309155 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603019296230-11517-0/minicluster-data/ts-0-root/instance:
uuid: "9dbac62bc9fe4f96a350fe935538b020"
format_stamp: "Formatted at 2026-07-09 13:17:12 on dist-test-slave-qv7r"
I20260709 13:17:12.313907 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.001s
I20260709 13:17:12.317277 12107 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:12.318168 11517 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:17:12.318460 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603019296230-11517-0/minicluster-data/ts-0-root
uuid: "9dbac62bc9fe4f96a350fe935538b020"
format_stamp: "Formatted at 2026-07-09 13:17:12 on dist-test-slave-qv7r"
I20260709 13:17:12.318748 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603019296230-11517-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603019296230-11517-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603019296230-11517-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:12.335548 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:12.336840 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:12.338380 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:12.340704 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:12.340938 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:12.341174 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:12.341367 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:12.383993 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.65:39933
I20260709 13:17:12.384080 12169 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.65:39933 every 8 connection(s)
I20260709 13:17:12.388993 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:12.396126 12174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:12.396238 12175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:12.400615 11517 server_base.cc:1061] running on GCE node
W20260709 13:17:12.401544 12177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:12.402225 12170 heartbeater.cc:344] Connected to a master server at 127.11.63.126:43933
I20260709 13:17:12.402616 12170 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:12.402694 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:12.402978 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:12.403167 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603032403149 us; error 0 us; skew 500 ppm
I20260709 13:17:12.403537 12170 heartbeater.cc:507] Master 127.11.63.126:43933 requested a full tablet report, sending...
I20260709 13:17:12.403957 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:12.406018 12043 ts_manager.cc:194] Registered new tserver with Master: 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933)
I20260709 13:17:12.406812 11517 webserver.cc:533] Webserver started at http://127.11.63.66:42749/ using document root <none> and password file <none>
I20260709 13:17:12.407323 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:12.407512 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:12.407793 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:12.408099 12043 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41466
I20260709 13:17:12.409003 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603019296230-11517-0/minicluster-data/ts-1-root/instance:
uuid: "c3d2109084424941b5c909182d5bfe07"
format_stamp: "Formatted at 2026-07-09 13:17:12 on dist-test-slave-qv7r"
I20260709 13:17:12.413704 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.001s
I20260709 13:17:12.417106 12182 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:12.417943 11517 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:17:12.418241 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603019296230-11517-0/minicluster-data/ts-1-root
uuid: "c3d2109084424941b5c909182d5bfe07"
format_stamp: "Formatted at 2026-07-09 13:17:12 on dist-test-slave-qv7r"
I20260709 13:17:12.418493 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603019296230-11517-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603019296230-11517-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603019296230-11517-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:12.439363 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:12.440649 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:12.442173 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:12.444283 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:12.444515 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:12.444754 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:12.444921 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:12.486850 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.66:39197
I20260709 13:17:12.487061 12244 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.66:39197 every 8 connection(s)
I20260709 13:17:12.491413 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:12.498273 12248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:12.499706 12249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:12.501444 12251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:12.503062 11517 server_base.cc:1061] running on GCE node
I20260709 13:17:12.503922 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:12.504148 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:12.504356 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603032504340 us; error 0 us; skew 500 ppm
I20260709 13:17:12.504988 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:12.507938 11517 webserver.cc:533] Webserver started at http://127.11.63.67:40515/ using document root <none> and password file <none>
I20260709 13:17:12.508335 12245 heartbeater.cc:344] Connected to a master server at 127.11.63.126:43933
I20260709 13:17:12.508477 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:12.508755 12245 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:12.508803 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:12.509248 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:12.509773 12245 heartbeater.cc:507] Master 127.11.63.126:43933 requested a full tablet report, sending...
I20260709 13:17:12.510516 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603019296230-11517-0/minicluster-data/ts-2-root/instance:
uuid: "eaee9010d8714afea747f296df4322f0"
format_stamp: "Formatted at 2026-07-09 13:17:12 on dist-test-slave-qv7r"
I20260709 13:17:12.512171 12043 ts_manager.cc:194] Registered new tserver with Master: c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197)
I20260709 13:17:12.513531 12043 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41474
I20260709 13:17:12.515427 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:17:12.518894 12256 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:12.520015 11517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 13:17:12.520341 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603019296230-11517-0/minicluster-data/ts-2-root
uuid: "eaee9010d8714afea747f296df4322f0"
format_stamp: "Formatted at 2026-07-09 13:17:12 on dist-test-slave-qv7r"
I20260709 13:17:12.520717 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603019296230-11517-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603019296230-11517-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603019296230-11517-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:12.536234 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:12.537410 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:12.538890 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:12.541066 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:12.541298 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:12.541587 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:12.541774 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:17:12.581341 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.67:40843
I20260709 13:17:12.581434 12318 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.67:40843 every 8 connection(s)
I20260709 13:17:12.596338 12319 heartbeater.cc:344] Connected to a master server at 127.11.63.126:43933
I20260709 13:17:12.596760 12319 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:12.597710 12319 heartbeater.cc:507] Master 127.11.63.126:43933 requested a full tablet report, sending...
I20260709 13:17:12.599622 12043 ts_manager.cc:194] Registered new tserver with Master: eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:12.600657 11517 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016098139s
I20260709 13:17:12.600982 12043 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41484
I20260709 13:17:12.628093 12043 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41488:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:12.631001 12043 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:12.718711 12209 tablet_service.cc:1511] Processing CreateTablet for tablet 0a20e6b89add4f61bb32e0148c51271c (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:17:12.720291 12209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a20e6b89add4f61bb32e0148c51271c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.719947 12208 tablet_service.cc:1511] Processing CreateTablet for tablet b5066daaafb345b68502e7a69ce523da (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:17:12.718852 12210 tablet_service.cc:1511] Processing CreateTablet for tablet c20e9b895ed4401e97300148b98276e3 (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:17:12.721316 12208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5066daaafb345b68502e7a69ce523da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.722788 12207 tablet_service.cc:1511] Processing CreateTablet for tablet 25f7341f9ba04efb8912fcbc4f6b190a (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:17:12.723332 12205 tablet_service.cc:1511] Processing CreateTablet for tablet c9544a57d3924366b84a3f2eda5e6140 (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:17:12.725755 12206 tablet_service.cc:1511] Processing CreateTablet for tablet 6d830eb2a7b547f38ce277c1a68da16e (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:17:12.730113 12204 tablet_service.cc:1511] Processing CreateTablet for tablet 1488366d31f84925b97ed8f1843b957f (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:17:12.733867 12205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9544a57d3924366b84a3f2eda5e6140. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.735200 12203 tablet_service.cc:1511] Processing CreateTablet for tablet 0d9dbc6498c743e1a2fda71e38f4c3db (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:17:12.736554 12203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d9dbc6498c743e1a2fda71e38f4c3db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.737210 12204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1488366d31f84925b97ed8f1843b957f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.737900 12206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d830eb2a7b547f38ce277c1a68da16e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.741923 12207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25f7341f9ba04efb8912fcbc4f6b190a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.746031 12210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c20e9b895ed4401e97300148b98276e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.748420 12202 tablet_service.cc:1511] Processing CreateTablet for tablet 66472167c0814930aaa3398db1d2a421 (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:17:12.749789 12202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66472167c0814930aaa3398db1d2a421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.750982 12201 tablet_service.cc:1511] Processing CreateTablet for tablet ee84f8cd8c0242f3ba7a3b0dda942288 (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:17:12.752300 12201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee84f8cd8c0242f3ba7a3b0dda942288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.752688 12284 tablet_service.cc:1511] Processing CreateTablet for tablet c20e9b895ed4401e97300148b98276e3 (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:17:12.753966 12284 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c20e9b895ed4401e97300148b98276e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.755743 12279 tablet_service.cc:1511] Processing CreateTablet for tablet c9544a57d3924366b84a3f2eda5e6140 (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:17:12.757071 12279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9544a57d3924366b84a3f2eda5e6140. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.758783 12283 tablet_service.cc:1511] Processing CreateTablet for tablet 0a20e6b89add4f61bb32e0148c51271c (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:17:12.760134 12283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a20e6b89add4f61bb32e0148c51271c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.760752 12277 tablet_service.cc:1511] Processing CreateTablet for tablet 0d9dbc6498c743e1a2fda71e38f4c3db (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:17:12.762025 12277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d9dbc6498c743e1a2fda71e38f4c3db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.762681 12282 tablet_service.cc:1511] Processing CreateTablet for tablet b5066daaafb345b68502e7a69ce523da (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:17:12.764037 12282 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5066daaafb345b68502e7a69ce523da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.767761 12276 tablet_service.cc:1511] Processing CreateTablet for tablet 66472167c0814930aaa3398db1d2a421 (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:17:12.769455 12334 tablet_bootstrap.cc:492] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07: Bootstrap starting.
I20260709 13:17:12.769272 12281 tablet_service.cc:1511] Processing CreateTablet for tablet 25f7341f9ba04efb8912fcbc4f6b190a (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:17:12.758772 12278 tablet_service.cc:1511] Processing CreateTablet for tablet 1488366d31f84925b97ed8f1843b957f (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:17:12.767944 12280 tablet_service.cc:1511] Processing CreateTablet for tablet 6d830eb2a7b547f38ce277c1a68da16e (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:17:12.771548 12275 tablet_service.cc:1511] Processing CreateTablet for tablet ee84f8cd8c0242f3ba7a3b0dda942288 (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:17:12.773710 12276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66472167c0814930aaa3398db1d2a421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.774745 12281 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25f7341f9ba04efb8912fcbc4f6b190a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.777251 12334 tablet_bootstrap.cc:654] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:12.777891 12275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee84f8cd8c0242f3ba7a3b0dda942288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.781889 12278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1488366d31f84925b97ed8f1843b957f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.786160 12280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d830eb2a7b547f38ce277c1a68da16e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.793601 12135 tablet_service.cc:1511] Processing CreateTablet for tablet c20e9b895ed4401e97300148b98276e3 (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:17:12.795598 12135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c20e9b895ed4401e97300148b98276e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.802703 12134 tablet_service.cc:1511] Processing CreateTablet for tablet 0a20e6b89add4f61bb32e0148c51271c (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:17:12.804692 12134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a20e6b89add4f61bb32e0148c51271c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.863024 12133 tablet_service.cc:1511] Processing CreateTablet for tablet b5066daaafb345b68502e7a69ce523da (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:17:12.866360 12133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5066daaafb345b68502e7a69ce523da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.881186 12132 tablet_service.cc:1511] Processing CreateTablet for tablet 25f7341f9ba04efb8912fcbc4f6b190a (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:17:12.892642 12130 tablet_service.cc:1511] Processing CreateTablet for tablet c9544a57d3924366b84a3f2eda5e6140 (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:17:12.892648 12127 tablet_service.cc:1511] Processing CreateTablet for tablet 66472167c0814930aaa3398db1d2a421 (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:17:12.892693 12131 tablet_service.cc:1511] Processing CreateTablet for tablet 6d830eb2a7b547f38ce277c1a68da16e (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:17:12.894228 12130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9544a57d3924366b84a3f2eda5e6140. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.894860 12131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d830eb2a7b547f38ce277c1a68da16e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.906689 12336 tablet_bootstrap.cc:492] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020: Bootstrap starting.
I20260709 13:17:12.905181 12127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66472167c0814930aaa3398db1d2a421. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.906945 12128 tablet_service.cc:1511] Processing CreateTablet for tablet 0d9dbc6498c743e1a2fda71e38f4c3db (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:17:12.908401 12128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d9dbc6498c743e1a2fda71e38f4c3db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.903371 12129 tablet_service.cc:1511] Processing CreateTablet for tablet 1488366d31f84925b97ed8f1843b957f (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:17:12.909854 12132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25f7341f9ba04efb8912fcbc4f6b190a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.910804 12129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1488366d31f84925b97ed8f1843b957f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.903332 12126 tablet_service.cc:1511] Processing CreateTablet for tablet ee84f8cd8c0242f3ba7a3b0dda942288 (DEFAULT_TABLE table=test-workload [id=ae37933d63e14ae3a408b9255f3d84fe]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:17:12.914713 12126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee84f8cd8c0242f3ba7a3b0dda942288. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:12.915629 12334 tablet_bootstrap.cc:492] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07: No bootstrap required, opened a new log
I20260709 13:17:12.922381 12334 ts_tablet_manager.cc:1403] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07: Time spent bootstrapping tablet: real 0.153s	user 0.040s	sys 0.069s
I20260709 13:17:12.932216 12336 tablet_bootstrap.cc:654] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:12.925282 12334 raft_consensus.cc:359] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:12.945847 12334 raft_consensus.cc:385] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:12.946307 12337 tablet_bootstrap.cc:492] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0: Bootstrap starting.
I20260709 13:17:12.946278 12334 raft_consensus.cc:740] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3d2109084424941b5c909182d5bfe07, State: Initialized, Role: FOLLOWER
I20260709 13:17:12.950037 12334 consensus_queue.cc:260] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:12.953661 12337 tablet_bootstrap.cc:654] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:12.980647 12334 ts_tablet_manager.cc:1434] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07: Time spent starting tablet: real 0.058s	user 0.013s	sys 0.013s
I20260709 13:17:12.981384 12245 heartbeater.cc:499] Master 127.11.63.126:43933 was elected leader, sending a full tablet report...
I20260709 13:17:12.981681 12334 tablet_bootstrap.cc:492] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07: Bootstrap starting.
I20260709 13:17:12.982000 12336 tablet_bootstrap.cc:492] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020: No bootstrap required, opened a new log
I20260709 13:17:12.982470 12336 ts_tablet_manager.cc:1403] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020: Time spent bootstrapping tablet: real 0.076s	user 0.017s	sys 0.019s
I20260709 13:17:12.985425 12336 raft_consensus.cc:359] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:12.986231 12336 raft_consensus.cc:385] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:12.986537 12336 raft_consensus.cc:740] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Initialized, Role: FOLLOWER
I20260709 13:17:12.987210 12336 consensus_queue.cc:260] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:12.988219 12334 tablet_bootstrap.cc:654] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:12.993491 12337 tablet_bootstrap.cc:492] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0: No bootstrap required, opened a new log
I20260709 13:17:12.993973 12337 ts_tablet_manager.cc:1403] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0: Time spent bootstrapping tablet: real 0.048s	user 0.011s	sys 0.014s
I20260709 13:17:12.996887 12337 raft_consensus.cc:359] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:12.997649 12337 raft_consensus.cc:385] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:12.997951 12337 raft_consensus.cc:740] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaee9010d8714afea747f296df4322f0, State: Initialized, Role: FOLLOWER
I20260709 13:17:12.998875 12170 heartbeater.cc:499] Master 127.11.63.126:43933 was elected leader, sending a full tablet report...
I20260709 13:17:12.998853 12337 consensus_queue.cc:260] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.013300 12336 ts_tablet_manager.cc:1434] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.008s
I20260709 13:17:13.014741 12336 tablet_bootstrap.cc:492] T 66472167c0814930aaa3398db1d2a421 P 9dbac62bc9fe4f96a350fe935538b020: Bootstrap starting.
I20260709 13:17:13.023607 12336 tablet_bootstrap.cc:654] T 66472167c0814930aaa3398db1d2a421 P 9dbac62bc9fe4f96a350fe935538b020: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.027992 12319 heartbeater.cc:499] Master 127.11.63.126:43933 was elected leader, sending a full tablet report...
I20260709 13:17:13.028374 12337 ts_tablet_manager.cc:1434] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0: Time spent starting tablet: real 0.034s	user 0.015s	sys 0.005s
I20260709 13:17:13.029520 12337 tablet_bootstrap.cc:492] T 0d9dbc6498c743e1a2fda71e38f4c3db P eaee9010d8714afea747f296df4322f0: Bootstrap starting.
I20260709 13:17:13.033339 12334 tablet_bootstrap.cc:492] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07: No bootstrap required, opened a new log
I20260709 13:17:13.033981 12334 ts_tablet_manager.cc:1403] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07: Time spent bootstrapping tablet: real 0.053s	user 0.018s	sys 0.027s
I20260709 13:17:13.037220 12337 tablet_bootstrap.cc:654] T 0d9dbc6498c743e1a2fda71e38f4c3db P eaee9010d8714afea747f296df4322f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.037819 12343 raft_consensus.cc:493] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.038542 12343 raft_consensus.cc:515] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.036926 12334 raft_consensus.cc:359] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.046269 12334 raft_consensus.cc:385] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.046685 12334 raft_consensus.cc:740] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3d2109084424941b5c909182d5bfe07, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.047622 12341 raft_consensus.cc:493] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.048080 12341 raft_consensus.cc:515] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.049248 12343 leader_election.cc:290] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197)
I20260709 13:17:13.054684 12341 leader_election.cc:290] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:13.047616 12334 consensus_queue.cc:260] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.070536 12334 ts_tablet_manager.cc:1434] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07: Time spent starting tablet: real 0.036s	user 0.003s	sys 0.004s
I20260709 13:17:13.071586 12334 tablet_bootstrap.cc:492] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07: Bootstrap starting.
I20260709 13:17:13.087031 12337 tablet_bootstrap.cc:492] T 0d9dbc6498c743e1a2fda71e38f4c3db P eaee9010d8714afea747f296df4322f0: No bootstrap required, opened a new log
I20260709 13:17:13.087965 12337 ts_tablet_manager.cc:1403] T 0d9dbc6498c743e1a2fda71e38f4c3db P eaee9010d8714afea747f296df4322f0: Time spent bootstrapping tablet: real 0.059s	user 0.016s	sys 0.011s
I20260709 13:17:13.087031 12336 tablet_bootstrap.cc:492] T 66472167c0814930aaa3398db1d2a421 P 9dbac62bc9fe4f96a350fe935538b020: No bootstrap required, opened a new log
I20260709 13:17:13.092070 12337 raft_consensus.cc:359] T 0d9dbc6498c743e1a2fda71e38f4c3db P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.093773 12337 raft_consensus.cc:385] T 0d9dbc6498c743e1a2fda71e38f4c3db P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.094242 12334 tablet_bootstrap.cc:654] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.094161 12337 raft_consensus.cc:740] T 0d9dbc6498c743e1a2fda71e38f4c3db P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaee9010d8714afea747f296df4322f0, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.095192 12337 consensus_queue.cc:260] T 0d9dbc6498c743e1a2fda71e38f4c3db P eaee9010d8714afea747f296df4322f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.096848 12338 raft_consensus.cc:493] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.097342 12336 ts_tablet_manager.cc:1403] T 66472167c0814930aaa3398db1d2a421 P 9dbac62bc9fe4f96a350fe935538b020: Time spent bootstrapping tablet: real 0.083s	user 0.014s	sys 0.043s
I20260709 13:17:13.097357 12338 raft_consensus.cc:515] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.099960 12337 ts_tablet_manager.cc:1434] T 0d9dbc6498c743e1a2fda71e38f4c3db P eaee9010d8714afea747f296df4322f0: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 13:17:13.100960 12337 tablet_bootstrap.cc:492] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0: Bootstrap starting.
I20260709 13:17:13.106267 12338 leader_election.cc:290] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:13.114063 12336 raft_consensus.cc:359] T 66472167c0814930aaa3398db1d2a421 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.114820 12336 raft_consensus.cc:385] T 66472167c0814930aaa3398db1d2a421 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.115139 12336 raft_consensus.cc:740] T 66472167c0814930aaa3398db1d2a421 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.116155 12336 consensus_queue.cc:260] T 66472167c0814930aaa3398db1d2a421 P 9dbac62bc9fe4f96a350fe935538b020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.124279 12336 ts_tablet_manager.cc:1434] T 66472167c0814930aaa3398db1d2a421 P 9dbac62bc9fe4f96a350fe935538b020: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 13:17:13.128916 12337 tablet_bootstrap.cc:654] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.135562 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d830eb2a7b547f38ce277c1a68da16e" candidate_uuid: "eaee9010d8714afea747f296df4322f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbac62bc9fe4f96a350fe935538b020" is_pre_election: true
W20260709 13:17:13.137230 12257 leader_election.cc:343] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:13.138957 12220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20e9b895ed4401e97300148b98276e3" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d2109084424941b5c909182d5bfe07" is_pre_election: true
I20260709 13:17:13.139635 12220 raft_consensus.cc:2468] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 0.
I20260709 13:17:13.140929 12110 leader_election.cc:304] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dbac62bc9fe4f96a350fe935538b020, c3d2109084424941b5c909182d5bfe07; no voters: 
I20260709 13:17:13.141733 12341 raft_consensus.cc:2804] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:13.142117 12341 raft_consensus.cc:493] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:13.142457 12341 raft_consensus.cc:3060] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.143127 12334 tablet_bootstrap.cc:492] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07: No bootstrap required, opened a new log
I20260709 13:17:13.143620 12334 ts_tablet_manager.cc:1403] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07: Time spent bootstrapping tablet: real 0.072s	user 0.013s	sys 0.004s
I20260709 13:17:13.145800 12336 tablet_bootstrap.cc:492] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020: Bootstrap starting.
I20260709 13:17:13.147603 12334 raft_consensus.cc:359] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.148162 12337 tablet_bootstrap.cc:492] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0: No bootstrap required, opened a new log
I20260709 13:17:13.148327 12334 raft_consensus.cc:385] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.148720 12337 ts_tablet_manager.cc:1403] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0: Time spent bootstrapping tablet: real 0.048s	user 0.014s	sys 0.000s
I20260709 13:17:13.148685 12334 raft_consensus.cc:740] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3d2109084424941b5c909182d5bfe07, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.151918 12341 raft_consensus.cc:515] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.153995 12341 leader_election.cc:290] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 1 election: Requested vote from peers c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:13.150731 12334 consensus_queue.cc:260] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.157867 12294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20e9b895ed4401e97300148b98276e3" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaee9010d8714afea747f296df4322f0" is_pre_election: true
I20260709 13:17:13.157984 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1488366d31f84925b97ed8f1843b957f" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbac62bc9fe4f96a350fe935538b020" is_pre_election: true
I20260709 13:17:13.158499 12337 raft_consensus.cc:359] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.158951 12293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20e9b895ed4401e97300148b98276e3" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaee9010d8714afea747f296df4322f0"
W20260709 13:17:13.159394 12183 leader_election.cc:343] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:13.159561 12220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20e9b895ed4401e97300148b98276e3" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d2109084424941b5c909182d5bfe07"
I20260709 13:17:13.160113 12220 raft_consensus.cc:3060] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.163477 12334 ts_tablet_manager.cc:1434] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.002s
I20260709 13:17:13.164527 12334 tablet_bootstrap.cc:492] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07: Bootstrap starting.
I20260709 13:17:13.166055 12336 tablet_bootstrap.cc:654] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.167614 12220 raft_consensus.cc:2468] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 1.
I20260709 13:17:13.168884 12110 leader_election.cc:304] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9dbac62bc9fe4f96a350fe935538b020, c3d2109084424941b5c909182d5bfe07; no voters: eaee9010d8714afea747f296df4322f0
I20260709 13:17:13.169924 12341 raft_consensus.cc:2804] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:13.170879 12334 tablet_bootstrap.cc:654] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.159195 12337 raft_consensus.cc:385] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.176898 12337 raft_consensus.cc:740] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaee9010d8714afea747f296df4322f0, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.178194 12337 consensus_queue.cc:260] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.178732 12220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d830eb2a7b547f38ce277c1a68da16e" candidate_uuid: "eaee9010d8714afea747f296df4322f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d2109084424941b5c909182d5bfe07" is_pre_election: true
I20260709 13:17:13.179517 12294 raft_consensus.cc:2468] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 0.
I20260709 13:17:13.183667 12293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1488366d31f84925b97ed8f1843b957f" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaee9010d8714afea747f296df4322f0" is_pre_election: true
W20260709 13:17:13.185264 12259 leader_election.cc:343] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:13.185992 12259 leader_election.cc:304] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eaee9010d8714afea747f296df4322f0; no voters: 9dbac62bc9fe4f96a350fe935538b020, c3d2109084424941b5c909182d5bfe07
I20260709 13:17:13.190548 12341 raft_consensus.cc:697] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 1 LEADER]: Becoming Leader. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Running, Role: LEADER
I20260709 13:17:13.191149 12334 tablet_bootstrap.cc:492] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07: No bootstrap required, opened a new log
I20260709 13:17:13.191530 12334 ts_tablet_manager.cc:1403] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.012s
I20260709 13:17:13.191874 12341 consensus_queue.cc:237] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.193810 12343 raft_consensus.cc:2749] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:13.193670 12334 raft_consensus.cc:359] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.194404 12334 raft_consensus.cc:385] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.194713 12334 raft_consensus.cc:740] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3d2109084424941b5c909182d5bfe07, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.195827 12337 ts_tablet_manager.cc:1434] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0: Time spent starting tablet: real 0.047s	user 0.006s	sys 0.003s
I20260709 13:17:13.195468 12334 consensus_queue.cc:260] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.196908 12337 tablet_bootstrap.cc:492] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0: Bootstrap starting.
I20260709 13:17:13.199644 12334 ts_tablet_manager.cc:1434] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:17:13.200670 12334 tablet_bootstrap.cc:492] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07: Bootstrap starting.
I20260709 13:17:13.207157 12334 tablet_bootstrap.cc:654] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:13.222373 12185 leader_election.cc:343] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:13.223150 12185 leader_election.cc:304] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3d2109084424941b5c909182d5bfe07; no voters: 9dbac62bc9fe4f96a350fe935538b020, eaee9010d8714afea747f296df4322f0
I20260709 13:17:13.225001 12338 raft_consensus.cc:2749] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:17:13.227957 12091 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:13.227556 12043 catalog_manager.cc:5697] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "9dbac62bc9fe4f96a350fe935538b020" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:13.231607 12337 tablet_bootstrap.cc:654] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.238992 12336 tablet_bootstrap.cc:492] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020: No bootstrap required, opened a new log
I20260709 13:17:13.239589 12334 tablet_bootstrap.cc:492] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07: No bootstrap required, opened a new log
I20260709 13:17:13.239715 12337 tablet_bootstrap.cc:492] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0: No bootstrap required, opened a new log
I20260709 13:17:13.239681 12336 ts_tablet_manager.cc:1403] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020: Time spent bootstrapping tablet: real 0.107s	user 0.039s	sys 0.032s
I20260709 13:17:13.240139 12334 ts_tablet_manager.cc:1403] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07: Time spent bootstrapping tablet: real 0.040s	user 0.009s	sys 0.027s
I20260709 13:17:13.240476 12337 ts_tablet_manager.cc:1403] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0: Time spent bootstrapping tablet: real 0.044s	user 0.012s	sys 0.004s
I20260709 13:17:13.243554 12334 raft_consensus.cc:359] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.243803 12337 raft_consensus.cc:359] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.244397 12334 raft_consensus.cc:385] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.244513 12337 raft_consensus.cc:385] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.244783 12334 raft_consensus.cc:740] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3d2109084424941b5c909182d5bfe07, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.244885 12337 raft_consensus.cc:740] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaee9010d8714afea747f296df4322f0, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.245757 12334 consensus_queue.cc:260] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.247206 12336 raft_consensus.cc:359] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.248286 12336 raft_consensus.cc:385] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.248523 12334 ts_tablet_manager.cc:1434] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.001s
I20260709 13:17:13.248886 12336 raft_consensus.cc:740] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.246134 12337 consensus_queue.cc:260] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.252578 12337 ts_tablet_manager.cc:1434] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 13:17:13.249640 12334 tablet_bootstrap.cc:492] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07: Bootstrap starting.
I20260709 13:17:13.253795 12337 tablet_bootstrap.cc:492] T 25f7341f9ba04efb8912fcbc4f6b190a P eaee9010d8714afea747f296df4322f0: Bootstrap starting.
I20260709 13:17:13.253252 12336 consensus_queue.cc:260] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.256624 12336 ts_tablet_manager.cc:1434] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.002s
I20260709 13:17:13.257819 12336 tablet_bootstrap.cc:492] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020: Bootstrap starting.
I20260709 13:17:13.260924 12334 tablet_bootstrap.cc:654] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.261267 12338 raft_consensus.cc:493] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.261722 12337 tablet_bootstrap.cc:654] T 25f7341f9ba04efb8912fcbc4f6b190a P eaee9010d8714afea747f296df4322f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.261818 12338 raft_consensus.cc:515] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.265944 12336 tablet_bootstrap.cc:654] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.266340 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d9dbc6498c743e1a2fda71e38f4c3db" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbac62bc9fe4f96a350fe935538b020" is_pre_election: true
I20260709 13:17:13.266676 12338 leader_election.cc:290] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:13.267879 12294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d9dbc6498c743e1a2fda71e38f4c3db" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaee9010d8714afea747f296df4322f0" is_pre_election: true
I20260709 13:17:13.268563 12294 raft_consensus.cc:2468] T 0d9dbc6498c743e1a2fda71e38f4c3db P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3d2109084424941b5c909182d5bfe07 in term 0.
W20260709 13:17:13.270234 12183 leader_election.cc:343] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:13.271039 12185 leader_election.cc:304] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c3d2109084424941b5c909182d5bfe07, eaee9010d8714afea747f296df4322f0; no voters: 9dbac62bc9fe4f96a350fe935538b020
I20260709 13:17:13.279505 12334 tablet_bootstrap.cc:492] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07: No bootstrap required, opened a new log
I20260709 13:17:13.279556 12338 raft_consensus.cc:2804] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:13.280055 12334 ts_tablet_manager.cc:1403] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.002s
I20260709 13:17:13.280319 12338 raft_consensus.cc:493] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:13.280716 12338 raft_consensus.cc:3060] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.283396 12334 raft_consensus.cc:359] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.284196 12334 raft_consensus.cc:385] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.284534 12334 raft_consensus.cc:740] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3d2109084424941b5c909182d5bfe07, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.285306 12334 consensus_queue.cc:260] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.289341 12337 tablet_bootstrap.cc:492] T 25f7341f9ba04efb8912fcbc4f6b190a P eaee9010d8714afea747f296df4322f0: No bootstrap required, opened a new log
I20260709 13:17:13.289995 12337 ts_tablet_manager.cc:1403] T 25f7341f9ba04efb8912fcbc4f6b190a P eaee9010d8714afea747f296df4322f0: Time spent bootstrapping tablet: real 0.037s	user 0.020s	sys 0.014s
I20260709 13:17:13.293854 12337 raft_consensus.cc:359] T 25f7341f9ba04efb8912fcbc4f6b190a P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.294848 12337 raft_consensus.cc:385] T 25f7341f9ba04efb8912fcbc4f6b190a P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.295238 12337 raft_consensus.cc:740] T 25f7341f9ba04efb8912fcbc4f6b190a P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaee9010d8714afea747f296df4322f0, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.296530 12337 consensus_queue.cc:260] T 25f7341f9ba04efb8912fcbc4f6b190a P eaee9010d8714afea747f296df4322f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.302044 12337 ts_tablet_manager.cc:1434] T 25f7341f9ba04efb8912fcbc4f6b190a P eaee9010d8714afea747f296df4322f0: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:17:13.303330 12337 tablet_bootstrap.cc:492] T c9544a57d3924366b84a3f2eda5e6140 P eaee9010d8714afea747f296df4322f0: Bootstrap starting.
I20260709 13:17:13.308699 12338 raft_consensus.cc:515] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.310678 12337 tablet_bootstrap.cc:654] T c9544a57d3924366b84a3f2eda5e6140 P eaee9010d8714afea747f296df4322f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.312476 12341 raft_consensus.cc:493] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.317876 12341 raft_consensus.cc:515] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.321260 12341 leader_election.cc:290] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:13.321382 12220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25f7341f9ba04efb8912fcbc4f6b190a" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d2109084424941b5c909182d5bfe07" is_pre_election: true
I20260709 13:17:13.323657 12294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25f7341f9ba04efb8912fcbc4f6b190a" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaee9010d8714afea747f296df4322f0" is_pre_election: true
I20260709 13:17:13.325701 12293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d9dbc6498c743e1a2fda71e38f4c3db" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaee9010d8714afea747f296df4322f0"
I20260709 13:17:13.325906 12294 raft_consensus.cc:2468] T 25f7341f9ba04efb8912fcbc4f6b190a P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 0.
I20260709 13:17:13.326333 12293 raft_consensus.cc:3060] T 0d9dbc6498c743e1a2fda71e38f4c3db P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.324947 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d9dbc6498c743e1a2fda71e38f4c3db" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
W20260709 13:17:13.326869 12110 leader_election.cc:343] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:13.328238 12110 leader_election.cc:304] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9dbac62bc9fe4f96a350fe935538b020, eaee9010d8714afea747f296df4322f0; no voters: c3d2109084424941b5c909182d5bfe07
I20260709 13:17:13.329134 12357 raft_consensus.cc:2804] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:17:13.329264 12183 leader_election.cc:343] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:13.329552 12357 raft_consensus.cc:493] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:13.329923 12357 raft_consensus.cc:3060] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.314909 12368 raft_consensus.cc:493] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.330415 12368 raft_consensus.cc:515] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.334558 12293 raft_consensus.cc:2468] T 0d9dbc6498c743e1a2fda71e38f4c3db P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3d2109084424941b5c909182d5bfe07 in term 1.
I20260709 13:17:13.334743 12294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a20e6b89add4f61bb32e0148c51271c" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaee9010d8714afea747f296df4322f0" is_pre_election: true
I20260709 13:17:13.334776 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a20e6b89add4f61bb32e0148c51271c" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbac62bc9fe4f96a350fe935538b020" is_pre_election: true
I20260709 13:17:13.335944 12368 leader_election.cc:290] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:13.335875 12185 leader_election.cc:304] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c3d2109084424941b5c909182d5bfe07, eaee9010d8714afea747f296df4322f0; no voters: 9dbac62bc9fe4f96a350fe935538b020
I20260709 13:17:13.337136 12368 raft_consensus.cc:2804] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:17:13.337338 12185 leader_election.cc:343] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:13.337674 12368 raft_consensus.cc:697] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [term 1 LEADER]: Becoming Leader. State: Replica: c3d2109084424941b5c909182d5bfe07, State: Running, Role: LEADER
I20260709 13:17:13.337260 12357 raft_consensus.cc:515] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.338266 12338 leader_election.cc:290] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 election: Requested vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
W20260709 13:17:13.338548 12183 leader_election.cc:343] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:13.339751 12183 leader_election.cc:304] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3d2109084424941b5c909182d5bfe07; no voters: 9dbac62bc9fe4f96a350fe935538b020, eaee9010d8714afea747f296df4322f0
I20260709 13:17:13.339183 12334 ts_tablet_manager.cc:1434] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07: Time spent starting tablet: real 0.059s	user 0.025s	sys 0.025s
I20260709 13:17:13.342468 12220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25f7341f9ba04efb8912fcbc4f6b190a" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d2109084424941b5c909182d5bfe07"
W20260709 13:17:13.343806 12110 leader_election.cc:343] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:13.338560 12368 consensus_queue.cc:237] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.344229 12357 leader_election.cc:290] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 1 election: Requested vote from peers c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:13.343932 12334 tablet_bootstrap.cc:492] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07: Bootstrap starting.
I20260709 13:17:13.343951 12366 raft_consensus.cc:2749] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:13.353017 12334 tablet_bootstrap.cc:654] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.354023 12293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25f7341f9ba04efb8912fcbc4f6b190a" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaee9010d8714afea747f296df4322f0"
I20260709 13:17:13.356272 12293 raft_consensus.cc:3060] T 25f7341f9ba04efb8912fcbc4f6b190a P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.359315 12336 tablet_bootstrap.cc:492] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020: No bootstrap required, opened a new log
I20260709 13:17:13.359949 12336 ts_tablet_manager.cc:1403] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020: Time spent bootstrapping tablet: real 0.102s	user 0.031s	sys 0.036s
I20260709 13:17:13.363933 12336 raft_consensus.cc:359] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.364917 12336 raft_consensus.cc:385] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.365265 12336 raft_consensus.cc:740] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.368393 12334 tablet_bootstrap.cc:492] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07: No bootstrap required, opened a new log
I20260709 13:17:13.368942 12334 ts_tablet_manager.cc:1403] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.006s
I20260709 13:17:13.367664 12042 catalog_manager.cc:5697] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3d2109084424941b5c909182d5bfe07 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "c3d2109084424941b5c909182d5bfe07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:13.371793 12293 raft_consensus.cc:2468] T 25f7341f9ba04efb8912fcbc4f6b190a P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 1.
I20260709 13:17:13.371999 12334 raft_consensus.cc:359] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.373027 12334 raft_consensus.cc:385] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.372819 12336 consensus_queue.cc:260] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.373237 12110 leader_election.cc:304] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9dbac62bc9fe4f96a350fe935538b020, eaee9010d8714afea747f296df4322f0; no voters: c3d2109084424941b5c909182d5bfe07
I20260709 13:17:13.374145 12357 raft_consensus.cc:2804] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:13.374616 12357 raft_consensus.cc:697] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020 [term 1 LEADER]: Becoming Leader. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Running, Role: LEADER
I20260709 13:17:13.375145 12336 ts_tablet_manager.cc:1434] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.003s
I20260709 13:17:13.375420 12357 consensus_queue.cc:237] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.376055 12336 tablet_bootstrap.cc:492] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020: Bootstrap starting.
I20260709 13:17:13.373435 12334 raft_consensus.cc:740] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3d2109084424941b5c909182d5bfe07, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.381367 12334 consensus_queue.cc:260] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.384099 12336 tablet_bootstrap.cc:654] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.387698 12334 ts_tablet_manager.cc:1434] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.002s
I20260709 13:17:13.389262 12337 tablet_bootstrap.cc:492] T c9544a57d3924366b84a3f2eda5e6140 P eaee9010d8714afea747f296df4322f0: No bootstrap required, opened a new log
I20260709 13:17:13.390458 12334 tablet_bootstrap.cc:492] T 25f7341f9ba04efb8912fcbc4f6b190a P c3d2109084424941b5c909182d5bfe07: Bootstrap starting.
I20260709 13:17:13.392951 12337 ts_tablet_manager.cc:1403] T c9544a57d3924366b84a3f2eda5e6140 P eaee9010d8714afea747f296df4322f0: Time spent bootstrapping tablet: real 0.090s	user 0.029s	sys 0.013s
I20260709 13:17:13.393750 12043 catalog_manager.cc:5697] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "9dbac62bc9fe4f96a350fe935538b020" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:13.397406 12334 tablet_bootstrap.cc:654] T 25f7341f9ba04efb8912fcbc4f6b190a P c3d2109084424941b5c909182d5bfe07: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.398165 12336 tablet_bootstrap.cc:492] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020: No bootstrap required, opened a new log
I20260709 13:17:13.399523 12343 raft_consensus.cc:493] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.398701 12336 ts_tablet_manager.cc:1403] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.006s
I20260709 13:17:13.400069 12343 raft_consensus.cc:515] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.402185 12343 leader_election.cc:290] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197)
I20260709 13:17:13.403519 12336 raft_consensus.cc:359] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.404115 12220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66472167c0814930aaa3398db1d2a421" candidate_uuid: "eaee9010d8714afea747f296df4322f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d2109084424941b5c909182d5bfe07" is_pre_election: true
I20260709 13:17:13.404338 12336 raft_consensus.cc:385] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.404862 12334 tablet_bootstrap.cc:492] T 25f7341f9ba04efb8912fcbc4f6b190a P c3d2109084424941b5c909182d5bfe07: No bootstrap required, opened a new log
I20260709 13:17:13.404760 12220 raft_consensus.cc:2468] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaee9010d8714afea747f296df4322f0 in term 0.
I20260709 13:17:13.404800 12336 raft_consensus.cc:740] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.405481 12334 ts_tablet_manager.cc:1403] T 25f7341f9ba04efb8912fcbc4f6b190a P c3d2109084424941b5c909182d5bfe07: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.000s
I20260709 13:17:13.406263 12259 leader_election.cc:304] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3d2109084424941b5c909182d5bfe07, eaee9010d8714afea747f296df4322f0; no voters: 
I20260709 13:17:13.406144 12336 consensus_queue.cc:260] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.408054 12343 raft_consensus.cc:2804] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:13.403793 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66472167c0814930aaa3398db1d2a421" candidate_uuid: "eaee9010d8714afea747f296df4322f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbac62bc9fe4f96a350fe935538b020" is_pre_election: true
I20260709 13:17:13.408139 12334 raft_consensus.cc:359] T 25f7341f9ba04efb8912fcbc4f6b190a P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.408560 12343 raft_consensus.cc:493] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:13.408639 12336 ts_tablet_manager.cc:1434] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:17:13.408917 12334 raft_consensus.cc:385] T 25f7341f9ba04efb8912fcbc4f6b190a P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.408983 12343 raft_consensus.cc:3060] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.409408 12334 raft_consensus.cc:740] T 25f7341f9ba04efb8912fcbc4f6b190a P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3d2109084424941b5c909182d5bfe07, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.408954 12145 raft_consensus.cc:2468] T 66472167c0814930aaa3398db1d2a421 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaee9010d8714afea747f296df4322f0 in term 0.
I20260709 13:17:13.410156 12336 tablet_bootstrap.cc:492] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020: Bootstrap starting.
I20260709 13:17:13.410122 12334 consensus_queue.cc:260] T 25f7341f9ba04efb8912fcbc4f6b190a P c3d2109084424941b5c909182d5bfe07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.412253 12334 ts_tablet_manager.cc:1434] T 25f7341f9ba04efb8912fcbc4f6b190a P c3d2109084424941b5c909182d5bfe07: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:17:13.413256 12334 tablet_bootstrap.cc:492] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07: Bootstrap starting.
I20260709 13:17:13.417601 12336 tablet_bootstrap.cc:654] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.418856 12334 tablet_bootstrap.cc:654] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.420195 12368 raft_consensus.cc:493] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.420681 12368 raft_consensus.cc:515] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.395375 12337 raft_consensus.cc:359] T c9544a57d3924366b84a3f2eda5e6140 P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.422804 12368 leader_election.cc:290] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:13.422879 12337 raft_consensus.cc:385] T c9544a57d3924366b84a3f2eda5e6140 P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.423231 12337 raft_consensus.cc:740] T c9544a57d3924366b84a3f2eda5e6140 P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaee9010d8714afea747f296df4322f0, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.424902 12293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee84f8cd8c0242f3ba7a3b0dda942288" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaee9010d8714afea747f296df4322f0" is_pre_election: true
I20260709 13:17:13.425196 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee84f8cd8c0242f3ba7a3b0dda942288" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbac62bc9fe4f96a350fe935538b020" is_pre_election: true
W20260709 13:17:13.426327 12185 leader_election.cc:343] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:13.426853 12183 leader_election.cc:343] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:13.427280 12183 leader_election.cc:304] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3d2109084424941b5c909182d5bfe07; no voters: 9dbac62bc9fe4f96a350fe935538b020, eaee9010d8714afea747f296df4322f0
I20260709 13:17:13.428571 12368 raft_consensus.cc:2749] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:13.429982 12334 tablet_bootstrap.cc:492] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07: No bootstrap required, opened a new log
I20260709 13:17:13.429540 12337 consensus_queue.cc:260] T c9544a57d3924366b84a3f2eda5e6140 P eaee9010d8714afea747f296df4322f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.430449 12334 ts_tablet_manager.cc:1403] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.006s
I20260709 13:17:13.433254 12334 raft_consensus.cc:359] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.433908 12334 raft_consensus.cc:385] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.434196 12334 raft_consensus.cc:740] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3d2109084424941b5c909182d5bfe07, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.434288 12343 raft_consensus.cc:515] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.435091 12334 consensus_queue.cc:260] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.436435 12337 ts_tablet_manager.cc:1434] T c9544a57d3924366b84a3f2eda5e6140 P eaee9010d8714afea747f296df4322f0: Time spent starting tablet: real 0.043s	user 0.003s	sys 0.003s
I20260709 13:17:13.436971 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66472167c0814930aaa3398db1d2a421" candidate_uuid: "eaee9010d8714afea747f296df4322f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
I20260709 13:17:13.437546 12145 raft_consensus.cc:3060] T 66472167c0814930aaa3398db1d2a421 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.438098 12368 raft_consensus.cc:493] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.438571 12368 raft_consensus.cc:515] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.441829 12368 leader_election.cc:290] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:13.442050 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9544a57d3924366b84a3f2eda5e6140" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbac62bc9fe4f96a350fe935538b020" is_pre_election: true
I20260709 13:17:13.442425 12337 tablet_bootstrap.cc:492] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0: Bootstrap starting.
I20260709 13:17:13.443884 12334 ts_tablet_manager.cc:1434] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:17:13.442015 12293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9544a57d3924366b84a3f2eda5e6140" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaee9010d8714afea747f296df4322f0" is_pre_election: true
W20260709 13:17:13.444252 12183 leader_election.cc:343] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:13.444790 12334 tablet_bootstrap.cc:492] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07: Bootstrap starting.
I20260709 13:17:13.444375 12293 raft_consensus.cc:2468] T c9544a57d3924366b84a3f2eda5e6140 P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3d2109084424941b5c909182d5bfe07 in term 0.
I20260709 13:17:13.446250 12185 leader_election.cc:304] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c3d2109084424941b5c909182d5bfe07, eaee9010d8714afea747f296df4322f0; no voters: 9dbac62bc9fe4f96a350fe935538b020
I20260709 13:17:13.447744 12368 raft_consensus.cc:2804] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:13.448176 12368 raft_consensus.cc:493] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:13.448551 12368 raft_consensus.cc:3060] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.448827 12145 raft_consensus.cc:2468] T 66472167c0814930aaa3398db1d2a421 P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaee9010d8714afea747f296df4322f0 in term 1.
I20260709 13:17:13.450133 12257 leader_election.cc:304] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dbac62bc9fe4f96a350fe935538b020, eaee9010d8714afea747f296df4322f0; no voters: 
I20260709 13:17:13.450937 12334 tablet_bootstrap.cc:654] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.450999 12344 raft_consensus.cc:2804] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:13.466122 12343 leader_election.cc:290] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 1 election: Requested vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197)
I20260709 13:17:13.467268 12220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66472167c0814930aaa3398db1d2a421" candidate_uuid: "eaee9010d8714afea747f296df4322f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d2109084424941b5c909182d5bfe07"
I20260709 13:17:13.467921 12220 raft_consensus.cc:3060] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.468547 12368 raft_consensus.cc:515] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.474979 12344 raft_consensus.cc:697] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [term 1 LEADER]: Becoming Leader. State: Replica: eaee9010d8714afea747f296df4322f0, State: Running, Role: LEADER
I20260709 13:17:13.471432 12337 tablet_bootstrap.cc:654] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.473325 12336 tablet_bootstrap.cc:492] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020: No bootstrap required, opened a new log
I20260709 13:17:13.475988 12336 ts_tablet_manager.cc:1403] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020: Time spent bootstrapping tablet: real 0.066s	user 0.024s	sys 0.016s
I20260709 13:17:13.476866 12293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9544a57d3924366b84a3f2eda5e6140" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaee9010d8714afea747f296df4322f0"
I20260709 13:17:13.476261 12344 consensus_queue.cc:237] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.477481 12293 raft_consensus.cc:3060] T c9544a57d3924366b84a3f2eda5e6140 P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.479406 12336 raft_consensus.cc:359] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.480190 12336 raft_consensus.cc:385] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.483573 12336 raft_consensus.cc:740] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.483985 12334 tablet_bootstrap.cc:492] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07: No bootstrap required, opened a new log
I20260709 13:17:13.475172 12368 leader_election.cc:290] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 election: Requested vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:13.484488 12334 ts_tablet_manager.cc:1403] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07: Time spent bootstrapping tablet: real 0.040s	user 0.016s	sys 0.020s
I20260709 13:17:13.484992 12293 raft_consensus.cc:2468] T c9544a57d3924366b84a3f2eda5e6140 P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3d2109084424941b5c909182d5bfe07 in term 1.
I20260709 13:17:13.485121 12336 consensus_queue.cc:260] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.485934 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9544a57d3924366b84a3f2eda5e6140" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
I20260709 13:17:13.487123 12334 raft_consensus.cc:359] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.487618 12334 raft_consensus.cc:385] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.487852 12334 raft_consensus.cc:740] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3d2109084424941b5c909182d5bfe07, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.488193 12336 ts_tablet_manager.cc:1434] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.002s
I20260709 13:17:13.488590 12334 consensus_queue.cc:260] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.489418 12336 tablet_bootstrap.cc:492] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020: Bootstrap starting.
I20260709 13:17:13.490754 12334 ts_tablet_manager.cc:1434] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:17:13.497099 12336 tablet_bootstrap.cc:654] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.500929 12220 raft_consensus.cc:2468] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaee9010d8714afea747f296df4322f0 in term 1.
W20260709 13:17:13.503804 12183 leader_election.cc:343] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:13.503878 12336 tablet_bootstrap.cc:492] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020: No bootstrap required, opened a new log
I20260709 13:17:13.504670 12336 ts_tablet_manager.cc:1403] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.005s
I20260709 13:17:13.504633 12185 leader_election.cc:304] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c3d2109084424941b5c909182d5bfe07, eaee9010d8714afea747f296df4322f0; no voters: 9dbac62bc9fe4f96a350fe935538b020
I20260709 13:17:13.505837 12368 raft_consensus.cc:2804] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:13.506311 12368 raft_consensus.cc:697] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [term 1 LEADER]: Becoming Leader. State: Replica: c3d2109084424941b5c909182d5bfe07, State: Running, Role: LEADER
I20260709 13:17:13.507105 12368 consensus_queue.cc:237] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.509282 12336 raft_consensus.cc:359] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.510430 12336 raft_consensus.cc:385] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.510975 12336 raft_consensus.cc:740] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.512086 12336 consensus_queue.cc:260] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.512634 12042 catalog_manager.cc:5697] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to eaee9010d8714afea747f296df4322f0 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "eaee9010d8714afea747f296df4322f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:13.514866 12336 ts_tablet_manager.cc:1434] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:17:13.516055 12336 tablet_bootstrap.cc:492] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020: Bootstrap starting.
I20260709 13:17:13.524115 12337 tablet_bootstrap.cc:492] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0: No bootstrap required, opened a new log
I20260709 13:17:13.524616 12337 ts_tablet_manager.cc:1403] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0: Time spent bootstrapping tablet: real 0.082s	user 0.015s	sys 0.014s
I20260709 13:17:13.524912 12336 tablet_bootstrap.cc:654] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.527521 12337 raft_consensus.cc:359] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.532384 12040 catalog_manager.cc:5697] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3d2109084424941b5c909182d5bfe07 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "c3d2109084424941b5c909182d5bfe07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:13.532075 12337 raft_consensus.cc:385] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.534530 12337 raft_consensus.cc:740] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaee9010d8714afea747f296df4322f0, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.535530 12337 consensus_queue.cc:260] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.536943 12336 tablet_bootstrap.cc:492] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020: No bootstrap required, opened a new log
I20260709 13:17:13.537434 12336 ts_tablet_manager.cc:1403] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.007s
I20260709 13:17:13.538422 12337 ts_tablet_manager.cc:1434] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.006s
I20260709 13:17:13.539546 12337 tablet_bootstrap.cc:492] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0: Bootstrap starting.
I20260709 13:17:13.540354 12336 raft_consensus.cc:359] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.541110 12336 raft_consensus.cc:385] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.541409 12336 raft_consensus.cc:740] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.542164 12336 consensus_queue.cc:260] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.547537 12337 tablet_bootstrap.cc:654] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.547539 12336 ts_tablet_manager.cc:1434] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:17:13.549186 12336 tablet_bootstrap.cc:492] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020: Bootstrap starting.
I20260709 13:17:13.557081 12336 tablet_bootstrap.cc:654] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.559252 12337 tablet_bootstrap.cc:492] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0: No bootstrap required, opened a new log
I20260709 13:17:13.559576 12337 ts_tablet_manager.cc:1403] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.004s
I20260709 13:17:13.561302 12337 raft_consensus.cc:359] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.561830 12337 raft_consensus.cc:385] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.562006 12337 raft_consensus.cc:740] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaee9010d8714afea747f296df4322f0, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.562377 12336 tablet_bootstrap.cc:492] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020: No bootstrap required, opened a new log
I20260709 13:17:13.562525 12337 consensus_queue.cc:260] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.562821 12336 ts_tablet_manager.cc:1403] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.000s
I20260709 13:17:13.564020 12337 ts_tablet_manager.cc:1434] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0: Time spent starting tablet: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:17:13.564792 12337 tablet_bootstrap.cc:492] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0: Bootstrap starting.
I20260709 13:17:13.565536 12336 raft_consensus.cc:359] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.566282 12336 raft_consensus.cc:385] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.566576 12336 raft_consensus.cc:740] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.567291 12336 consensus_queue.cc:260] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.569766 12336 ts_tablet_manager.cc:1434] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:17:13.570207 12337 tablet_bootstrap.cc:654] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.570731 12336 tablet_bootstrap.cc:492] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020: Bootstrap starting.
I20260709 13:17:13.577747 12336 tablet_bootstrap.cc:654] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.583568 12337 tablet_bootstrap.cc:492] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0: No bootstrap required, opened a new log
I20260709 13:17:13.584039 12337 ts_tablet_manager.cc:1403] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 13:17:13.584606 12336 tablet_bootstrap.cc:492] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020: No bootstrap required, opened a new log
I20260709 13:17:13.585000 12336 ts_tablet_manager.cc:1403] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020: Time spent bootstrapping tablet: real 0.014s	user 0.007s	sys 0.004s
I20260709 13:17:13.586176 12337 raft_consensus.cc:359] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.586621 12337 raft_consensus.cc:385] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.586779 12337 raft_consensus.cc:740] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaee9010d8714afea747f296df4322f0, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.587217 12337 consensus_queue.cc:260] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.587291 12336 raft_consensus.cc:359] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.588032 12336 raft_consensus.cc:385] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.588181 12368 raft_consensus.cc:493] T 25f7341f9ba04efb8912fcbc4f6b190a P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.588364 12336 raft_consensus.cc:740] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.588725 12368 raft_consensus.cc:515] T 25f7341f9ba04efb8912fcbc4f6b190a P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.589172 12337 ts_tablet_manager.cc:1434] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:17:13.589213 12336 consensus_queue.cc:260] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.590221 12337 tablet_bootstrap.cc:492] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0: Bootstrap starting.
I20260709 13:17:13.591041 12368 leader_election.cc:290] T 25f7341f9ba04efb8912fcbc4f6b190a P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:13.594046 12336 ts_tablet_manager.cc:1434] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:17:13.593889 12293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25f7341f9ba04efb8912fcbc4f6b190a" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaee9010d8714afea747f296df4322f0" is_pre_election: true
I20260709 13:17:13.594501 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25f7341f9ba04efb8912fcbc4f6b190a" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbac62bc9fe4f96a350fe935538b020" is_pre_election: true
I20260709 13:17:13.595681 12293 raft_consensus.cc:2393] T 25f7341f9ba04efb8912fcbc4f6b190a P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c3d2109084424941b5c909182d5bfe07 in current term 1: Already voted for candidate 9dbac62bc9fe4f96a350fe935538b020 in this term.
I20260709 13:17:13.596990 12337 tablet_bootstrap.cc:654] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:13.596990 12185 leader_election.cc:304] T 25f7341f9ba04efb8912fcbc4f6b190a P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c3d2109084424941b5c909182d5bfe07; no voters: 9dbac62bc9fe4f96a350fe935538b020, eaee9010d8714afea747f296df4322f0
I20260709 13:17:13.597978 12368 raft_consensus.cc:3060] T 25f7341f9ba04efb8912fcbc4f6b190a P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.606588 12368 raft_consensus.cc:2749] T 25f7341f9ba04efb8912fcbc4f6b190a P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:13.609076 12337 tablet_bootstrap.cc:492] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0: No bootstrap required, opened a new log
I20260709 13:17:13.609552 12337 ts_tablet_manager.cc:1403] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.008s
I20260709 13:17:13.611909 12337 raft_consensus.cc:359] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.612571 12337 raft_consensus.cc:385] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:13.612798 12337 raft_consensus.cc:740] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaee9010d8714afea747f296df4322f0, State: Initialized, Role: FOLLOWER
I20260709 13:17:13.613281 12337 consensus_queue.cc:260] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.615000 12337 ts_tablet_manager.cc:1434] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:17:13.644914 12344 raft_consensus.cc:493] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.645527 12344 raft_consensus.cc:515] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.647413 12344 leader_election.cc:290] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197)
I20260709 13:17:13.648289 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a20e6b89add4f61bb32e0148c51271c" candidate_uuid: "eaee9010d8714afea747f296df4322f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbac62bc9fe4f96a350fe935538b020" is_pre_election: true
I20260709 13:17:13.648461 12220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a20e6b89add4f61bb32e0148c51271c" candidate_uuid: "eaee9010d8714afea747f296df4322f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d2109084424941b5c909182d5bfe07" is_pre_election: true
I20260709 13:17:13.648814 12145 raft_consensus.cc:2468] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaee9010d8714afea747f296df4322f0 in term 0.
I20260709 13:17:13.649013 12220 raft_consensus.cc:2468] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaee9010d8714afea747f296df4322f0 in term 0.
I20260709 13:17:13.649890 12257 leader_election.cc:304] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dbac62bc9fe4f96a350fe935538b020, eaee9010d8714afea747f296df4322f0; no voters: 
I20260709 13:17:13.650611 12344 raft_consensus.cc:2804] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:13.650866 12344 raft_consensus.cc:493] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:13.651139 12344 raft_consensus.cc:3060] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.655932 12344 raft_consensus.cc:515] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.657435 12344 leader_election.cc:290] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 1 election: Requested vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197)
I20260709 13:17:13.658421 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a20e6b89add4f61bb32e0148c51271c" candidate_uuid: "eaee9010d8714afea747f296df4322f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
I20260709 13:17:13.658949 12220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a20e6b89add4f61bb32e0148c51271c" candidate_uuid: "eaee9010d8714afea747f296df4322f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d2109084424941b5c909182d5bfe07"
I20260709 13:17:13.659049 12145 raft_consensus.cc:3060] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.659476 12220 raft_consensus.cc:3060] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.664103 12145 raft_consensus.cc:2468] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaee9010d8714afea747f296df4322f0 in term 1.
I20260709 13:17:13.664122 12220 raft_consensus.cc:2468] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaee9010d8714afea747f296df4322f0 in term 1.
I20260709 13:17:13.665243 12259 leader_election.cc:304] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3d2109084424941b5c909182d5bfe07, eaee9010d8714afea747f296df4322f0; no voters: 
I20260709 13:17:13.665970 12344 raft_consensus.cc:2804] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:13.666401 12344 raft_consensus.cc:697] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 1 LEADER]: Becoming Leader. State: Replica: eaee9010d8714afea747f296df4322f0, State: Running, Role: LEADER
I20260709 13:17:13.667155 12344 consensus_queue.cc:237] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.675302 12041 catalog_manager.cc:5697] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to eaee9010d8714afea747f296df4322f0 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "eaee9010d8714afea747f296df4322f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:13.700881 12357 raft_consensus.cc:493] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.701702 12357 raft_consensus.cc:515] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.703464 12357 leader_election.cc:290] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:13.704535 12220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee84f8cd8c0242f3ba7a3b0dda942288" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d2109084424941b5c909182d5bfe07" is_pre_election: true
I20260709 13:17:13.704770 12293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee84f8cd8c0242f3ba7a3b0dda942288" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaee9010d8714afea747f296df4322f0" is_pre_election: true
I20260709 13:17:13.705309 12220 raft_consensus.cc:2468] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 0.
I20260709 13:17:13.705351 12293 raft_consensus.cc:2468] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 0.
I20260709 13:17:13.706487 12110 leader_election.cc:304] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dbac62bc9fe4f96a350fe935538b020, c3d2109084424941b5c909182d5bfe07; no voters: 
I20260709 13:17:13.707120 12357 raft_consensus.cc:2804] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:13.707487 12357 raft_consensus.cc:493] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:13.707813 12357 raft_consensus.cc:3060] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.713131 12357 raft_consensus.cc:515] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.715370 12357 leader_election.cc:290] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 1 election: Requested vote from peers c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:13.716473 12293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee84f8cd8c0242f3ba7a3b0dda942288" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaee9010d8714afea747f296df4322f0"
I20260709 13:17:13.716735 12357 consensus_queue.cc:1048] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:13.716995 12220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee84f8cd8c0242f3ba7a3b0dda942288" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d2109084424941b5c909182d5bfe07"
I20260709 13:17:13.717084 12293 raft_consensus.cc:3060] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.717608 12220 raft_consensus.cc:3060] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.724246 12294 raft_consensus.cc:3060] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.725114 12293 raft_consensus.cc:2468] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 1.
I20260709 13:17:13.725093 12220 raft_consensus.cc:2468] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 1.
I20260709 13:17:13.728835 12110 leader_election.cc:304] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dbac62bc9fe4f96a350fe935538b020, c3d2109084424941b5c909182d5bfe07; no voters: 
I20260709 13:17:13.731822 12341 raft_consensus.cc:2804] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:13.733032 12341 raft_consensus.cc:697] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [term 1 LEADER]: Becoming Leader. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Running, Role: LEADER
I20260709 13:17:13.734002 12341 consensus_queue.cc:237] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.735747 12357 consensus_queue.cc:1048] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:13.745880 12041 catalog_manager.cc:5697] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "9dbac62bc9fe4f96a350fe935538b020" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:13.751909 12344 raft_consensus.cc:493] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.752463 12344 raft_consensus.cc:515] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.754362 12344 leader_election.cc:290] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197)
I20260709 13:17:13.755554 12219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5066daaafb345b68502e7a69ce523da" candidate_uuid: "eaee9010d8714afea747f296df4322f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d2109084424941b5c909182d5bfe07" is_pre_election: true
I20260709 13:17:13.755841 12145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5066daaafb345b68502e7a69ce523da" candidate_uuid: "eaee9010d8714afea747f296df4322f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbac62bc9fe4f96a350fe935538b020" is_pre_election: true
I20260709 13:17:13.756449 12145 raft_consensus.cc:2468] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaee9010d8714afea747f296df4322f0 in term 0.
I20260709 13:17:13.756202 12219 raft_consensus.cc:2468] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaee9010d8714afea747f296df4322f0 in term 0.
I20260709 13:17:13.757959 12259 leader_election.cc:304] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3d2109084424941b5c909182d5bfe07, eaee9010d8714afea747f296df4322f0; no voters: 
I20260709 13:17:13.759301 12357 raft_consensus.cc:493] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.758968 12344 raft_consensus.cc:2804] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:13.759976 12344 raft_consensus.cc:493] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:13.759749 12357 raft_consensus.cc:515] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.762041 12357 leader_election.cc:290] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:13.760365 12344 raft_consensus.cc:3060] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.763180 12219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d9dbc6498c743e1a2fda71e38f4c3db" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d2109084424941b5c909182d5bfe07" is_pre_election: true
I20260709 13:17:13.764269 12294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d9dbc6498c743e1a2fda71e38f4c3db" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaee9010d8714afea747f296df4322f0" is_pre_election: true
I20260709 13:17:13.765033 12294 raft_consensus.cc:2393] T 0d9dbc6498c743e1a2fda71e38f4c3db P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9dbac62bc9fe4f96a350fe935538b020 in current term 1: Already voted for candidate c3d2109084424941b5c909182d5bfe07 in this term.
I20260709 13:17:13.766137 12110 leader_election.cc:304] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9dbac62bc9fe4f96a350fe935538b020; no voters: c3d2109084424941b5c909182d5bfe07, eaee9010d8714afea747f296df4322f0
I20260709 13:17:13.766815 12357 raft_consensus.cc:3060] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.769187 12341 raft_consensus.cc:493] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.769675 12341 raft_consensus.cc:515] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.770370 12344 raft_consensus.cc:515] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.771694 12341 leader_election.cc:290] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:13.772419 12344 leader_election.cc:290] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 1 election: Requested vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197)
I20260709 13:17:13.772989 12219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1488366d31f84925b97ed8f1843b957f" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d2109084424941b5c909182d5bfe07" is_pre_election: true
I20260709 13:17:13.773463 12220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5066daaafb345b68502e7a69ce523da" candidate_uuid: "eaee9010d8714afea747f296df4322f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d2109084424941b5c909182d5bfe07"
I20260709 13:17:13.773615 12219 raft_consensus.cc:2468] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 0.
I20260709 13:17:13.773679 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5066daaafb345b68502e7a69ce523da" candidate_uuid: "eaee9010d8714afea747f296df4322f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
I20260709 13:17:13.774067 12220 raft_consensus.cc:3060] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.774194 12294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1488366d31f84925b97ed8f1843b957f" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaee9010d8714afea747f296df4322f0" is_pre_election: true
I20260709 13:17:13.774290 12144 raft_consensus.cc:3060] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.774830 12294 raft_consensus.cc:2468] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 0.
I20260709 13:17:13.775096 12110 leader_election.cc:304] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dbac62bc9fe4f96a350fe935538b020, c3d2109084424941b5c909182d5bfe07; no voters: 
I20260709 13:17:13.775983 12341 raft_consensus.cc:2804] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:13.776345 12341 raft_consensus.cc:493] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:13.776684 12341 raft_consensus.cc:3060] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.777426 12357 raft_consensus.cc:2749] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:13.780455 12144 raft_consensus.cc:2468] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaee9010d8714afea747f296df4322f0 in term 1.
I20260709 13:17:13.781325 12220 raft_consensus.cc:2468] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaee9010d8714afea747f296df4322f0 in term 1.
I20260709 13:17:13.781951 12257 leader_election.cc:304] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dbac62bc9fe4f96a350fe935538b020, eaee9010d8714afea747f296df4322f0; no voters: 
I20260709 13:17:13.781980 12368 consensus_queue.cc:1048] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:13.783299 12344 raft_consensus.cc:2804] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:13.783815 12344 raft_consensus.cc:697] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [term 1 LEADER]: Becoming Leader. State: Replica: eaee9010d8714afea747f296df4322f0, State: Running, Role: LEADER
I20260709 13:17:13.784615 12344 consensus_queue.cc:237] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.786558 12341 raft_consensus.cc:515] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.788697 12341 leader_election.cc:290] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 1 election: Requested vote from peers c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:13.789384 12220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1488366d31f84925b97ed8f1843b957f" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d2109084424941b5c909182d5bfe07"
I20260709 13:17:13.790000 12220 raft_consensus.cc:3060] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.790303 12294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1488366d31f84925b97ed8f1843b957f" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaee9010d8714afea747f296df4322f0"
I20260709 13:17:13.790835 12294 raft_consensus.cc:3060] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.797313 12220 raft_consensus.cc:2468] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 1.
I20260709 13:17:13.797775 12294 raft_consensus.cc:2468] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 1.
I20260709 13:17:13.799150 12110 leader_election.cc:304] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dbac62bc9fe4f96a350fe935538b020, eaee9010d8714afea747f296df4322f0; no voters: 
I20260709 13:17:13.800498 12341 raft_consensus.cc:2804] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:13.800937 12341 raft_consensus.cc:697] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 1 LEADER]: Becoming Leader. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Running, Role: LEADER
I20260709 13:17:13.801787 12341 consensus_queue.cc:237] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.813097 12366 consensus_queue.cc:1048] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:13.804960 12041 catalog_manager.cc:5697] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to eaee9010d8714afea747f296df4322f0 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "eaee9010d8714afea747f296df4322f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:13.816020 12366 raft_consensus.cc:493] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.816463 12366 raft_consensus.cc:515] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.816851 12040 catalog_manager.cc:5697] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "9dbac62bc9fe4f96a350fe935538b020" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:13.821686 12293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d830eb2a7b547f38ce277c1a68da16e" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaee9010d8714afea747f296df4322f0" is_pre_election: true
I20260709 13:17:13.822592 12293 raft_consensus.cc:2468] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3d2109084424941b5c909182d5bfe07 in term 0.
I20260709 13:17:13.824263 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d830eb2a7b547f38ce277c1a68da16e" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbac62bc9fe4f96a350fe935538b020" is_pre_election: true
I20260709 13:17:13.824589 12185 leader_election.cc:304] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3d2109084424941b5c909182d5bfe07, eaee9010d8714afea747f296df4322f0; no voters: 
I20260709 13:17:13.826069 12338 raft_consensus.cc:2804] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:13.826509 12338 raft_consensus.cc:493] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:13.826946 12338 raft_consensus.cc:3060] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.833709 12366 leader_election.cc:290] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:13.824931 12144 raft_consensus.cc:2468] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3d2109084424941b5c909182d5bfe07 in term 0.
I20260709 13:17:13.835067 12338 raft_consensus.cc:515] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.837172 12338 leader_election.cc:290] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 election: Requested vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:13.839136 12293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d830eb2a7b547f38ce277c1a68da16e" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaee9010d8714afea747f296df4322f0"
I20260709 13:17:13.839428 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d830eb2a7b547f38ce277c1a68da16e" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
I20260709 13:17:13.839743 12293 raft_consensus.cc:3060] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.840016 12144 raft_consensus.cc:3060] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.846652 12293 raft_consensus.cc:2468] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3d2109084424941b5c909182d5bfe07 in term 1.
I20260709 13:17:13.846681 12144 raft_consensus.cc:2468] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3d2109084424941b5c909182d5bfe07 in term 1.
I20260709 13:17:13.847859 12185 leader_election.cc:304] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3d2109084424941b5c909182d5bfe07, eaee9010d8714afea747f296df4322f0; no voters: 
I20260709 13:17:13.848541 12338 raft_consensus.cc:2804] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:13.849035 12338 raft_consensus.cc:697] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [term 1 LEADER]: Becoming Leader. State: Replica: c3d2109084424941b5c909182d5bfe07, State: Running, Role: LEADER
I20260709 13:17:13.849772 12338 consensus_queue.cc:237] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.858209 12040 catalog_manager.cc:5697] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 reported cstate change: term changed from 0 to 1, leader changed from <none> to c3d2109084424941b5c909182d5bfe07 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "c3d2109084424941b5c909182d5bfe07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:13.874851 12341 consensus_queue.cc:1048] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:13.879169 12357 consensus_queue.cc:1048] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:13.908777 12395 raft_consensus.cc:493] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:13.909417 12395 raft_consensus.cc:515] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:13.911170 12395 leader_election.cc:290] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:13.912195 12220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9544a57d3924366b84a3f2eda5e6140" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3d2109084424941b5c909182d5bfe07" is_pre_election: true
I20260709 13:17:13.912791 12293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9544a57d3924366b84a3f2eda5e6140" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaee9010d8714afea747f296df4322f0" is_pre_election: true
I20260709 13:17:13.913524 12293 raft_consensus.cc:2393] T c9544a57d3924366b84a3f2eda5e6140 P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9dbac62bc9fe4f96a350fe935538b020 in current term 1: Already voted for candidate c3d2109084424941b5c909182d5bfe07 in this term.
I20260709 13:17:13.914556 12110 leader_election.cc:304] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9dbac62bc9fe4f96a350fe935538b020; no voters: c3d2109084424941b5c909182d5bfe07, eaee9010d8714afea747f296df4322f0
I20260709 13:17:13.915300 12395 raft_consensus.cc:3060] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:13.920181 12395 raft_consensus.cc:2749] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:13.961299 12344 consensus_queue.cc:1048] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:13.964991 12338 consensus_queue.cc:1048] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:13.979094 12338 consensus_queue.cc:1048] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:13.980350 12410 consensus_queue.cc:1048] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:14.096187 12293 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a20e6b89add4f61bb32e0148c51271c"
dest_uuid: "eaee9010d8714afea747f296df4322f0"
mode: GRACEFUL
new_leader_uuid: "9dbac62bc9fe4f96a350fe935538b020"
 from {username='slave'} at 127.0.0.1:49270
I20260709 13:17:14.096932 12293 raft_consensus.cc:606] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 1 LEADER]: Received request to transfer leadership to TS 9dbac62bc9fe4f96a350fe935538b020
I20260709 13:17:14.102954 12293 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5066daaafb345b68502e7a69ce523da"
dest_uuid: "eaee9010d8714afea747f296df4322f0"
mode: GRACEFUL
new_leader_uuid: "9dbac62bc9fe4f96a350fe935538b020"
 from {username='slave'} at 127.0.0.1:49270
I20260709 13:17:14.103505 12293 raft_consensus.cc:606] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [term 1 LEADER]: Received request to transfer leadership to TS 9dbac62bc9fe4f96a350fe935538b020
I20260709 13:17:14.123369 12220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d830eb2a7b547f38ce277c1a68da16e"
dest_uuid: "c3d2109084424941b5c909182d5bfe07"
mode: GRACEFUL
new_leader_uuid: "9dbac62bc9fe4f96a350fe935538b020"
 from {username='slave'} at 127.0.0.1:54998
I20260709 13:17:14.123924 12220 raft_consensus.cc:606] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [term 1 LEADER]: Received request to transfer leadership to TS 9dbac62bc9fe4f96a350fe935538b020
I20260709 13:17:14.140066 12145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1488366d31f84925b97ed8f1843b957f"
dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
mode: GRACEFUL
new_leader_uuid: "c3d2109084424941b5c909182d5bfe07"
 from {username='slave'} at 127.0.0.1:44500
I20260709 13:17:14.140587 12145 raft_consensus.cc:606] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 1 LEADER]: Received request to transfer leadership to TS c3d2109084424941b5c909182d5bfe07
I20260709 13:17:14.149524 12293 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66472167c0814930aaa3398db1d2a421"
dest_uuid: "eaee9010d8714afea747f296df4322f0"
mode: GRACEFUL
new_leader_uuid: "c3d2109084424941b5c909182d5bfe07"
 from {username='slave'} at 127.0.0.1:49270
I20260709 13:17:14.150141 12293 raft_consensus.cc:606] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [term 1 LEADER]: Received request to transfer leadership to TS c3d2109084424941b5c909182d5bfe07
I20260709 13:17:14.155186 12145 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee84f8cd8c0242f3ba7a3b0dda942288"
dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
mode: GRACEFUL
new_leader_uuid: "eaee9010d8714afea747f296df4322f0"
 from {username='slave'} at 127.0.0.1:44500
I20260709 13:17:14.155717 12145 raft_consensus.cc:606] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [term 1 LEADER]: Received request to transfer leadership to TS eaee9010d8714afea747f296df4322f0
I20260709 13:17:14.159340 12395 consensus_queue.cc:1048] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:14.166254 12357 raft_consensus.cc:993] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020: : Instructing follower eaee9010d8714afea747f296df4322f0 to start an election
I20260709 13:17:14.166734 12394 raft_consensus.cc:1081] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [term 1 LEADER]: Signalling peer eaee9010d8714afea747f296df4322f0 to start an election
I20260709 13:17:14.168329 12294 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee84f8cd8c0242f3ba7a3b0dda942288"
dest_uuid: "eaee9010d8714afea747f296df4322f0"
 from {username='slave'} at 127.0.0.1:49248
I20260709 13:17:14.168941 12294 raft_consensus.cc:493] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:14.169329 12294 raft_consensus.cc:3060] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:14.171628 12341 consensus_queue.cc:1048] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:14.176782 12294 raft_consensus.cc:515] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:14.178236 12293 raft_consensus.cc:1240] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Rejecting Update request from peer 9dbac62bc9fe4f96a350fe935538b020 for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:14.178265 12410 consensus_queue.cc:1048] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:17:14.181608 12394 consensus_queue.cc:1059] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:14.182346 12341 raft_consensus.cc:3055] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:14.182682 12341 raft_consensus.cc:740] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Running, Role: LEADER
I20260709 13:17:14.183455 12341 consensus_queue.cc:260] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:14.184201 12220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee84f8cd8c0242f3ba7a3b0dda942288" candidate_uuid: "eaee9010d8714afea747f296df4322f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c3d2109084424941b5c909182d5bfe07"
I20260709 13:17:14.184486 12341 raft_consensus.cc:3060] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:14.184810 12220 raft_consensus.cc:3060] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:14.186620 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee84f8cd8c0242f3ba7a3b0dda942288" candidate_uuid: "eaee9010d8714afea747f296df4322f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
I20260709 13:17:14.190811 12220 raft_consensus.cc:2468] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaee9010d8714afea747f296df4322f0 in term 2.
I20260709 13:17:14.192305 12259 leader_election.cc:304] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3d2109084424941b5c909182d5bfe07, eaee9010d8714afea747f296df4322f0; no voters: 
I20260709 13:17:14.192986 12294 leader_election.cc:290] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 2 election: Requested vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197)
I20260709 13:17:14.193312 12410 raft_consensus.cc:2804] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:14.193883 12410 raft_consensus.cc:697] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0 [term 2 LEADER]: Becoming Leader. State: Replica: eaee9010d8714afea747f296df4322f0, State: Running, Role: LEADER
I20260709 13:17:14.194857 12410 consensus_queue.cc:237] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:14.198985 12344 raft_consensus.cc:993] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0: : Instructing follower 9dbac62bc9fe4f96a350fe935538b020 to start an election
I20260709 13:17:14.199446 12344 raft_consensus.cc:1081] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [term 1 LEADER]: Signalling peer 9dbac62bc9fe4f96a350fe935538b020 to start an election
I20260709 13:17:14.203301 12380 consensus_queue.cc:1048] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:14.208127 12143 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b5066daaafb345b68502e7a69ce523da"
dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
 from {username='slave'} at 127.0.0.1:44474
I20260709 13:17:14.208468 12380 consensus_queue.cc:1048] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:14.208871 12143 raft_consensus.cc:493] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:14.208153 12041 catalog_manager.cc:5697] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0 reported cstate change: term changed from 1 to 2, leader changed from 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65) to eaee9010d8714afea747f296df4322f0 (127.11.63.67). New cstate: current_term: 2 leader_uuid: "eaee9010d8714afea747f296df4322f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:14.209522 12143 raft_consensus.cc:3060] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:14.209530 12144 raft_consensus.cc:2468] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaee9010d8714afea747f296df4322f0 in term 2.
I20260709 13:17:14.219112 12143 raft_consensus.cc:515] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:14.223966 12219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5066daaafb345b68502e7a69ce523da" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c3d2109084424941b5c909182d5bfe07"
I20260709 13:17:14.229130 12294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5066daaafb345b68502e7a69ce523da" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eaee9010d8714afea747f296df4322f0"
I20260709 13:17:14.230003 12294 raft_consensus.cc:3055] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:14.230389 12294 raft_consensus.cc:740] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eaee9010d8714afea747f296df4322f0, State: Running, Role: LEADER
I20260709 13:17:14.231304 12294 consensus_queue.cc:260] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:14.232321 12294 raft_consensus.cc:3060] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:14.244971 12294 raft_consensus.cc:2468] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 2.
I20260709 13:17:14.228145 12143 leader_election.cc:290] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 2 election: Requested vote from peers c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:14.246236 12341 consensus_queue.cc:1048] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:14.246594 12110 leader_election.cc:304] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9dbac62bc9fe4f96a350fe935538b020, eaee9010d8714afea747f296df4322f0; no voters: c3d2109084424941b5c909182d5bfe07
I20260709 13:17:14.250128 12394 raft_consensus.cc:2804] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:14.259110 12380 raft_consensus.cc:993] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0: : Instructing follower 9dbac62bc9fe4f96a350fe935538b020 to start an election
I20260709 13:17:14.259608 12380 raft_consensus.cc:1081] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 1 LEADER]: Signalling peer 9dbac62bc9fe4f96a350fe935538b020 to start an election
I20260709 13:17:14.250845 12394 raft_consensus.cc:697] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [term 2 LEADER]: Becoming Leader. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Running, Role: LEADER
I20260709 13:17:14.261305 12394 consensus_queue.cc:237] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:14.265874 12143 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a20e6b89add4f61bb32e0148c51271c"
dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
 from {username='slave'} at 127.0.0.1:44474
I20260709 13:17:14.266526 12143 raft_consensus.cc:493] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:14.266965 12143 raft_consensus.cc:3060] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:14.271329 12380 consensus_queue.cc:1048] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:14.274565 12042 catalog_manager.cc:5697] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 reported cstate change: term changed from 1 to 2, leader changed from eaee9010d8714afea747f296df4322f0 (127.11.63.67) to 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65). New cstate: current_term: 2 leader_uuid: "9dbac62bc9fe4f96a350fe935538b020" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:14.275938 12143 raft_consensus.cc:515] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:14.281471 12143 leader_election.cc:290] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 2 election: Requested vote from peers c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:14.282159 12394 consensus_queue.cc:1048] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:14.283881 12219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a20e6b89add4f61bb32e0148c51271c" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c3d2109084424941b5c909182d5bfe07"
I20260709 13:17:14.285539 12293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a20e6b89add4f61bb32e0148c51271c" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eaee9010d8714afea747f296df4322f0"
I20260709 13:17:14.286100 12293 raft_consensus.cc:3055] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:14.286418 12293 raft_consensus.cc:740] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eaee9010d8714afea747f296df4322f0, State: Running, Role: LEADER
I20260709 13:17:14.287212 12293 consensus_queue.cc:260] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:14.290709 12293 raft_consensus.cc:3060] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:14.299043 12293 raft_consensus.cc:2468] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 2.
I20260709 13:17:14.301344 12110 leader_election.cc:304] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9dbac62bc9fe4f96a350fe935538b020, eaee9010d8714afea747f296df4322f0; no voters: c3d2109084424941b5c909182d5bfe07
I20260709 13:17:14.302718 12394 raft_consensus.cc:2804] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:14.303225 12394 raft_consensus.cc:697] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 2 LEADER]: Becoming Leader. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Running, Role: LEADER
I20260709 13:17:14.304207 12394 consensus_queue.cc:237] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:14.306805 12357 raft_consensus.cc:993] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020: : Instructing follower c3d2109084424941b5c909182d5bfe07 to start an election
I20260709 13:17:14.307245 12357 raft_consensus.cc:1081] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 1 LEADER]: Signalling peer c3d2109084424941b5c909182d5bfe07 to start an election
I20260709 13:17:14.308933 12219 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1488366d31f84925b97ed8f1843b957f"
dest_uuid: "c3d2109084424941b5c909182d5bfe07"
 from {username='slave'} at 127.0.0.1:54982
I20260709 13:17:14.309406 12219 raft_consensus.cc:493] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:14.309835 12219 raft_consensus.cc:3060] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:14.313679 12040 catalog_manager.cc:5697] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 reported cstate change: term changed from 1 to 2, leader changed from eaee9010d8714afea747f296df4322f0 (127.11.63.67) to 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65). New cstate: current_term: 2 leader_uuid: "9dbac62bc9fe4f96a350fe935538b020" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:14.317272 12219 raft_consensus.cc:515] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:14.319303 12219 leader_election.cc:290] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 2 election: Requested vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:14.320165 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1488366d31f84925b97ed8f1843b957f" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
I20260709 13:17:14.320443 12293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1488366d31f84925b97ed8f1843b957f" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eaee9010d8714afea747f296df4322f0"
I20260709 13:17:14.320772 12143 raft_consensus.cc:3055] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:14.321026 12293 raft_consensus.cc:3060] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:14.321151 12143 raft_consensus.cc:740] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Running, Role: LEADER
I20260709 13:17:14.322547 12143 consensus_queue.cc:260] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:14.325344 12143 raft_consensus.cc:3060] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:14.327724 12293 raft_consensus.cc:2468] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3d2109084424941b5c909182d5bfe07 in term 2.
I20260709 13:17:14.328873 12185 leader_election.cc:304] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3d2109084424941b5c909182d5bfe07, eaee9010d8714afea747f296df4322f0; no voters: 
I20260709 13:17:14.329689 12366 raft_consensus.cc:2804] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:14.330168 12366 raft_consensus.cc:697] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 2 LEADER]: Becoming Leader. State: Replica: c3d2109084424941b5c909182d5bfe07, State: Running, Role: LEADER
I20260709 13:17:14.330973 12366 consensus_queue.cc:237] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:14.332459 12143 raft_consensus.cc:2468] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3d2109084424941b5c909182d5bfe07 in term 2.
I20260709 13:17:14.339064 12040 catalog_manager.cc:5697] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 reported cstate change: term changed from 1 to 2, leader changed from 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65) to c3d2109084424941b5c909182d5bfe07 (127.11.63.66). New cstate: current_term: 2 leader_uuid: "c3d2109084424941b5c909182d5bfe07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:14.408793 12366 consensus_queue.cc:1048] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:14.416193 12368 raft_consensus.cc:993] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07: : Instructing follower 9dbac62bc9fe4f96a350fe935538b020 to start an election
I20260709 13:17:14.416625 12338 raft_consensus.cc:1081] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [term 1 LEADER]: Signalling peer 9dbac62bc9fe4f96a350fe935538b020 to start an election
I20260709 13:17:14.418737 12144 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d830eb2a7b547f38ce277c1a68da16e"
dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
 from {username='slave'} at 127.0.0.1:44486
I20260709 13:17:14.421439 12368 consensus_queue.cc:1048] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:14.422755 12144 raft_consensus.cc:493] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:14.423203 12144 raft_consensus.cc:3060] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:14.431483 12144 raft_consensus.cc:515] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:14.435245 12144 leader_election.cc:290] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 2 election: Requested vote from peers c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:14.435946 12219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d830eb2a7b547f38ce277c1a68da16e" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c3d2109084424941b5c909182d5bfe07"
I20260709 13:17:14.436344 12293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d830eb2a7b547f38ce277c1a68da16e" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eaee9010d8714afea747f296df4322f0"
I20260709 13:17:14.436635 12219 raft_consensus.cc:3055] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:14.436964 12293 raft_consensus.cc:3060] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:14.437096 12219 raft_consensus.cc:740] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c3d2109084424941b5c909182d5bfe07, State: Running, Role: LEADER
I20260709 13:17:14.438055 12219 consensus_queue.cc:260] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:14.439038 12219 raft_consensus.cc:3060] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:14.442204 12293 raft_consensus.cc:2468] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 2.
I20260709 13:17:14.443322 12110 leader_election.cc:304] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dbac62bc9fe4f96a350fe935538b020, eaee9010d8714afea747f296df4322f0; no voters: 
I20260709 13:17:14.444136 12341 raft_consensus.cc:2804] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:14.444453 12219 raft_consensus.cc:2468] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 2.
I20260709 13:17:14.444697 12341 raft_consensus.cc:697] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020 [term 2 LEADER]: Becoming Leader. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Running, Role: LEADER
I20260709 13:17:14.446237 12341 consensus_queue.cc:237] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:14.452710 12040 catalog_manager.cc:5697] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020 reported cstate change: term changed from 1 to 2, leader changed from c3d2109084424941b5c909182d5bfe07 (127.11.63.66) to 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65). New cstate: current_term: 2 leader_uuid: "9dbac62bc9fe4f96a350fe935538b020" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:14.532480 12410 raft_consensus.cc:993] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0: : Instructing follower c3d2109084424941b5c909182d5bfe07 to start an election
I20260709 13:17:14.532980 12380 raft_consensus.cc:1081] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [term 1 LEADER]: Signalling peer c3d2109084424941b5c909182d5bfe07 to start an election
I20260709 13:17:14.534468 12219 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "66472167c0814930aaa3398db1d2a421"
dest_uuid: "c3d2109084424941b5c909182d5bfe07"
 from {username='slave'} at 127.0.0.1:54970
I20260709 13:17:14.534901 12219 raft_consensus.cc:493] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:14.535162 12219 raft_consensus.cc:3060] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:14.539604 12219 raft_consensus.cc:515] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:14.541208 12219 leader_election.cc:290] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 2 election: Requested vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:14.542240 12144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66472167c0814930aaa3398db1d2a421" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
I20260709 13:17:14.542294 12293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66472167c0814930aaa3398db1d2a421" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eaee9010d8714afea747f296df4322f0"
I20260709 13:17:14.542888 12293 raft_consensus.cc:3055] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:14.542863 12144 raft_consensus.cc:3060] T 66472167c0814930aaa3398db1d2a421 P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:14.543311 12293 raft_consensus.cc:740] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eaee9010d8714afea747f296df4322f0, State: Running, Role: LEADER
I20260709 13:17:14.544075 12293 consensus_queue.cc:260] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:14.544919 12293 raft_consensus.cc:3060] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:14.547915 12144 raft_consensus.cc:2468] T 66472167c0814930aaa3398db1d2a421 P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3d2109084424941b5c909182d5bfe07 in term 2.
I20260709 13:17:14.549085 12183 leader_election.cc:304] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dbac62bc9fe4f96a350fe935538b020, c3d2109084424941b5c909182d5bfe07; no voters: 
I20260709 13:17:14.549973 12338 raft_consensus.cc:2804] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:14.550420 12338 raft_consensus.cc:697] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07 [term 2 LEADER]: Becoming Leader. State: Replica: c3d2109084424941b5c909182d5bfe07, State: Running, Role: LEADER
I20260709 13:17:14.550434 12293 raft_consensus.cc:2468] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3d2109084424941b5c909182d5bfe07 in term 2.
I20260709 13:17:14.551306 12338 consensus_queue.cc:237] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:14.557197 12040 catalog_manager.cc:5697] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07 reported cstate change: term changed from 1 to 2, leader changed from eaee9010d8714afea747f296df4322f0 (127.11.63.67) to c3d2109084424941b5c909182d5bfe07 (127.11.63.66). New cstate: current_term: 2 leader_uuid: "c3d2109084424941b5c909182d5bfe07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:14.726701 12293 raft_consensus.cc:1275] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Refusing update from remote peer 9dbac62bc9fe4f96a350fe935538b020: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:14.728032 12341 consensus_queue.cc:1048] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:14.735877 12219 raft_consensus.cc:3060] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:14.744760 12219 raft_consensus.cc:1275] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Refusing update from remote peer 9dbac62bc9fe4f96a350fe935538b020: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:14.746789 12394 consensus_queue.cc:1048] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:14.756553 12293 raft_consensus.cc:1275] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Refusing update from remote peer 9dbac62bc9fe4f96a350fe935538b020: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:14.757686 12144 raft_consensus.cc:1275] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Refusing update from remote peer eaee9010d8714afea747f296df4322f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:14.758224 12394 consensus_queue.cc:1048] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:14.759034 12380 consensus_queue.cc:1048] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:14.768838 12219 raft_consensus.cc:3060] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:14.772933 12220 raft_consensus.cc:1275] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Refusing update from remote peer eaee9010d8714afea747f296df4322f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:14.774900 12380 consensus_queue.cc:1048] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:14.780673 12219 raft_consensus.cc:1275] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Refusing update from remote peer 9dbac62bc9fe4f96a350fe935538b020: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:14.782480 12395 consensus_queue.cc:1048] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:14.830636 12293 raft_consensus.cc:1275] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Refusing update from remote peer c3d2109084424941b5c909182d5bfe07: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:14.832829 12338 consensus_queue.cc:1048] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:14.835021 12219 raft_consensus.cc:1275] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Refusing update from remote peer 9dbac62bc9fe4f96a350fe935538b020: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:14.836447 12395 consensus_queue.cc:1048] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:14.847834 12144 raft_consensus.cc:1275] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Refusing update from remote peer c3d2109084424941b5c909182d5bfe07: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:14.850646 12294 raft_consensus.cc:1275] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Refusing update from remote peer 9dbac62bc9fe4f96a350fe935538b020: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:14.854444 12371 consensus_queue.cc:1048] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:14.857196 12394 consensus_queue.cc:1048] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:14.978070 12144 raft_consensus.cc:1275] T 66472167c0814930aaa3398db1d2a421 P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Refusing update from remote peer c3d2109084424941b5c909182d5bfe07: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:14.979394 12371 consensus_queue.cc:1048] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:14.988173 12293 raft_consensus.cc:1275] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Refusing update from remote peer c3d2109084424941b5c909182d5bfe07: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:14.989822 12368 consensus_queue.cc:1048] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:17.191159 11517 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:17:17.192124 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:17.210641 12143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a20e6b89add4f61bb32e0148c51271c"
dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
mode: GRACEFUL
new_leader_uuid: "eaee9010d8714afea747f296df4322f0"
 from {username='slave'} at 127.0.0.1:44508
I20260709 13:17:17.211633 12143 raft_consensus.cc:606] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 2 LEADER]: Received request to transfer leadership to TS eaee9010d8714afea747f296df4322f0
I20260709 13:17:17.215152 12143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c20e9b895ed4401e97300148b98276e3"
dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
mode: GRACEFUL
new_leader_uuid: "eaee9010d8714afea747f296df4322f0"
 from {username='slave'} at 127.0.0.1:44508
I20260709 13:17:17.215783 12143 raft_consensus.cc:606] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 1 LEADER]: Received request to transfer leadership to TS eaee9010d8714afea747f296df4322f0
I20260709 13:17:17.216950 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:17.217492 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:17.577185 12436 raft_consensus.cc:993] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020: : Instructing follower eaee9010d8714afea747f296df4322f0 to start an election
I20260709 13:17:17.577711 12431 raft_consensus.cc:1081] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 2 LEADER]: Signalling peer eaee9010d8714afea747f296df4322f0 to start an election
I20260709 13:17:17.579448 12293 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a20e6b89add4f61bb32e0148c51271c"
dest_uuid: "eaee9010d8714afea747f296df4322f0"
 from {username='slave'} at 127.0.0.1:49248
I20260709 13:17:17.579879 12293 raft_consensus.cc:493] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:17.580147 12293 raft_consensus.cc:3060] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:17.584337 12293 raft_consensus.cc:515] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:17.585963 12293 leader_election.cc:290] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 3 election: Requested vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197)
I20260709 13:17:17.586771 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a20e6b89add4f61bb32e0148c51271c" candidate_uuid: "eaee9010d8714afea747f296df4322f0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
I20260709 13:17:17.586998 12219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a20e6b89add4f61bb32e0148c51271c" candidate_uuid: "eaee9010d8714afea747f296df4322f0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c3d2109084424941b5c909182d5bfe07"
I20260709 13:17:17.587560 12219 raft_consensus.cc:3060] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:17.587555 12143 raft_consensus.cc:3055] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:17.588107 12143 raft_consensus.cc:740] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Running, Role: LEADER
I20260709 13:17:17.588922 12143 consensus_queue.cc:260] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:17.590157 12143 raft_consensus.cc:3060] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:17.594736 12219 raft_consensus.cc:2468] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaee9010d8714afea747f296df4322f0 in term 3.
I20260709 13:17:17.596107 12259 leader_election.cc:304] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3d2109084424941b5c909182d5bfe07, eaee9010d8714afea747f296df4322f0; no voters: 
I20260709 13:17:17.596952 12432 raft_consensus.cc:2804] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:17.597084 12143 raft_consensus.cc:2468] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaee9010d8714afea747f296df4322f0 in term 3.
I20260709 13:17:17.598099 12432 raft_consensus.cc:697] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 3 LEADER]: Becoming Leader. State: Replica: eaee9010d8714afea747f296df4322f0, State: Running, Role: LEADER
I20260709 13:17:17.598870 12432 consensus_queue.cc:237] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:17.602560 12436 raft_consensus.cc:993] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020: : Instructing follower eaee9010d8714afea747f296df4322f0 to start an election
I20260709 13:17:17.603123 12431 raft_consensus.cc:1081] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 1 LEADER]: Signalling peer eaee9010d8714afea747f296df4322f0 to start an election
I20260709 13:17:17.604707 12293 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c20e9b895ed4401e97300148b98276e3"
dest_uuid: "eaee9010d8714afea747f296df4322f0"
 from {username='slave'} at 127.0.0.1:49248
I20260709 13:17:17.605177 12293 raft_consensus.cc:493] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:17.605489 12293 raft_consensus.cc:3060] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:17.609202 12040 catalog_manager.cc:5697] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 reported cstate change: term changed from 2 to 3, leader changed from 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65) to eaee9010d8714afea747f296df4322f0 (127.11.63.67). New cstate: current_term: 3 leader_uuid: "eaee9010d8714afea747f296df4322f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:17.611572 12293 raft_consensus.cc:515] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:17.614447 12219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20e9b895ed4401e97300148b98276e3" candidate_uuid: "eaee9010d8714afea747f296df4322f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c3d2109084424941b5c909182d5bfe07"
I20260709 13:17:17.614944 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20e9b895ed4401e97300148b98276e3" candidate_uuid: "eaee9010d8714afea747f296df4322f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
I20260709 13:17:17.615104 12219 raft_consensus.cc:3060] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:17.615584 12143 raft_consensus.cc:3055] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:17.615904 12143 raft_consensus.cc:740] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Running, Role: LEADER
I20260709 13:17:17.616756 12143 consensus_queue.cc:260] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:17.617770 12143 raft_consensus.cc:3060] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:17.624074 12219 raft_consensus.cc:2468] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaee9010d8714afea747f296df4322f0 in term 2.
I20260709 13:17:17.614399 12293 leader_election.cc:290] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 2 election: Requested vote from peers c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197), 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933)
I20260709 13:17:17.625775 12259 leader_election.cc:304] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3d2109084424941b5c909182d5bfe07, eaee9010d8714afea747f296df4322f0; no voters: 
I20260709 13:17:17.626067 12143 raft_consensus.cc:2468] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaee9010d8714afea747f296df4322f0 in term 2.
I20260709 13:17:17.626842 12432 raft_consensus.cc:2804] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:17.627342 12432 raft_consensus.cc:697] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [term 2 LEADER]: Becoming Leader. State: Replica: eaee9010d8714afea747f296df4322f0, State: Running, Role: LEADER
I20260709 13:17:17.628233 12432 consensus_queue.cc:237] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:17.636875 12040 catalog_manager.cc:5697] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 reported cstate change: term changed from 1 to 2, leader changed from 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65) to eaee9010d8714afea747f296df4322f0 (127.11.63.67). New cstate: current_term: 2 leader_uuid: "eaee9010d8714afea747f296df4322f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:18.029109 12220 raft_consensus.cc:1275] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Refusing update from remote peer eaee9010d8714afea747f296df4322f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:18.030583 12432 consensus_queue.cc:1048] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:18.040033 12143 raft_consensus.cc:1275] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Refusing update from remote peer eaee9010d8714afea747f296df4322f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:18.041478 12437 consensus_queue.cc:1048] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:18.063609 12220 raft_consensus.cc:1275] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [term 3 FOLLOWER]: Refusing update from remote peer eaee9010d8714afea747f296df4322f0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:18.065093 12437 consensus_queue.cc:1048] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:18.075932 12143 raft_consensus.cc:1275] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 3 FOLLOWER]: Refusing update from remote peer eaee9010d8714afea747f296df4322f0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:18.077545 12437 consensus_queue.cc:1048] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:18.218596 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:18.229367 12293 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c20e9b895ed4401e97300148b98276e3"
dest_uuid: "eaee9010d8714afea747f296df4322f0"
mode: GRACEFUL
new_leader_uuid: "9dbac62bc9fe4f96a350fe935538b020"
 from {username='slave'} at 127.0.0.1:49270
I20260709 13:17:18.229931 12293 raft_consensus.cc:606] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [term 2 LEADER]: Received request to transfer leadership to TS 9dbac62bc9fe4f96a350fe935538b020
I20260709 13:17:18.235282 12293 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a20e6b89add4f61bb32e0148c51271c"
dest_uuid: "eaee9010d8714afea747f296df4322f0"
mode: GRACEFUL
new_leader_uuid: "9dbac62bc9fe4f96a350fe935538b020"
 from {username='slave'} at 127.0.0.1:49270
I20260709 13:17:18.235776 12293 raft_consensus.cc:606] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 3 LEADER]: Received request to transfer leadership to TS 9dbac62bc9fe4f96a350fe935538b020
I20260709 13:17:18.249338 12220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9544a57d3924366b84a3f2eda5e6140"
dest_uuid: "c3d2109084424941b5c909182d5bfe07"
mode: GRACEFUL
new_leader_uuid: "9dbac62bc9fe4f96a350fe935538b020"
 from {username='slave'} at 127.0.0.1:54998
I20260709 13:17:18.249964 12220 raft_consensus.cc:606] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [term 1 LEADER]: Received request to transfer leadership to TS 9dbac62bc9fe4f96a350fe935538b020
I20260709 13:17:18.255085 12220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1488366d31f84925b97ed8f1843b957f"
dest_uuid: "c3d2109084424941b5c909182d5bfe07"
mode: GRACEFUL
new_leader_uuid: "9dbac62bc9fe4f96a350fe935538b020"
 from {username='slave'} at 127.0.0.1:54998
I20260709 13:17:18.255621 12220 raft_consensus.cc:606] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 2 LEADER]: Received request to transfer leadership to TS 9dbac62bc9fe4f96a350fe935538b020
I20260709 13:17:18.260030 12220 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d9dbc6498c743e1a2fda71e38f4c3db"
dest_uuid: "c3d2109084424941b5c909182d5bfe07"
mode: GRACEFUL
new_leader_uuid: "9dbac62bc9fe4f96a350fe935538b020"
 from {username='slave'} at 127.0.0.1:54998
I20260709 13:17:18.260548 12220 raft_consensus.cc:606] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [term 1 LEADER]: Received request to transfer leadership to TS 9dbac62bc9fe4f96a350fe935538b020
I20260709 13:17:18.382404 12435 raft_consensus.cc:993] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07: : Instructing follower 9dbac62bc9fe4f96a350fe935538b020 to start an election
I20260709 13:17:18.382845 12435 raft_consensus.cc:1081] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [term 1 LEADER]: Signalling peer 9dbac62bc9fe4f96a350fe935538b020 to start an election
I20260709 13:17:18.384465 12143 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0d9dbc6498c743e1a2fda71e38f4c3db"
dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
 from {username='slave'} at 127.0.0.1:44486
I20260709 13:17:18.384972 12143 raft_consensus.cc:493] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:18.385238 12143 raft_consensus.cc:3060] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:18.390142 12143 raft_consensus.cc:515] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:18.391885 12143 leader_election.cc:290] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 2 election: Requested vote from peers c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:18.392752 12220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d9dbc6498c743e1a2fda71e38f4c3db" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c3d2109084424941b5c909182d5bfe07"
I20260709 13:17:18.392930 12293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d9dbc6498c743e1a2fda71e38f4c3db" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eaee9010d8714afea747f296df4322f0"
I20260709 13:17:18.393586 12293 raft_consensus.cc:3060] T 0d9dbc6498c743e1a2fda71e38f4c3db P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:18.393589 12220 raft_consensus.cc:3055] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:18.394093 12220 raft_consensus.cc:740] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c3d2109084424941b5c909182d5bfe07, State: Running, Role: LEADER
I20260709 13:17:18.394908 12220 consensus_queue.cc:260] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:18.396047 12220 raft_consensus.cc:3060] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:18.400207 12293 raft_consensus.cc:2468] T 0d9dbc6498c743e1a2fda71e38f4c3db P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 2.
I20260709 13:17:18.401449 12110 leader_election.cc:304] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dbac62bc9fe4f96a350fe935538b020, eaee9010d8714afea747f296df4322f0; no voters: 
I20260709 13:17:18.402278 12431 raft_consensus.cc:2804] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:18.402874 12220 raft_consensus.cc:2468] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 2.
I20260709 13:17:18.403344 12431 raft_consensus.cc:697] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020 [term 2 LEADER]: Becoming Leader. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Running, Role: LEADER
I20260709 13:17:18.404313 12431 consensus_queue.cc:237] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:18.411055 12040 catalog_manager.cc:5697] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020 reported cstate change: term changed from 1 to 2, leader changed from c3d2109084424941b5c909182d5bfe07 (127.11.63.66) to 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65). New cstate: current_term: 2 leader_uuid: "9dbac62bc9fe4f96a350fe935538b020" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:18.423297 12450 raft_consensus.cc:993] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07: : Instructing follower 9dbac62bc9fe4f96a350fe935538b020 to start an election
I20260709 13:17:18.423739 12435 raft_consensus.cc:1081] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [term 1 LEADER]: Signalling peer 9dbac62bc9fe4f96a350fe935538b020 to start an election
I20260709 13:17:18.425151 12143 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c9544a57d3924366b84a3f2eda5e6140"
dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
 from {username='slave'} at 127.0.0.1:44486
I20260709 13:17:18.425834 12143 raft_consensus.cc:493] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:18.426158 12143 raft_consensus.cc:3060] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:18.431448 12143 raft_consensus.cc:515] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:18.432981 12143 leader_election.cc:290] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 2 election: Requested vote from peers c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:18.433991 12220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9544a57d3924366b84a3f2eda5e6140" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c3d2109084424941b5c909182d5bfe07"
I20260709 13:17:18.434334 12293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9544a57d3924366b84a3f2eda5e6140" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eaee9010d8714afea747f296df4322f0"
I20260709 13:17:18.434695 12220 raft_consensus.cc:3055] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:18.434855 12293 raft_consensus.cc:3060] T c9544a57d3924366b84a3f2eda5e6140 P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:18.435075 12220 raft_consensus.cc:740] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c3d2109084424941b5c909182d5bfe07, State: Running, Role: LEADER
I20260709 13:17:18.435963 12220 consensus_queue.cc:260] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:18.437268 12220 raft_consensus.cc:3060] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:18.442085 12293 raft_consensus.cc:2468] T c9544a57d3924366b84a3f2eda5e6140 P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 2.
I20260709 13:17:18.443331 12110 leader_election.cc:304] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dbac62bc9fe4f96a350fe935538b020, eaee9010d8714afea747f296df4322f0; no voters: 
I20260709 13:17:18.444139 12431 raft_consensus.cc:2804] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:18.444468 12220 raft_consensus.cc:2468] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 2.
I20260709 13:17:18.444597 12431 raft_consensus.cc:697] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020 [term 2 LEADER]: Becoming Leader. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Running, Role: LEADER
I20260709 13:17:18.445607 12431 consensus_queue.cc:237] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:18.452692 12040 catalog_manager.cc:5697] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020 reported cstate change: term changed from 1 to 2, leader changed from c3d2109084424941b5c909182d5bfe07 (127.11.63.66) to 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65). New cstate: current_term: 2 leader_uuid: "9dbac62bc9fe4f96a350fe935538b020" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:18.624631 12459 raft_consensus.cc:993] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0: : Instructing follower 9dbac62bc9fe4f96a350fe935538b020 to start an election
I20260709 13:17:18.625144 12460 raft_consensus.cc:1081] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 3 LEADER]: Signalling peer 9dbac62bc9fe4f96a350fe935538b020 to start an election
I20260709 13:17:18.626816 12143 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a20e6b89add4f61bb32e0148c51271c"
dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
 from {username='slave'} at 127.0.0.1:44474
I20260709 13:17:18.627357 12143 raft_consensus.cc:493] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:18.627692 12143 raft_consensus.cc:3060] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:18.632375 12143 raft_consensus.cc:515] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:18.634158 12143 leader_election.cc:290] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 4 election: Requested vote from peers c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:18.635311 12293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a20e6b89add4f61bb32e0148c51271c" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "eaee9010d8714afea747f296df4322f0"
I20260709 13:17:18.635182 12220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a20e6b89add4f61bb32e0148c51271c" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c3d2109084424941b5c909182d5bfe07"
I20260709 13:17:18.635881 12220 raft_consensus.cc:3060] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:18.635834 12293 raft_consensus.cc:3055] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:18.636353 12293 raft_consensus.cc:740] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: eaee9010d8714afea747f296df4322f0, State: Running, Role: LEADER
I20260709 13:17:18.637081 12293 consensus_queue.cc:260] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:18.638181 12293 raft_consensus.cc:3060] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:18.643246 12220 raft_consensus.cc:2468] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 4.
I20260709 13:17:18.644575 12110 leader_election.cc:304] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dbac62bc9fe4f96a350fe935538b020, c3d2109084424941b5c909182d5bfe07; no voters: 
I20260709 13:17:18.645475 12431 raft_consensus.cc:2804] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:18.646289 12437 raft_consensus.cc:993] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0: : Instructing follower 9dbac62bc9fe4f96a350fe935538b020 to start an election
I20260709 13:17:18.646309 12293 raft_consensus.cc:2468] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 4.
I20260709 13:17:18.646494 12431 raft_consensus.cc:697] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 4 LEADER]: Becoming Leader. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Running, Role: LEADER
I20260709 13:17:18.646833 12437 raft_consensus.cc:1081] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [term 2 LEADER]: Signalling peer 9dbac62bc9fe4f96a350fe935538b020 to start an election
I20260709 13:17:18.647807 12431 consensus_queue.cc:237] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:18.648733 12143 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c20e9b895ed4401e97300148b98276e3"
dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
 from {username='slave'} at 127.0.0.1:44474
I20260709 13:17:18.649255 12143 raft_consensus.cc:493] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:18.649650 12143 raft_consensus.cc:3060] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:18.656073 12143 raft_consensus.cc:515] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:18.655608 12040 catalog_manager.cc:5697] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 reported cstate change: term changed from 3 to 4, leader changed from eaee9010d8714afea747f296df4322f0 (127.11.63.67) to 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65). New cstate: current_term: 4 leader_uuid: "9dbac62bc9fe4f96a350fe935538b020" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:18.658088 12143 leader_election.cc:290] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 3 election: Requested vote from peers c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:18.658780 12220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20e9b895ed4401e97300148b98276e3" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c3d2109084424941b5c909182d5bfe07"
I20260709 13:17:18.659047 12293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20e9b895ed4401e97300148b98276e3" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "eaee9010d8714afea747f296df4322f0"
I20260709 13:17:18.659309 12220 raft_consensus.cc:3060] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:18.659612 12293 raft_consensus.cc:3055] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:18.659917 12293 raft_consensus.cc:740] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eaee9010d8714afea747f296df4322f0, State: Running, Role: LEADER
I20260709 13:17:18.660609 12293 consensus_queue.cc:260] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:18.661545 12293 raft_consensus.cc:3060] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:18.666076 12220 raft_consensus.cc:2468] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 3.
I20260709 13:17:18.666535 12293 raft_consensus.cc:2468] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 3.
I20260709 13:17:18.666975 12110 leader_election.cc:304] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dbac62bc9fe4f96a350fe935538b020, c3d2109084424941b5c909182d5bfe07; no voters: 
I20260709 13:17:18.667879 12431 raft_consensus.cc:2804] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:18.668330 12431 raft_consensus.cc:697] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 3 LEADER]: Becoming Leader. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Running, Role: LEADER
I20260709 13:17:18.669042 12431 consensus_queue.cc:237] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:18.675060 12040 catalog_manager.cc:5697] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 reported cstate change: term changed from 2 to 3, leader changed from eaee9010d8714afea747f296df4322f0 (127.11.63.67) to 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65). New cstate: current_term: 3 leader_uuid: "9dbac62bc9fe4f96a350fe935538b020" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:18.744421 12450 raft_consensus.cc:993] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07: : Instructing follower 9dbac62bc9fe4f96a350fe935538b020 to start an election
I20260709 13:17:18.744958 12435 raft_consensus.cc:1081] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 2 LEADER]: Signalling peer 9dbac62bc9fe4f96a350fe935538b020 to start an election
I20260709 13:17:18.746822 12143 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1488366d31f84925b97ed8f1843b957f"
dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
 from {username='slave'} at 127.0.0.1:44486
I20260709 13:17:18.747462 12143 raft_consensus.cc:493] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:18.747792 12143 raft_consensus.cc:3060] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:18.753304 12143 raft_consensus.cc:515] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:18.754844 12143 leader_election.cc:290] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 3 election: Requested vote from peers c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:18.755637 12220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1488366d31f84925b97ed8f1843b957f" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c3d2109084424941b5c909182d5bfe07"
I20260709 13:17:18.755795 12293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1488366d31f84925b97ed8f1843b957f" candidate_uuid: "9dbac62bc9fe4f96a350fe935538b020" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "eaee9010d8714afea747f296df4322f0"
I20260709 13:17:18.756120 12220 raft_consensus.cc:3055] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:18.756273 12293 raft_consensus.cc:3060] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:18.756348 12220 raft_consensus.cc:740] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c3d2109084424941b5c909182d5bfe07, State: Running, Role: LEADER
I20260709 13:17:18.757088 12220 consensus_queue.cc:260] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:18.757879 12220 raft_consensus.cc:3060] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:18.760526 12293 raft_consensus.cc:2468] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 3.
I20260709 13:17:18.761432 12110 leader_election.cc:304] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dbac62bc9fe4f96a350fe935538b020, eaee9010d8714afea747f296df4322f0; no voters: 
I20260709 13:17:18.761919 12220 raft_consensus.cc:2468] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9dbac62bc9fe4f96a350fe935538b020 in term 3.
I20260709 13:17:18.762161 12431 raft_consensus.cc:2804] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:18.762624 12431 raft_consensus.cc:697] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 3 LEADER]: Becoming Leader. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Running, Role: LEADER
I20260709 13:17:18.763298 12431 consensus_queue.cc:237] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:18.770165 12040 catalog_manager.cc:5697] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 reported cstate change: term changed from 2 to 3, leader changed from c3d2109084424941b5c909182d5bfe07 (127.11.63.66) to 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65). New cstate: current_term: 3 leader_uuid: "9dbac62bc9fe4f96a350fe935538b020" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:18.844663 12293 raft_consensus.cc:1275] T 0d9dbc6498c743e1a2fda71e38f4c3db P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Refusing update from remote peer 9dbac62bc9fe4f96a350fe935538b020: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:18.846138 12431 consensus_queue.cc:1048] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:18.857038 12220 raft_consensus.cc:1275] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Refusing update from remote peer 9dbac62bc9fe4f96a350fe935538b020: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:18.858484 12464 consensus_queue.cc:1048] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:18.906407 12293 raft_consensus.cc:1275] T c9544a57d3924366b84a3f2eda5e6140 P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Refusing update from remote peer 9dbac62bc9fe4f96a350fe935538b020: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:18.907866 12464 consensus_queue.cc:1048] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:18.917853 12220 raft_consensus.cc:1275] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Refusing update from remote peer 9dbac62bc9fe4f96a350fe935538b020: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:18.919556 12431 consensus_queue.cc:1048] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:19.096371 12293 raft_consensus.cc:1275] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 4 FOLLOWER]: Refusing update from remote peer 9dbac62bc9fe4f96a350fe935538b020: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:19.097761 12453 consensus_queue.cc:1048] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:19.106459 12220 raft_consensus.cc:1275] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [term 4 FOLLOWER]: Refusing update from remote peer 9dbac62bc9fe4f96a350fe935538b020: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:19.107939 12469 consensus_queue.cc:1048] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:19.163154 12293 raft_consensus.cc:1275] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [term 3 FOLLOWER]: Refusing update from remote peer 9dbac62bc9fe4f96a350fe935538b020: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:19.164394 12469 consensus_queue.cc:1048] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:19.172583 12220 raft_consensus.cc:1275] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [term 3 FOLLOWER]: Refusing update from remote peer 9dbac62bc9fe4f96a350fe935538b020: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:19.174033 12469 consensus_queue.cc:1048] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:19.265002 12220 raft_consensus.cc:1275] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 3 FOLLOWER]: Refusing update from remote peer 9dbac62bc9fe4f96a350fe935538b020: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:19.266381 12469 consensus_queue.cc:1048] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:19.276854 12293 raft_consensus.cc:1275] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [term 3 FOLLOWER]: Refusing update from remote peer 9dbac62bc9fe4f96a350fe935538b020: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:19.280622 12469 consensus_queue.cc:1048] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:21.268297 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:21.277629 12143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a20e6b89add4f61bb32e0148c51271c"
dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
mode: GRACEFUL
new_leader_uuid: "eaee9010d8714afea747f296df4322f0"
 from {username='slave'} at 127.0.0.1:44508
I20260709 13:17:21.278398 12143 raft_consensus.cc:606] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 4 LEADER]: Received request to transfer leadership to TS eaee9010d8714afea747f296df4322f0
I20260709 13:17:21.281113 12143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1488366d31f84925b97ed8f1843b957f"
dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
mode: GRACEFUL
new_leader_uuid: "eaee9010d8714afea747f296df4322f0"
 from {username='slave'} at 127.0.0.1:44508
I20260709 13:17:21.281627 12143 raft_consensus.cc:606] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 3 LEADER]: Received request to transfer leadership to TS eaee9010d8714afea747f296df4322f0
I20260709 13:17:21.284186 12143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5066daaafb345b68502e7a69ce523da"
dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
mode: GRACEFUL
new_leader_uuid: "c3d2109084424941b5c909182d5bfe07"
 from {username='slave'} at 127.0.0.1:44508
I20260709 13:17:21.284670 12143 raft_consensus.cc:606] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [term 2 LEADER]: Received request to transfer leadership to TS c3d2109084424941b5c909182d5bfe07
I20260709 13:17:21.286978 12143 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c20e9b895ed4401e97300148b98276e3"
dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
mode: GRACEFUL
new_leader_uuid: "c3d2109084424941b5c909182d5bfe07"
 from {username='slave'} at 127.0.0.1:44508
I20260709 13:17:21.287489 12143 raft_consensus.cc:606] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 3 LEADER]: Received request to transfer leadership to TS c3d2109084424941b5c909182d5bfe07
I20260709 13:17:21.288553 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:17:21.289199 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:21.364014 12431 raft_consensus.cc:993] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020: : Instructing follower c3d2109084424941b5c909182d5bfe07 to start an election
I20260709 13:17:21.364575 12464 raft_consensus.cc:1081] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [term 2 LEADER]: Signalling peer c3d2109084424941b5c909182d5bfe07 to start an election
I20260709 13:17:21.366381 12219 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b5066daaafb345b68502e7a69ce523da"
dest_uuid: "c3d2109084424941b5c909182d5bfe07"
 from {username='slave'} at 127.0.0.1:54982
I20260709 13:17:21.366819 12219 raft_consensus.cc:493] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:21.367087 12219 raft_consensus.cc:3060] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:21.372823 12219 raft_consensus.cc:515] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:21.374589 12219 leader_election.cc:290] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 3 election: Requested vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:21.375882 12294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5066daaafb345b68502e7a69ce523da" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "eaee9010d8714afea747f296df4322f0"
I20260709 13:17:21.376504 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5066daaafb345b68502e7a69ce523da" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
I20260709 13:17:21.376664 12294 raft_consensus.cc:3060] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:21.377091 12143 raft_consensus.cc:3055] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:21.377416 12143 raft_consensus.cc:740] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Running, Role: LEADER
I20260709 13:17:21.378371 12143 consensus_queue.cc:260] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:21.379726 12143 raft_consensus.cc:3060] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:21.384469 12294 raft_consensus.cc:2468] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3d2109084424941b5c909182d5bfe07 in term 3.
I20260709 13:17:21.385939 12185 leader_election.cc:304] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3d2109084424941b5c909182d5bfe07, eaee9010d8714afea747f296df4322f0; no voters: 
I20260709 13:17:21.386909 12478 raft_consensus.cc:2804] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:21.386788 12143 raft_consensus.cc:2468] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3d2109084424941b5c909182d5bfe07 in term 3.
I20260709 13:17:21.388201 12478 raft_consensus.cc:697] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07 [term 3 LEADER]: Becoming Leader. State: Replica: c3d2109084424941b5c909182d5bfe07, State: Running, Role: LEADER
I20260709 13:17:21.389070 12478 consensus_queue.cc:237] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:21.396279 12042 catalog_manager.cc:5697] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07 reported cstate change: term changed from 2 to 3, leader changed from 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65) to c3d2109084424941b5c909182d5bfe07 (127.11.63.66). New cstate: current_term: 3 leader_uuid: "c3d2109084424941b5c909182d5bfe07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:21.461115 12431 raft_consensus.cc:993] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020: : Instructing follower c3d2109084424941b5c909182d5bfe07 to start an election
I20260709 13:17:21.461639 12431 raft_consensus.cc:1081] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 3 LEADER]: Signalling peer c3d2109084424941b5c909182d5bfe07 to start an election
I20260709 13:17:21.463120 12219 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c20e9b895ed4401e97300148b98276e3"
dest_uuid: "c3d2109084424941b5c909182d5bfe07"
 from {username='slave'} at 127.0.0.1:54982
I20260709 13:17:21.463557 12219 raft_consensus.cc:493] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:21.463822 12219 raft_consensus.cc:3060] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:21.469066 12219 raft_consensus.cc:515] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:21.470535 12219 leader_election.cc:290] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 4 election: Requested vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), eaee9010d8714afea747f296df4322f0 (127.11.63.67:40843)
I20260709 13:17:21.471210 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20e9b895ed4401e97300148b98276e3" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
I20260709 13:17:21.471863 12143 raft_consensus.cc:3055] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:21.471969 12294 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c20e9b895ed4401e97300148b98276e3" candidate_uuid: "c3d2109084424941b5c909182d5bfe07" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "eaee9010d8714afea747f296df4322f0"
I20260709 13:17:21.472229 12143 raft_consensus.cc:740] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Running, Role: LEADER
I20260709 13:17:21.472522 12294 raft_consensus.cc:3060] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:21.473011 12143 consensus_queue.cc:260] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:21.474190 12143 raft_consensus.cc:3060] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:21.479669 12294 raft_consensus.cc:2468] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3d2109084424941b5c909182d5bfe07 in term 4.
I20260709 13:17:21.480952 12185 leader_election.cc:304] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3d2109084424941b5c909182d5bfe07, eaee9010d8714afea747f296df4322f0; no voters: 
I20260709 13:17:21.481133 12143 raft_consensus.cc:2468] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3d2109084424941b5c909182d5bfe07 in term 4.
I20260709 13:17:21.481837 12478 raft_consensus.cc:2804] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:21.482318 12478 raft_consensus.cc:697] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [term 4 LEADER]: Becoming Leader. State: Replica: c3d2109084424941b5c909182d5bfe07, State: Running, Role: LEADER
I20260709 13:17:21.483063 12478 consensus_queue.cc:237] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:21.490443 12042 catalog_manager.cc:5697] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 reported cstate change: term changed from 3 to 4, leader changed from 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65) to c3d2109084424941b5c909182d5bfe07 (127.11.63.66). New cstate: current_term: 4 leader_uuid: "c3d2109084424941b5c909182d5bfe07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:21.540948 12464 raft_consensus.cc:993] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020: : Instructing follower eaee9010d8714afea747f296df4322f0 to start an election
I20260709 13:17:21.541610 12431 raft_consensus.cc:1081] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 3 LEADER]: Signalling peer eaee9010d8714afea747f296df4322f0 to start an election
I20260709 13:17:21.543160 12294 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1488366d31f84925b97ed8f1843b957f"
dest_uuid: "eaee9010d8714afea747f296df4322f0"
 from {username='slave'} at 127.0.0.1:49248
I20260709 13:17:21.543629 12294 raft_consensus.cc:493] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:21.543978 12294 raft_consensus.cc:3060] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:21.548605 12294 raft_consensus.cc:515] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:21.550666 12294 leader_election.cc:290] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 4 election: Requested vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197)
I20260709 13:17:21.551647 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1488366d31f84925b97ed8f1843b957f" candidate_uuid: "eaee9010d8714afea747f296df4322f0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
I20260709 13:17:21.551895 12219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1488366d31f84925b97ed8f1843b957f" candidate_uuid: "eaee9010d8714afea747f296df4322f0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c3d2109084424941b5c909182d5bfe07"
I20260709 13:17:21.552232 12143 raft_consensus.cc:3055] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:21.552372 12219 raft_consensus.cc:3060] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:21.552562 12143 raft_consensus.cc:740] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Running, Role: LEADER
I20260709 13:17:21.553375 12143 consensus_queue.cc:260] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:21.554487 12143 raft_consensus.cc:3060] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:21.554584 12464 raft_consensus.cc:993] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020: : Instructing follower eaee9010d8714afea747f296df4322f0 to start an election
I20260709 13:17:21.556192 12431 raft_consensus.cc:1081] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 4 LEADER]: Signalling peer eaee9010d8714afea747f296df4322f0 to start an election
I20260709 13:17:21.560074 12219 raft_consensus.cc:2468] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaee9010d8714afea747f296df4322f0 in term 4.
I20260709 13:17:21.560779 12143 raft_consensus.cc:2468] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaee9010d8714afea747f296df4322f0 in term 4.
I20260709 13:17:21.560750 12294 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a20e6b89add4f61bb32e0148c51271c"
dest_uuid: "eaee9010d8714afea747f296df4322f0"
 from {username='slave'} at 127.0.0.1:49248
I20260709 13:17:21.561586 12259 leader_election.cc:304] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3d2109084424941b5c909182d5bfe07, eaee9010d8714afea747f296df4322f0; no voters: 
I20260709 13:17:21.562742 12294 raft_consensus.cc:493] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:21.563161 12437 raft_consensus.cc:2804] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:21.563172 12294 raft_consensus.cc:3060] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:21.564466 12437 raft_consensus.cc:697] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [term 4 LEADER]: Becoming Leader. State: Replica: eaee9010d8714afea747f296df4322f0, State: Running, Role: LEADER
I20260709 13:17:21.565464 12437 consensus_queue.cc:237] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:21.568588 12294 raft_consensus.cc:515] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:21.570960 12294 leader_election.cc:290] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 5 election: Requested vote from peers 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933), c3d2109084424941b5c909182d5bfe07 (127.11.63.66:39197)
I20260709 13:17:21.571702 12143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a20e6b89add4f61bb32e0148c51271c" candidate_uuid: "eaee9010d8714afea747f296df4322f0" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "9dbac62bc9fe4f96a350fe935538b020"
I20260709 13:17:21.572268 12143 raft_consensus.cc:3055] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:21.572183 12219 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a20e6b89add4f61bb32e0148c51271c" candidate_uuid: "eaee9010d8714afea747f296df4322f0" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "c3d2109084424941b5c909182d5bfe07"
I20260709 13:17:21.572649 12143 raft_consensus.cc:740] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9dbac62bc9fe4f96a350fe935538b020, State: Running, Role: LEADER
I20260709 13:17:21.572841 12219 raft_consensus.cc:3060] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:21.573453 12143 consensus_queue.cc:260] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:21.573742 12042 catalog_manager.cc:5697] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 reported cstate change: term changed from 3 to 4, leader changed from 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65) to eaee9010d8714afea747f296df4322f0 (127.11.63.67). New cstate: current_term: 4 leader_uuid: "eaee9010d8714afea747f296df4322f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:21.574615 12143 raft_consensus.cc:3060] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:21.577865 12219 raft_consensus.cc:2468] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaee9010d8714afea747f296df4322f0 in term 5.
I20260709 13:17:21.579195 12259 leader_election.cc:304] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c3d2109084424941b5c909182d5bfe07, eaee9010d8714afea747f296df4322f0; no voters: 
I20260709 13:17:21.579706 12143 raft_consensus.cc:2468] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaee9010d8714afea747f296df4322f0 in term 5.
I20260709 13:17:21.580289 12437 raft_consensus.cc:2804] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:21.580778 12437 raft_consensus.cc:697] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 5 LEADER]: Becoming Leader. State: Replica: eaee9010d8714afea747f296df4322f0, State: Running, Role: LEADER
I20260709 13:17:21.581658 12437 consensus_queue.cc:237] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } }
I20260709 13:17:21.588951 12042 catalog_manager.cc:5697] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 reported cstate change: term changed from 4 to 5, leader changed from 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65) to eaee9010d8714afea747f296df4322f0 (127.11.63.67). New cstate: current_term: 5 leader_uuid: "eaee9010d8714afea747f296df4322f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:21.801805 12294 raft_consensus.cc:1275] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [term 3 FOLLOWER]: Refusing update from remote peer c3d2109084424941b5c909182d5bfe07: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:21.803092 12478 consensus_queue.cc:1048] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:21.815954 12143 raft_consensus.cc:1275] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [term 3 FOLLOWER]: Refusing update from remote peer c3d2109084424941b5c909182d5bfe07: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:21.818466 12478 consensus_queue.cc:1048] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:21.866736 12294 raft_consensus.cc:1275] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [term 4 FOLLOWER]: Refusing update from remote peer c3d2109084424941b5c909182d5bfe07: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:21.868319 12478 consensus_queue.cc:1048] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaee9010d8714afea747f296df4322f0" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 40843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:21.877512 12143 raft_consensus.cc:1275] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 4 FOLLOWER]: Refusing update from remote peer c3d2109084424941b5c909182d5bfe07: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:21.879179 12479 consensus_queue.cc:1048] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:21.998238 12143 raft_consensus.cc:1275] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 4 FOLLOWER]: Refusing update from remote peer eaee9010d8714afea747f296df4322f0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:21.999677 12437 consensus_queue.cc:1048] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:22.009356 12219 raft_consensus.cc:1275] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 4 FOLLOWER]: Refusing update from remote peer eaee9010d8714afea747f296df4322f0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:22.010790 12437 consensus_queue.cc:1048] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:22.168466 12219 raft_consensus.cc:1275] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [term 5 FOLLOWER]: Refusing update from remote peer eaee9010d8714afea747f296df4322f0: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:17:22.169628 12499 consensus_queue.cc:1048] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3d2109084424941b5c909182d5bfe07" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 39197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:22.178457 12143 raft_consensus.cc:1275] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 5 FOLLOWER]: Refusing update from remote peer eaee9010d8714afea747f296df4322f0: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:17:22.179857 12499 consensus_queue.cc:1048] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dbac62bc9fe4f96a350fe935538b020" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 39933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:22.290344 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:22.297233 11517 tablet_server.cc:179] TabletServer@127.11.63.65:0 shutting down...
W20260709 13:17:22.304224 12183 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 6 similar messages]
W20260709 13:17:22.314960 12183 consensus_peers.cc:597] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07 -> Peer 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933): Couldn't send request to peer 9dbac62bc9fe4f96a350fe935538b020. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:22.335574 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:17:22.335565 12257 consensus_peers.cc:597] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0 -> Peer 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933): Couldn't send request to peer 9dbac62bc9fe4f96a350fe935538b020. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:22.336577 11517 tablet_replica.cc:333] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020: stopping tablet replica
I20260709 13:17:22.337633 11517 raft_consensus.cc:2243] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.338559 11517 raft_consensus.cc:2272] T c9544a57d3924366b84a3f2eda5e6140 P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.340363 11517 tablet_replica.cc:333] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020: stopping tablet replica
I20260709 13:17:22.340847 11517 raft_consensus.cc:2243] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.341727 11517 raft_consensus.cc:2272] T 6d830eb2a7b547f38ce277c1a68da16e P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.343525 11517 tablet_replica.cc:333] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020: stopping tablet replica
I20260709 13:17:22.344177 11517 raft_consensus.cc:2243] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.344532 11517 raft_consensus.cc:2272] T b5066daaafb345b68502e7a69ce523da P 9dbac62bc9fe4f96a350fe935538b020 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.346990 11517 tablet_replica.cc:333] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020: stopping tablet replica
I20260709 13:17:22.347465 11517 raft_consensus.cc:2243] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.347818 11517 raft_consensus.cc:2272] T 0a20e6b89add4f61bb32e0148c51271c P 9dbac62bc9fe4f96a350fe935538b020 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.349884 11517 tablet_replica.cc:333] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020: stopping tablet replica
I20260709 13:17:22.350347 11517 raft_consensus.cc:2243] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.350682 11517 raft_consensus.cc:2272] T 1488366d31f84925b97ed8f1843b957f P 9dbac62bc9fe4f96a350fe935538b020 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.353165 11517 tablet_replica.cc:333] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020: stopping tablet replica
I20260709 13:17:22.353910 11517 raft_consensus.cc:2243] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.354835 11517 raft_consensus.cc:2272] T 0d9dbc6498c743e1a2fda71e38f4c3db P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.357049 11517 tablet_replica.cc:333] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020: stopping tablet replica
I20260709 13:17:22.357879 11517 raft_consensus.cc:2243] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.358875 11517 raft_consensus.cc:2272] T 25f7341f9ba04efb8912fcbc4f6b190a P 9dbac62bc9fe4f96a350fe935538b020 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.361330 11517 tablet_replica.cc:333] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020: stopping tablet replica
I20260709 13:17:22.362048 11517 raft_consensus.cc:2243] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.362653 11517 raft_consensus.cc:2272] T ee84f8cd8c0242f3ba7a3b0dda942288 P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.365172 11517 tablet_replica.cc:333] T 66472167c0814930aaa3398db1d2a421 P 9dbac62bc9fe4f96a350fe935538b020: stopping tablet replica
I20260709 13:17:22.365903 11517 raft_consensus.cc:2243] T 66472167c0814930aaa3398db1d2a421 P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.366336 11517 raft_consensus.cc:2272] T 66472167c0814930aaa3398db1d2a421 P 9dbac62bc9fe4f96a350fe935538b020 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.367993 11517 tablet_replica.cc:333] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020: stopping tablet replica
I20260709 13:17:22.368376 11517 raft_consensus.cc:2243] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.368698 11517 raft_consensus.cc:2272] T c20e9b895ed4401e97300148b98276e3 P 9dbac62bc9fe4f96a350fe935538b020 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.389832 11517 tablet_server.cc:196] TabletServer@127.11.63.65:0 shutdown complete.
I20260709 13:17:22.415390 11517 tablet_server.cc:179] TabletServer@127.11.63.66:0 shutting down...
W20260709 13:17:22.417274 12183 consensus_peers.cc:597] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 -> Peer 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933): Couldn't send request to peer 9dbac62bc9fe4f96a350fe935538b020. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:39933: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:22.425706 12183 consensus_peers.cc:597] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07 -> Peer 9dbac62bc9fe4f96a350fe935538b020 (127.11.63.65:39933): Couldn't send request to peer 9dbac62bc9fe4f96a350fe935538b020. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:39933: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:22.439250 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:22.440013 11517 tablet_replica.cc:333] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07: stopping tablet replica
I20260709 13:17:22.440587 11517 raft_consensus.cc:2243] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.441025 11517 raft_consensus.cc:2272] T 6d830eb2a7b547f38ce277c1a68da16e P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.442795 11517 tablet_replica.cc:333] T 25f7341f9ba04efb8912fcbc4f6b190a P c3d2109084424941b5c909182d5bfe07: stopping tablet replica
I20260709 13:17:22.443300 11517 raft_consensus.cc:2243] T 25f7341f9ba04efb8912fcbc4f6b190a P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.443739 11517 raft_consensus.cc:2272] T 25f7341f9ba04efb8912fcbc4f6b190a P c3d2109084424941b5c909182d5bfe07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.445768 11517 tablet_replica.cc:333] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07: stopping tablet replica
I20260709 13:17:22.446270 11517 raft_consensus.cc:2243] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.447167 11517 raft_consensus.cc:2272] T b5066daaafb345b68502e7a69ce523da P c3d2109084424941b5c909182d5bfe07 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.449086 11517 tablet_replica.cc:333] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07: stopping tablet replica
I20260709 13:17:22.449636 11517 raft_consensus.cc:2243] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.450018 11517 raft_consensus.cc:2272] T 0a20e6b89add4f61bb32e0148c51271c P c3d2109084424941b5c909182d5bfe07 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.452216 11517 tablet_replica.cc:333] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07: stopping tablet replica
I20260709 13:17:22.452699 11517 raft_consensus.cc:2243] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.453078 11517 raft_consensus.cc:2272] T c9544a57d3924366b84a3f2eda5e6140 P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.454838 11517 tablet_replica.cc:333] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07: stopping tablet replica
I20260709 13:17:22.455264 11517 raft_consensus.cc:2243] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.456040 11517 raft_consensus.cc:2272] T 66472167c0814930aaa3398db1d2a421 P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.457718 11517 tablet_replica.cc:333] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07: stopping tablet replica
I20260709 13:17:22.458168 11517 raft_consensus.cc:2243] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.458530 11517 raft_consensus.cc:2272] T ee84f8cd8c0242f3ba7a3b0dda942288 P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.460216 11517 tablet_replica.cc:333] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07: stopping tablet replica
I20260709 13:17:22.460623 11517 raft_consensus.cc:2243] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.460978 11517 raft_consensus.cc:2272] T 0d9dbc6498c743e1a2fda71e38f4c3db P c3d2109084424941b5c909182d5bfe07 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.462636 11517 tablet_replica.cc:333] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07: stopping tablet replica
I20260709 13:17:22.463057 11517 raft_consensus.cc:2243] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.463438 11517 raft_consensus.cc:2272] T 1488366d31f84925b97ed8f1843b957f P c3d2109084424941b5c909182d5bfe07 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.465514 11517 tablet_replica.cc:333] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07: stopping tablet replica
I20260709 13:17:22.466009 11517 raft_consensus.cc:2243] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.466655 11517 raft_consensus.cc:2272] T c20e9b895ed4401e97300148b98276e3 P c3d2109084424941b5c909182d5bfe07 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.495914 11517 tablet_server.cc:196] TabletServer@127.11.63.66:0 shutdown complete.
I20260709 13:17:22.514371 11517 tablet_server.cc:179] TabletServer@127.11.63.67:0 shutting down...
I20260709 13:17:22.539387 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:22.540161 11517 tablet_replica.cc:333] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0: stopping tablet replica
I20260709 13:17:22.540761 11517 raft_consensus.cc:2243] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.541523 11517 raft_consensus.cc:2272] T 0a20e6b89add4f61bb32e0148c51271c P eaee9010d8714afea747f296df4322f0 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.544377 11517 tablet_replica.cc:333] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0: stopping tablet replica
I20260709 13:17:22.544907 11517 raft_consensus.cc:2243] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.545356 11517 raft_consensus.cc:2272] T b5066daaafb345b68502e7a69ce523da P eaee9010d8714afea747f296df4322f0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.547744 11517 tablet_replica.cc:333] T c9544a57d3924366b84a3f2eda5e6140 P eaee9010d8714afea747f296df4322f0: stopping tablet replica
I20260709 13:17:22.548277 11517 raft_consensus.cc:2243] T c9544a57d3924366b84a3f2eda5e6140 P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.548719 11517 raft_consensus.cc:2272] T c9544a57d3924366b84a3f2eda5e6140 P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.550493 11517 tablet_replica.cc:333] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0: stopping tablet replica
I20260709 13:17:22.550997 11517 raft_consensus.cc:2243] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.551409 11517 raft_consensus.cc:2272] T c20e9b895ed4401e97300148b98276e3 P eaee9010d8714afea747f296df4322f0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.553701 11517 tablet_replica.cc:333] T 25f7341f9ba04efb8912fcbc4f6b190a P eaee9010d8714afea747f296df4322f0: stopping tablet replica
I20260709 13:17:22.554217 11517 raft_consensus.cc:2243] T 25f7341f9ba04efb8912fcbc4f6b190a P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.554641 11517 raft_consensus.cc:2272] T 25f7341f9ba04efb8912fcbc4f6b190a P eaee9010d8714afea747f296df4322f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.556226 11517 tablet_replica.cc:333] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0: stopping tablet replica
I20260709 13:17:22.556710 11517 raft_consensus.cc:2243] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.557471 11517 raft_consensus.cc:2272] T ee84f8cd8c0242f3ba7a3b0dda942288 P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.559149 11517 tablet_replica.cc:333] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0: stopping tablet replica
I20260709 13:17:22.559644 11517 raft_consensus.cc:2243] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.560047 11517 raft_consensus.cc:2272] T 66472167c0814930aaa3398db1d2a421 P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.561681 11517 tablet_replica.cc:333] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0: stopping tablet replica
I20260709 13:17:22.562155 11517 raft_consensus.cc:2243] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.562814 11517 raft_consensus.cc:2272] T 1488366d31f84925b97ed8f1843b957f P eaee9010d8714afea747f296df4322f0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.564749 11517 tablet_replica.cc:333] T 0d9dbc6498c743e1a2fda71e38f4c3db P eaee9010d8714afea747f296df4322f0: stopping tablet replica
I20260709 13:17:22.565210 11517 raft_consensus.cc:2243] T 0d9dbc6498c743e1a2fda71e38f4c3db P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.565652 11517 raft_consensus.cc:2272] T 0d9dbc6498c743e1a2fda71e38f4c3db P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.567219 11517 tablet_replica.cc:333] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0: stopping tablet replica
I20260709 13:17:22.567638 11517 raft_consensus.cc:2243] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.568033 11517 raft_consensus.cc:2272] T 6d830eb2a7b547f38ce277c1a68da16e P eaee9010d8714afea747f296df4322f0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.597184 11517 tablet_server.cc:196] TabletServer@127.11.63.67:0 shutdown complete.
I20260709 13:17:22.616026 11517 master.cc:562] Master@127.11.63.126:43933 shutting down...
I20260709 13:17:22.637925 11517 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.638500 11517 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.638850 11517 tablet_replica.cc:333] T 00000000000000000000000000000000 P 0ffef970145944f9be864f3ba5eb1068: stopping tablet replica
I20260709 13:17:22.659387 11517 master.cc:584] Master@127.11.63.126:43933 shutdown complete.
9dbac62bc9fe4f96a350fe935538b020  5
c3d2109084424941b5c909182d5bfe07  4
eaee9010d8714afea747f296df4322f0  1
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10621 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:17:22.703938 11517 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.11.63.126:38321
I20260709 13:17:22.705073 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:22.711732 12508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:22.711691 12507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:22.713930 12510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:22.715566 11517 server_base.cc:1061] running on GCE node
I20260709 13:17:22.716595 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:22.716843 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:22.717029 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603042717012 us; error 0 us; skew 500 ppm
I20260709 13:17:22.718070 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:22.720906 11517 webserver.cc:533] Webserver started at http://127.11.63.126:43825/ using document root <none> and password file <none>
I20260709 13:17:22.721669 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:22.721900 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:22.722211 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:22.723479 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/master-0-root/instance:
uuid: "480284ff372740a09dfd1f972d55d7eb"
format_stamp: "Formatted at 2026-07-09 13:17:22 on dist-test-slave-qv7r"
I20260709 13:17:22.728609 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.004s
I20260709 13:17:22.732713 12515 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:22.733673 11517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:17:22.733919 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/master-0-root
uuid: "480284ff372740a09dfd1f972d55d7eb"
format_stamp: "Formatted at 2026-07-09 13:17:22 on dist-test-slave-qv7r"
I20260709 13:17:22.734179 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:22.754385 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:22.755594 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:22.799480 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.126:38321
I20260709 13:17:22.799590 12566 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.126:38321 every 8 connection(s)
I20260709 13:17:22.803720 12567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.813786 12567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb: Bootstrap starting.
I20260709 13:17:22.818395 12567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.823217 12567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb: No bootstrap required, opened a new log
I20260709 13:17:22.825624 12567 raft_consensus.cc:359] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "480284ff372740a09dfd1f972d55d7eb" member_type: VOTER }
I20260709 13:17:22.826097 12567 raft_consensus.cc:385] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.826383 12567 raft_consensus.cc:740] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 480284ff372740a09dfd1f972d55d7eb, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.827229 12567 consensus_queue.cc:260] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "480284ff372740a09dfd1f972d55d7eb" member_type: VOTER }
I20260709 13:17:22.827682 12567 raft_consensus.cc:399] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:17:22.827914 12567 raft_consensus.cc:493] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:17:22.828235 12567 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:22.832620 12567 raft_consensus.cc:515] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "480284ff372740a09dfd1f972d55d7eb" member_type: VOTER }
I20260709 13:17:22.833175 12567 leader_election.cc:304] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 480284ff372740a09dfd1f972d55d7eb; no voters: 
I20260709 13:17:22.834419 12567 leader_election.cc:290] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:17:22.835012 12570 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:22.836756 12570 raft_consensus.cc:697] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb [term 1 LEADER]: Becoming Leader. State: Replica: 480284ff372740a09dfd1f972d55d7eb, State: Running, Role: LEADER
I20260709 13:17:22.837344 12567 sys_catalog.cc:565] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:17:22.837581 12570 consensus_queue.cc:237] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "480284ff372740a09dfd1f972d55d7eb" member_type: VOTER }
I20260709 13:17:22.843923 12572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 480284ff372740a09dfd1f972d55d7eb. Latest consensus state: current_term: 1 leader_uuid: "480284ff372740a09dfd1f972d55d7eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "480284ff372740a09dfd1f972d55d7eb" member_type: VOTER } }
I20260709 13:17:22.844604 12572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:22.845988 12571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "480284ff372740a09dfd1f972d55d7eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "480284ff372740a09dfd1f972d55d7eb" member_type: VOTER } }
I20260709 13:17:22.846481 12580 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:17:22.846665 12571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:22.852188 12580 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:17:22.853322 11517 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:17:22.862139 12580 catalog_manager.cc:1383] Generated new cluster ID: b36c48fff6614e1b8a3225e76cd8ea29
I20260709 13:17:22.862447 12580 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:17:22.880183 12580 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:17:22.881549 12580 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:17:22.896665 12580 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb: Generated new TSK 0
I20260709 13:17:22.897277 12580 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:17:22.920609 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:22.927560 12588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:22.929144 12589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:22.930859 12591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:22.930959 11517 server_base.cc:1061] running on GCE node
I20260709 13:17:22.931879 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:22.932102 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:22.932271 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603042932247 us; error 0 us; skew 500 ppm
I20260709 13:17:22.932777 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:22.935334 11517 webserver.cc:533] Webserver started at http://127.11.63.65:42177/ using document root <none> and password file <none>
I20260709 13:17:22.935956 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:22.936165 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:22.936468 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:22.937640 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-0-root/instance:
uuid: "19cd864c053744bebd96093a7982537e"
format_stamp: "Formatted at 2026-07-09 13:17:22 on dist-test-slave-qv7r"
I20260709 13:17:22.942363 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 13:17:22.945976 12596 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:22.946970 11517 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:17:22.947297 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-0-root
uuid: "19cd864c053744bebd96093a7982537e"
format_stamp: "Formatted at 2026-07-09 13:17:22 on dist-test-slave-qv7r"
I20260709 13:17:22.947574 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:22.978264 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:22.979689 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:22.981353 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:22.983773 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:22.984025 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:22.984309 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:22.984531 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:23.031179 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.65:43603
I20260709 13:17:23.031291 12658 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.65:43603 every 8 connection(s)
I20260709 13:17:23.036620 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:23.044853 12663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:23.046008 12664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:23.048759 12666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:23.049163 11517 server_base.cc:1061] running on GCE node
I20260709 13:17:23.050390 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:23.050568 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:23.050800 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603043050781 us; error 0 us; skew 500 ppm
I20260709 13:17:23.051353 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:23.054726 11517 webserver.cc:533] Webserver started at http://127.11.63.66:40655/ using document root <none> and password file <none>
I20260709 13:17:23.054787 12659 heartbeater.cc:344] Connected to a master server at 127.11.63.126:38321
I20260709 13:17:23.055265 12659 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:23.055334 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:23.055637 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:23.055908 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:23.056209 12659 heartbeater.cc:507] Master 127.11.63.126:38321 requested a full tablet report, sending...
I20260709 13:17:23.057103 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-1-root/instance:
uuid: "639ab2db265f4747bb7afb93e833f530"
format_stamp: "Formatted at 2026-07-09 13:17:23 on dist-test-slave-qv7r"
I20260709 13:17:23.058701 12532 ts_manager.cc:194] Registered new tserver with Master: 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:17:23.060506 12532 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54792
I20260709 13:17:23.062418 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.002s
I20260709 13:17:23.066279 12671 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:23.067251 11517 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:17:23.067517 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-1-root
uuid: "639ab2db265f4747bb7afb93e833f530"
format_stamp: "Formatted at 2026-07-09 13:17:23 on dist-test-slave-qv7r"
I20260709 13:17:23.067761 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:23.092475 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:23.093731 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:23.095264 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:23.097499 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:23.097697 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:23.098001 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:23.098150 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:23.140486 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.66:45173
I20260709 13:17:23.140590 12733 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.66:45173 every 8 connection(s)
I20260709 13:17:23.145906 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:23.154289 12737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:23.154768 12738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:23.158380 12734 heartbeater.cc:344] Connected to a master server at 127.11.63.126:38321
I20260709 13:17:23.158981 12734 heartbeater.cc:461] Registering TS with master...
W20260709 13:17:23.159471 12741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:23.160063 12734 heartbeater.cc:507] Master 127.11.63.126:38321 requested a full tablet report, sending...
I20260709 13:17:23.160447 11517 server_base.cc:1061] running on GCE node
I20260709 13:17:23.161695 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:23.161957 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:23.162166 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603043162149 us; error 0 us; skew 500 ppm
I20260709 13:17:23.162563 12532 ts_manager.cc:194] Registered new tserver with Master: 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:23.162743 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:23.164019 12532 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54800
I20260709 13:17:23.165783 11517 webserver.cc:533] Webserver started at http://127.11.63.67:41253/ using document root <none> and password file <none>
I20260709 13:17:23.166334 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:23.166574 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:23.166862 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:23.168047 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-2-root/instance:
uuid: "3d539f8d95d343d8b1696803f18b30d2"
format_stamp: "Formatted at 2026-07-09 13:17:23 on dist-test-slave-qv7r"
I20260709 13:17:23.173161 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.002s
I20260709 13:17:23.177436 12745 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:23.178486 11517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:17:23.178820 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-2-root
uuid: "3d539f8d95d343d8b1696803f18b30d2"
format_stamp: "Formatted at 2026-07-09 13:17:23 on dist-test-slave-qv7r"
I20260709 13:17:23.179095 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:23.206597 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:23.207969 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:23.209597 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:23.212221 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:23.212435 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:23.212774 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:23.212954 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:23.259339 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.67:41181
I20260709 13:17:23.259488 12807 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.67:41181 every 8 connection(s)
I20260709 13:17:23.274516 12808 heartbeater.cc:344] Connected to a master server at 127.11.63.126:38321
I20260709 13:17:23.274984 12808 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:23.275822 12808 heartbeater.cc:507] Master 127.11.63.126:38321 requested a full tablet report, sending...
I20260709 13:17:23.277933 12532 ts_manager.cc:194] Registered new tserver with Master: 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:23.278877 11517 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016338011s
I20260709 13:17:23.279513 12532 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54814
I20260709 13:17:23.309857 12532 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54816:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:23.316169 12532 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:23.524526 12698 tablet_service.cc:1511] Processing CreateTablet for tablet f2f0a3339d324fd4902b6ff970ff0272 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:17:23.525647 12699 tablet_service.cc:1511] Processing CreateTablet for tablet e79ffe63999346abaccd90ab600091d7 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:17:23.526105 12698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2f0a3339d324fd4902b6ff970ff0272. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.526945 12699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e79ffe63999346abaccd90ab600091d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.528263 12697 tablet_service.cc:1511] Processing CreateTablet for tablet 9119721f3ee04810b1f277c06c966a45 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:17:23.529788 12624 tablet_service.cc:1511] Processing CreateTablet for tablet e79ffe63999346abaccd90ab600091d7 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:17:23.531139 12624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e79ffe63999346abaccd90ab600091d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.530823 12696 tablet_service.cc:1511] Processing CreateTablet for tablet fa34731f37084c608660f0088a2dba09 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:17:23.533242 12618 tablet_service.cc:1511] Processing CreateTablet for tablet 3938a96da1794e32b484ac729226a8ce (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:17:23.534746 12618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3938a96da1794e32b484ac729226a8ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.534821 12771 tablet_service.cc:1511] Processing CreateTablet for tablet 9119721f3ee04810b1f277c06c966a45 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:17:23.535609 12773 tablet_service.cc:1511] Processing CreateTablet for tablet e79ffe63999346abaccd90ab600091d7 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:17:23.536334 12771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9119721f3ee04810b1f277c06c966a45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.536979 12773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e79ffe63999346abaccd90ab600091d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.536895 12619 tablet_service.cc:1511] Processing CreateTablet for tablet 5febc2df9d3c41359ceccc1bef8a54d8 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:17:23.534816 12623 tablet_service.cc:1511] Processing CreateTablet for tablet f2f0a3339d324fd4902b6ff970ff0272 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:17:23.538395 12619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5febc2df9d3c41359ceccc1bef8a54d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.533119 12695 tablet_service.cc:1511] Processing CreateTablet for tablet 414b06f3e5f8472ea4071ebdd0161b9d (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:17:23.538803 12772 tablet_service.cc:1511] Processing CreateTablet for tablet f2f0a3339d324fd4902b6ff970ff0272 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:17:23.539731 12623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2f0a3339d324fd4902b6ff970ff0272. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.540150 12772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2f0a3339d324fd4902b6ff970ff0272. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.540314 12620 tablet_service.cc:1511] Processing CreateTablet for tablet 414b06f3e5f8472ea4071ebdd0161b9d (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:17:23.542690 12770 tablet_service.cc:1511] Processing CreateTablet for tablet fa34731f37084c608660f0088a2dba09 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:17:23.544082 12770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa34731f37084c608660f0088a2dba09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.543731 12622 tablet_service.cc:1511] Processing CreateTablet for tablet 9119721f3ee04810b1f277c06c966a45 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:17:23.546259 12694 tablet_service.cc:1511] Processing CreateTablet for tablet 5febc2df9d3c41359ceccc1bef8a54d8 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:17:23.546721 12769 tablet_service.cc:1511] Processing CreateTablet for tablet 414b06f3e5f8472ea4071ebdd0161b9d (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:17:23.547987 12769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 414b06f3e5f8472ea4071ebdd0161b9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.548552 12620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 414b06f3e5f8472ea4071ebdd0161b9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.550889 12768 tablet_service.cc:1511] Processing CreateTablet for tablet 5febc2df9d3c41359ceccc1bef8a54d8 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:17:23.552309 12768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5febc2df9d3c41359ceccc1bef8a54d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.553949 12622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9119721f3ee04810b1f277c06c966a45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.542690 12621 tablet_service.cc:1511] Processing CreateTablet for tablet fa34731f37084c608660f0088a2dba09 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:17:23.558141 12621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa34731f37084c608660f0088a2dba09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.561801 12697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9119721f3ee04810b1f277c06c966a45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.562536 12695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 414b06f3e5f8472ea4071ebdd0161b9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.570286 12696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa34731f37084c608660f0088a2dba09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.580897 12694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5febc2df9d3c41359ceccc1bef8a54d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.599931 12693 tablet_service.cc:1511] Processing CreateTablet for tablet 3938a96da1794e32b484ac729226a8ce (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:17:23.603754 12693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3938a96da1794e32b484ac729226a8ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.604846 12823 tablet_bootstrap.cc:492] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:23.614380 12767 tablet_service.cc:1511] Processing CreateTablet for tablet 3938a96da1794e32b484ac729226a8ce (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:17:23.615972 12767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3938a96da1794e32b484ac729226a8ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.631546 12617 tablet_service.cc:1511] Processing CreateTablet for tablet 5fd9f4e830ed4a178665ad4e4ecc4452 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:17:23.632411 12823 tablet_bootstrap.cc:654] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.633750 12617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fd9f4e830ed4a178665ad4e4ecc4452. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.643851 12699 tablet_service.cc:1511] Processing CreateTablet for tablet 5fd9f4e830ed4a178665ad4e4ecc4452 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:17:23.645295 12699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fd9f4e830ed4a178665ad4e4ecc4452. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.661317 12622 tablet_service.cc:1511] Processing CreateTablet for tablet b1054d44ad14423fb3d51a5243499bb9 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:17:23.662262 12824 tablet_bootstrap.cc:492] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:23.662962 12622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1054d44ad14423fb3d51a5243499bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.663946 12766 tablet_service.cc:1511] Processing CreateTablet for tablet 5fd9f4e830ed4a178665ad4e4ecc4452 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:17:23.665194 12766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fd9f4e830ed4a178665ad4e4ecc4452. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.684204 12696 tablet_service.cc:1511] Processing CreateTablet for tablet b1054d44ad14423fb3d51a5243499bb9 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:17:23.684666 12768 tablet_service.cc:1511] Processing CreateTablet for tablet b1054d44ad14423fb3d51a5243499bb9 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:17:23.686524 12768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1054d44ad14423fb3d51a5243499bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.685650 12696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1054d44ad14423fb3d51a5243499bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.688886 12618 tablet_service.cc:1511] Processing CreateTablet for tablet 0db63389cf0d45de92e4b01621db07cd (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:17:23.690137 12618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0db63389cf0d45de92e4b01621db07cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.698706 12699 tablet_service.cc:1511] Processing CreateTablet for tablet 0db63389cf0d45de92e4b01621db07cd (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:17:23.700053 12699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0db63389cf0d45de92e4b01621db07cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.701496 12622 tablet_service.cc:1511] Processing CreateTablet for tablet 775128ed7b8d4283bf303c8d40247615 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:17:23.702775 12622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 775128ed7b8d4283bf303c8d40247615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.707444 12823 tablet_bootstrap.cc:492] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:23.707856 12823 ts_tablet_manager.cc:1403] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.103s	user 0.011s	sys 0.002s
I20260709 13:17:23.715389 12825 tablet_bootstrap.cc:492] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:23.715452 12696 tablet_service.cc:1511] Processing CreateTablet for tablet 775128ed7b8d4283bf303c8d40247615 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:17:23.717818 12696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 775128ed7b8d4283bf303c8d40247615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.717411 12770 tablet_service.cc:1511] Processing CreateTablet for tablet 775128ed7b8d4283bf303c8d40247615 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:17:23.718957 12770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 775128ed7b8d4283bf303c8d40247615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.715459 12767 tablet_service.cc:1511] Processing CreateTablet for tablet 0db63389cf0d45de92e4b01621db07cd (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:17:23.722851 12767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0db63389cf0d45de92e4b01621db07cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.731606 12825 tablet_bootstrap.cc:654] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.734946 12823 raft_consensus.cc:359] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } }
I20260709 13:17:23.735787 12823 raft_consensus.cc:385] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.736120 12823 raft_consensus.cc:740] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.736965 12823 consensus_queue.cc:260] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } }
I20260709 13:17:23.738582 12620 tablet_service.cc:1511] Processing CreateTablet for tablet a4d7de736149496b86a37ba2ca92670d (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:17:23.739957 12620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4d7de736149496b86a37ba2ca92670d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.746147 12824 tablet_bootstrap.cc:654] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.746917 12618 tablet_service.cc:1511] Processing CreateTablet for tablet 866523bf2d2f442f971d504267ed200f (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:17:23.748605 12618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 866523bf2d2f442f971d504267ed200f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.750924 12697 tablet_service.cc:1511] Processing CreateTablet for tablet a4d7de736149496b86a37ba2ca92670d (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:17:23.745872 12771 tablet_service.cc:1511] Processing CreateTablet for tablet a4d7de736149496b86a37ba2ca92670d (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:17:23.760520 12771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4d7de736149496b86a37ba2ca92670d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.767030 12699 tablet_service.cc:1511] Processing CreateTablet for tablet 866523bf2d2f442f971d504267ed200f (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:17:23.768388 12699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 866523bf2d2f442f971d504267ed200f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.772744 12823 ts_tablet_manager.cc:1434] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.065s	user 0.013s	sys 0.009s
I20260709 13:17:23.773226 12697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4d7de736149496b86a37ba2ca92670d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.775215 12734 heartbeater.cc:499] Master 127.11.63.126:38321 was elected leader, sending a full tablet report...
I20260709 13:17:23.781445 12622 tablet_service.cc:1511] Processing CreateTablet for tablet 199931c4ffc44fee980d8bb49cf37f00 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:17:23.783754 12622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 199931c4ffc44fee980d8bb49cf37f00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.783505 12698 tablet_service.cc:1511] Processing CreateTablet for tablet 199931c4ffc44fee980d8bb49cf37f00 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:17:23.784951 12698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 199931c4ffc44fee980d8bb49cf37f00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.786067 12766 tablet_service.cc:1511] Processing CreateTablet for tablet 866523bf2d2f442f971d504267ed200f (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:17:23.787309 12766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 866523bf2d2f442f971d504267ed200f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.788043 12623 tablet_service.cc:1511] Processing CreateTablet for tablet 059429c0c4024dfb941d1ff325fe8f98 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:17:23.789285 12623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 059429c0c4024dfb941d1ff325fe8f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.791179 12693 tablet_service.cc:1511] Processing CreateTablet for tablet 059429c0c4024dfb941d1ff325fe8f98 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:17:23.792495 12693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 059429c0c4024dfb941d1ff325fe8f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.808784 12769 tablet_service.cc:1511] Processing CreateTablet for tablet 059429c0c4024dfb941d1ff325fe8f98 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:17:23.810537 12769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 059429c0c4024dfb941d1ff325fe8f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.811772 12768 tablet_service.cc:1511] Processing CreateTablet for tablet 3ca762dedae144be828acb659b8d9e4b (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:17:23.813017 12768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ca762dedae144be828acb659b8d9e4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.813702 12772 tablet_service.cc:1511] Processing CreateTablet for tablet 199931c4ffc44fee980d8bb49cf37f00 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:17:23.814926 12772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 199931c4ffc44fee980d8bb49cf37f00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.816244 12773 tablet_service.cc:1511] Processing CreateTablet for tablet d46747dede234934bea0f0c324106c28 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:17:23.817477 12773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d46747dede234934bea0f0c324106c28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.823043 12823 tablet_bootstrap.cc:492] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:23.824445 12694 tablet_service.cc:1511] Processing CreateTablet for tablet 3ca762dedae144be828acb659b8d9e4b (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:17:23.825620 12619 tablet_service.cc:1511] Processing CreateTablet for tablet d1362166bcbb48e88ac7b5862a23517e (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:17:23.825939 12621 tablet_service.cc:1511] Processing CreateTablet for tablet d46747dede234934bea0f0c324106c28 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:17:23.826890 12619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1362166bcbb48e88ac7b5862a23517e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.827047 12696 tablet_service.cc:1511] Processing CreateTablet for tablet d46747dede234934bea0f0c324106c28 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:17:23.829085 12695 tablet_service.cc:1511] Processing CreateTablet for tablet d1362166bcbb48e88ac7b5862a23517e (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:17:23.832875 12620 tablet_service.cc:1511] Processing CreateTablet for tablet 3ca762dedae144be828acb659b8d9e4b (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:17:23.832970 12692 tablet_service.cc:1511] Processing CreateTablet for tablet fb247fe4862541ccadc22bb79417d66c (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:17:23.834400 12620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ca762dedae144be828acb659b8d9e4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.840083 12621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d46747dede234934bea0f0c324106c28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.840854 12618 tablet_service.cc:1511] Processing CreateTablet for tablet fb247fe4862541ccadc22bb79417d66c (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:17:23.842340 12618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb247fe4862541ccadc22bb79417d66c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.825963 12694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ca762dedae144be828acb659b8d9e4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.846474 12692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb247fe4862541ccadc22bb79417d66c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.850178 12695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1362166bcbb48e88ac7b5862a23517e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.853876 12696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d46747dede234934bea0f0c324106c28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.856189 12825 tablet_bootstrap.cc:492] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:23.856763 12825 ts_tablet_manager.cc:1403] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.142s	user 0.015s	sys 0.016s
I20260709 13:17:23.856942 12828 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.857412 12828 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } }
I20260709 13:17:23.860009 12825 raft_consensus.cc:359] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:23.860898 12825 raft_consensus.cc:385] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.861522 12825 raft_consensus.cc:740] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.862519 12825 consensus_queue.cc:260] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:23.864872 12765 tablet_service.cc:1511] Processing CreateTablet for tablet febe63321d6c417cbfc5ccbca7caeefa (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:17:23.866334 12765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet febe63321d6c417cbfc5ccbca7caeefa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.870241 12764 tablet_service.cc:1511] Processing CreateTablet for tablet 4d53f974eee5475b9cd2701948f533ac (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:17:23.874562 12764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d53f974eee5475b9cd2701948f533ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.880561 12624 tablet_service.cc:1511] Processing CreateTablet for tablet 4d53f974eee5475b9cd2701948f533ac (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:17:23.881930 12624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d53f974eee5475b9cd2701948f533ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.887112 12770 tablet_service.cc:1511] Processing CreateTablet for tablet fb247fe4862541ccadc22bb79417d66c (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:17:23.888788 12770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb247fe4862541ccadc22bb79417d66c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.897856 12828 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:17:23.902771 12767 tablet_service.cc:1511] Processing CreateTablet for tablet d1362166bcbb48e88ac7b5862a23517e (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:17:23.904685 12767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1362166bcbb48e88ac7b5862a23517e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.911828 12617 tablet_service.cc:1511] Processing CreateTablet for tablet febe63321d6c417cbfc5ccbca7caeefa (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:17:23.911715 12616 tablet_service.cc:1511] Processing CreateTablet for tablet c6cd31a55742463db9f0f0060bb60a77 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:17:23.914747 12615 tablet_service.cc:1511] Processing CreateTablet for tablet bc7827272b424bad88349b6b7437844b (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:17:23.916440 12615 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc7827272b424bad88349b6b7437844b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:17:23.917451 12579 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:23.922048 12618 tablet_service.cc:1511] Processing CreateTablet for tablet e1a0f7a1130449548d1659cd21760fc9 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:17:23.923425 12618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1a0f7a1130449548d1659cd21760fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.934955 12691 tablet_service.cc:1511] Processing CreateTablet for tablet febe63321d6c417cbfc5ccbca7caeefa (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:17:23.936465 12691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet febe63321d6c417cbfc5ccbca7caeefa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.941512 12623 tablet_service.cc:1511] Processing CreateTablet for tablet 5067d5ea86fa4774a6be5b882e5b289d (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:17:23.959326 12620 tablet_service.cc:1511] Processing CreateTablet for tablet 4679cef42d5540e9a5d3a66e52cdcaf9 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:17:23.963109 12617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet febe63321d6c417cbfc5ccbca7caeefa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.963812 12616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6cd31a55742463db9f0f0060bb60a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.969887 12620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4679cef42d5540e9a5d3a66e52cdcaf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.973881 12623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5067d5ea86fa4774a6be5b882e5b289d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.974332 12772 tablet_service.cc:1511] Processing CreateTablet for tablet c6cd31a55742463db9f0f0060bb60a77 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:17:23.975641 12772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6cd31a55742463db9f0f0060bb60a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.983424 12766 tablet_service.cc:1511] Processing CreateTablet for tablet e1a0f7a1130449548d1659cd21760fc9 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:17:24.033447 12771 tablet_service.cc:1511] Processing CreateTablet for tablet bc7827272b424bad88349b6b7437844b (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:17:24.070569 12659 heartbeater.cc:499] Master 127.11.63.126:38321 was elected leader, sending a full tablet report...
I20260709 13:17:24.094972 12808 heartbeater.cc:499] Master 127.11.63.126:38321 was elected leader, sending a full tablet report...
I20260709 13:17:24.095373 12691 tablet_service.cc:1511] Processing CreateTablet for tablet 4d53f974eee5475b9cd2701948f533ac (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:17:24.096868 12691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d53f974eee5475b9cd2701948f533ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.099334 12832 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.099939 12832 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:24.111212 12698 tablet_service.cc:1511] Processing CreateTablet for tablet e1a0f7a1130449548d1659cd21760fc9 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:17:24.113260 12698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1a0f7a1130449548d1659cd21760fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.125087 12771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc7827272b424bad88349b6b7437844b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.127925 12766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1a0f7a1130449548d1659cd21760fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.166194 12825 ts_tablet_manager.cc:1434] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.309s	user 0.029s	sys 0.054s
I20260709 13:17:24.167379 12825 tablet_bootstrap.cc:492] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:24.169302 12823 tablet_bootstrap.cc:654] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.188180 12832 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:24.212313 12770 tablet_service.cc:1511] Processing CreateTablet for tablet af96e52b004d4c8c942cc0ec33c95c99 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:17:24.214679 12770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af96e52b004d4c8c942cc0ec33c95c99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.215678 12694 tablet_service.cc:1511] Processing CreateTablet for tablet 4679cef42d5540e9a5d3a66e52cdcaf9 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:17:24.216298 12699 tablet_service.cc:1511] Processing CreateTablet for tablet 5067d5ea86fa4774a6be5b882e5b289d (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:17:24.217739 12697 tablet_service.cc:1511] Processing CreateTablet for tablet af96e52b004d4c8c942cc0ec33c95c99 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:17:24.220856 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:24.231815 12696 tablet_service.cc:1511] Processing CreateTablet for tablet bc7827272b424bad88349b6b7437844b (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:17:24.168848 12621 tablet_service.cc:1511] Processing CreateTablet for tablet af96e52b004d4c8c942cc0ec33c95c99 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:17:24.235553 12769 tablet_service.cc:1511] Processing CreateTablet for tablet 1b13b87a5cea4263937041d123c5889d (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:17:24.237464 12825 tablet_bootstrap.cc:654] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.244990 12699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5067d5ea86fa4774a6be5b882e5b289d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.244154 12621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af96e52b004d4c8c942cc0ec33c95c99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.250578 12623 tablet_service.cc:1511] Processing CreateTablet for tablet 1b13b87a5cea4263937041d123c5889d (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:17:24.251314 12694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4679cef42d5540e9a5d3a66e52cdcaf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.251894 12623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b13b87a5cea4263937041d123c5889d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.251993 12697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af96e52b004d4c8c942cc0ec33c95c99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.244140 12692 tablet_service.cc:1511] Processing CreateTablet for tablet c6cd31a55742463db9f0f0060bb60a77 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:17:24.255601 12692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6cd31a55742463db9f0f0060bb60a77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.255921 12765 tablet_service.cc:1511] Processing CreateTablet for tablet 4679cef42d5540e9a5d3a66e52cdcaf9 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:17:24.264317 12769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b13b87a5cea4263937041d123c5889d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.266886 12619 tablet_service.cc:1511] Processing CreateTablet for tablet ab5ddded6fd34f278dade99f2086c4e6 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:17:24.268226 12619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab5ddded6fd34f278dade99f2086c4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.270179 12615 tablet_service.cc:1511] Processing CreateTablet for tablet 6146f826f1904f6f8b10ad518d454c22 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:17:24.271361 12824 tablet_bootstrap.cc:492] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:24.271471 12615 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6146f826f1904f6f8b10ad518d454c22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.271881 12824 ts_tablet_manager.cc:1403] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.610s	user 0.026s	sys 0.044s
I20260709 13:17:24.273710 12773 tablet_service.cc:1511] Processing CreateTablet for tablet 6146f826f1904f6f8b10ad518d454c22 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:17:24.274436 12823 tablet_bootstrap.cc:492] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:24.274915 12823 ts_tablet_manager.cc:1403] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.498s	user 0.019s	sys 0.011s
I20260709 13:17:24.275203 12773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6146f826f1904f6f8b10ad518d454c22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.277911 12823 raft_consensus.cc:359] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:24.278767 12823 raft_consensus.cc:385] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.279073 12823 raft_consensus.cc:740] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.280825 12771 tablet_service.cc:1511] Processing CreateTablet for tablet ab5ddded6fd34f278dade99f2086c4e6 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:17:24.282117 12765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4679cef42d5540e9a5d3a66e52cdcaf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.283083 12771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab5ddded6fd34f278dade99f2086c4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.289433 12823 consensus_queue.cc:260] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:17:24.294562 12672 leader_election.cc:343] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.294863 12693 tablet_service.cc:1511] Processing CreateTablet for tablet 1b13b87a5cea4263937041d123c5889d (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:17:24.296695 12690 tablet_service.cc:1511] Processing CreateTablet for tablet ab5ddded6fd34f278dade99f2086c4e6 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:17:24.300071 12823 ts_tablet_manager.cc:1434] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
I20260709 13:17:24.310356 12764 tablet_service.cc:1511] Processing CreateTablet for tablet 5067d5ea86fa4774a6be5b882e5b289d (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:17:24.312000 12764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5067d5ea86fa4774a6be5b882e5b289d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.293264 12624 tablet_service.cc:1511] Processing CreateTablet for tablet 58239dd301f64758846dd758e449e4d8 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:17:24.335536 12624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58239dd301f64758846dd758e449e4d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.338240 12823 tablet_bootstrap.cc:492] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:24.346776 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:17:24.348657 12673 leader_election.cc:343] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.349128 12673 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:24.360996 12828 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:24.351512 12693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b13b87a5cea4263937041d123c5889d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.370138 12772 tablet_service.cc:1511] Processing CreateTablet for tablet 58239dd301f64758846dd758e449e4d8 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:17:24.371502 12772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58239dd301f64758846dd758e449e4d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.379060 12824 raft_consensus.cc:359] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } }
I20260709 13:17:24.379822 12824 raft_consensus.cc:385] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.380131 12824 raft_consensus.cc:740] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.380868 12824 consensus_queue.cc:260] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } }
I20260709 13:17:24.385044 12622 tablet_service.cc:1511] Processing CreateTablet for tablet bd1bc5f9b25e46cc866068bd346e11c6 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:17:24.402191 12622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd1bc5f9b25e46cc866068bd346e11c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.411880 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:17:24.413545 12747 leader_election.cc:343] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.420787 12699 tablet_service.cc:1511] Processing CreateTablet for tablet 58239dd301f64758846dd758e449e4d8 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:17:24.414471 12772 tablet_service.cc:1511] Processing CreateTablet for tablet bd1bc5f9b25e46cc866068bd346e11c6 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:17:24.429888 12690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab5ddded6fd34f278dade99f2086c4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.430009 12767 tablet_service.cc:1511] Processing CreateTablet for tablet 94e3333498524f14a17661cdb38ef144 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:17:24.431460 12699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58239dd301f64758846dd758e449e4d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.433470 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:17:24.435070 12747 leader_election.cc:343] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.443692 12767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94e3333498524f14a17661cdb38ef144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.443897 12696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc7827272b424bad88349b6b7437844b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.435478 12747 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:24.439653 12618 tablet_service.cc:1511] Processing CreateTablet for tablet 94e3333498524f14a17661cdb38ef144 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:17:24.452836 12832 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:24.453795 12772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd1bc5f9b25e46cc866068bd346e11c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.455950 12698 tablet_service.cc:1511] Processing CreateTablet for tablet bd1bc5f9b25e46cc866068bd346e11c6 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:17:24.457259 12698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd1bc5f9b25e46cc866068bd346e11c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.464162 12615 tablet_service.cc:1511] Processing CreateTablet for tablet 612a4639a7a749be8856bbc60766e312 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:17:24.464775 12825 tablet_bootstrap.cc:492] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:24.465305 12825 ts_tablet_manager.cc:1403] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.298s	user 0.013s	sys 0.022s
I20260709 13:17:24.466638 12694 tablet_service.cc:1511] Processing CreateTablet for tablet 94e3333498524f14a17661cdb38ef144 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:17:24.467919 12694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94e3333498524f14a17661cdb38ef144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.468314 12825 raft_consensus.cc:359] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:24.469012 12825 raft_consensus.cc:385] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:17:24.470532 12695 consensus_meta.cc:287] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.547s	user 0.009s	sys 0.020s
I20260709 13:17:24.473533 12695 tablet_service.cc:1511] Processing CreateTablet for tablet 612a4639a7a749be8856bbc60766e312 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:17:24.474808 12695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 612a4639a7a749be8856bbc60766e312. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.469318 12825 raft_consensus.cc:740] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.497131 12824 ts_tablet_manager.cc:1434] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.225s	user 0.007s	sys 0.003s
I20260709 13:17:24.498224 12824 tablet_bootstrap.cc:492] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:24.499001 12825 consensus_queue.cc:260] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:24.501941 12691 tablet_service.cc:1511] Processing CreateTablet for tablet 6146f826f1904f6f8b10ad518d454c22 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:17:24.503248 12691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6146f826f1904f6f8b10ad518d454c22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.510726 12771 tablet_service.cc:1511] Processing CreateTablet for tablet 612a4639a7a749be8856bbc60766e312 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:17:24.512244 12771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 612a4639a7a749be8856bbc60766e312. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.493718 12618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94e3333498524f14a17661cdb38ef144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.514174 12615 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 612a4639a7a749be8856bbc60766e312. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.522418 12692 tablet_service.cc:1511] Processing CreateTablet for tablet 20db0f202ea546699789c2325d241b35 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:17:24.512260 12823 tablet_bootstrap.cc:654] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.543525 12692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20db0f202ea546699789c2325d241b35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.530586 12619 tablet_service.cc:1511] Processing CreateTablet for tablet 20db0f202ea546699789c2325d241b35 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:17:24.546134 12619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20db0f202ea546699789c2325d241b35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.539753 12770 tablet_service.cc:1511] Processing CreateTablet for tablet 20db0f202ea546699789c2325d241b35 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:17:24.555332 12770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20db0f202ea546699789c2325d241b35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.540658 12824 tablet_bootstrap.cc:654] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.540218 12624 tablet_service.cc:1511] Processing CreateTablet for tablet ad2866c2079c4c5b9df5ad6a072e192d (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:17:24.501459 12825 ts_tablet_manager.cc:1434] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.036s	user 0.004s	sys 0.001s
I20260709 13:17:24.558758 12825 tablet_bootstrap.cc:492] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:24.562564 12693 tablet_service.cc:1511] Processing CreateTablet for tablet ad2866c2079c4c5b9df5ad6a072e192d (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:17:24.563851 12693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad2866c2079c4c5b9df5ad6a072e192d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.567189 12623 tablet_service.cc:1511] Processing CreateTablet for tablet 9132272ce4c847a99e395a53c1015230 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:17:24.568439 12623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9132272ce4c847a99e395a53c1015230. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.563460 12765 tablet_service.cc:1511] Processing CreateTablet for tablet ad2866c2079c4c5b9df5ad6a072e192d (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:17:24.574456 12765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad2866c2079c4c5b9df5ad6a072e192d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.575691 12825 tablet_bootstrap.cc:654] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.576364 12823 tablet_bootstrap.cc:492] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:24.576859 12823 ts_tablet_manager.cc:1403] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.276s	user 0.024s	sys 0.007s
I20260709 13:17:24.578397 12624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad2866c2079c4c5b9df5ad6a072e192d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.656620 12824 tablet_bootstrap.cc:492] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:24.657173 12824 ts_tablet_manager.cc:1403] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.159s	user 0.019s	sys 0.011s
I20260709 13:17:24.660342 12824 raft_consensus.cc:359] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:24.661082 12824 raft_consensus.cc:385] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.661381 12824 raft_consensus.cc:740] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.670928 12694 tablet_service.cc:1511] Processing CreateTablet for tablet 9132272ce4c847a99e395a53c1015230 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:17:24.672299 12694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9132272ce4c847a99e395a53c1015230. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.674894 12823 raft_consensus.cc:359] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:24.675638 12823 raft_consensus.cc:385] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.675926 12823 raft_consensus.cc:740] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.676667 12823 consensus_queue.cc:260] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:24.680205 12823 ts_tablet_manager.cc:1434] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.103s	user 0.007s	sys 0.011s
I20260709 13:17:24.680783 12619 tablet_service.cc:1511] Processing CreateTablet for tablet 49dad260ffe242d2b4e0976f119a19d7 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:17:24.691027 12619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49dad260ffe242d2b4e0976f119a19d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.701640 12823 tablet_bootstrap.cc:492] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:24.678647 12824 consensus_queue.cc:260] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:24.707144 12823 tablet_bootstrap.cc:654] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.707849 12824 ts_tablet_manager.cc:1434] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.050s	user 0.004s	sys 0.004s
I20260709 13:17:24.708889 12824 tablet_bootstrap.cc:492] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:24.710549 12771 tablet_service.cc:1511] Processing CreateTablet for tablet 9132272ce4c847a99e395a53c1015230 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:17:24.710943 12825 tablet_bootstrap.cc:492] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:24.710951 12618 tablet_service.cc:1511] Processing CreateTablet for tablet ea604dddf4c244a181921c4dbdd0e720 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:17:24.716952 12618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea604dddf4c244a181921c4dbdd0e720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.720234 12825 ts_tablet_manager.cc:1403] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.162s	user 0.035s	sys 0.029s
I20260709 13:17:24.720268 12770 tablet_service.cc:1511] Processing CreateTablet for tablet ea604dddf4c244a181921c4dbdd0e720 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:17:24.711030 12765 tablet_service.cc:1511] Processing CreateTablet for tablet 49dad260ffe242d2b4e0976f119a19d7 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:17:24.725998 12693 tablet_service.cc:1511] Processing CreateTablet for tablet ea604dddf4c244a181921c4dbdd0e720 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:17:24.712074 12771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9132272ce4c847a99e395a53c1015230. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.730207 12770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea604dddf4c244a181921c4dbdd0e720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.730522 12693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea604dddf4c244a181921c4dbdd0e720. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.730998 12765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49dad260ffe242d2b4e0976f119a19d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.726675 12624 tablet_service.cc:1511] Processing CreateTablet for tablet 1fa805ee07a7429abe6b520bbfb94cd9 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:17:24.712314 12694 tablet_service.cc:1511] Processing CreateTablet for tablet 49dad260ffe242d2b4e0976f119a19d7 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:17:24.734313 12624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fa805ee07a7429abe6b520bbfb94cd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.734330 12694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49dad260ffe242d2b4e0976f119a19d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.735302 12823 tablet_bootstrap.cc:492] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:24.735896 12823 ts_tablet_manager.cc:1403] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.035s	user 0.009s	sys 0.004s
I20260709 13:17:24.738519 12824 tablet_bootstrap.cc:654] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.739066 12823 raft_consensus.cc:359] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:24.739876 12823 raft_consensus.cc:385] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.740182 12823 raft_consensus.cc:740] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.723397 12825 raft_consensus.cc:359] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } }
I20260709 13:17:24.746520 12825 raft_consensus.cc:385] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.749516 12825 raft_consensus.cc:740] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.751369 12823 consensus_queue.cc:260] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:24.752290 12698 tablet_service.cc:1511] Processing CreateTablet for tablet 1fa805ee07a7429abe6b520bbfb94cd9 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:17:24.753731 12698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fa805ee07a7429abe6b520bbfb94cd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.758219 12828 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.758710 12828 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:24.789809 12841 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.790508 12841 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } }
I20260709 13:17:24.830477 12823 ts_tablet_manager.cc:1434] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.094s	user 0.016s	sys 0.008s
I20260709 13:17:24.831630 12823 tablet_bootstrap.cc:492] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:24.833393 12847 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.836736 12847 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:24.836799 12772 tablet_service.cc:1511] Processing CreateTablet for tablet 1fa805ee07a7429abe6b520bbfb94cd9 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:17:24.838353 12772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fa805ee07a7429abe6b520bbfb94cd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.838353 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:24.839025 12847 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:24.832691 12825 consensus_queue.cc:260] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } }
I20260709 13:17:24.839977 12828 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:24.840772 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:24.840853 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:24.841835 12825 ts_tablet_manager.cc:1434] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.121s	user 0.019s	sys 0.008s
I20260709 13:17:24.842865 12825 tablet_bootstrap.cc:492] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
W20260709 13:17:24.843118 12673 leader_election.cc:343] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:24.851208 12672 leader_election.cc:343] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.852615 12672 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:17:24.853917 12672 leader_election.cc:343] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.852013 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:17:24.857882 12673 leader_election.cc:343] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.863543 12673 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:24.864795 12847 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:24.868008 12771 tablet_service.cc:1511] Processing CreateTablet for tablet c4e4ae9d38ba4fd0a700998e4f4750be (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:17:24.869688 12771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4e4ae9d38ba4fd0a700998e4f4750be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.857383 12828 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:24.884737 12770 tablet_service.cc:1511] Processing CreateTablet for tablet 3b98bb10e3e04334b7fcf7f781593682 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:17:24.885739 12824 tablet_bootstrap.cc:492] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:24.886552 12824 ts_tablet_manager.cc:1403] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.178s	user 0.017s	sys 0.010s
I20260709 13:17:24.887825 12770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b98bb10e3e04334b7fcf7f781593682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.890866 12824 raft_consensus.cc:359] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:24.891460 12769 tablet_service.cc:1511] Processing CreateTablet for tablet 8f91059867c444d89718e83475c089cd (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:17:24.899573 12769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f91059867c444d89718e83475c089cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.895818 12767 tablet_service.cc:1511] Processing CreateTablet for tablet 98a43c5d78c945bb8c5541bc892f4865 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
W20260709 13:17:24.833427 12517 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 76873.4us
I20260709 13:17:24.898337 12823 tablet_bootstrap.cc:654] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.897892 12841 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:24.903074 12767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98a43c5d78c945bb8c5541bc892f4865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.904275 12825 tablet_bootstrap.cc:654] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.909673 12824 raft_consensus.cc:385] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.923797 12618 tablet_service.cc:1511] Processing CreateTablet for tablet c4e4ae9d38ba4fd0a700998e4f4750be (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:17:24.923903 12764 tablet_service.cc:1511] Processing CreateTablet for tablet 4c13fa2aff324f65ad0bf6191ea68fab (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
W20260709 13:17:24.924777 12579 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:24.925276 12764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c13fa2aff324f65ad0bf6191ea68fab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.926901 12768 tablet_service.cc:1511] Processing CreateTablet for tablet 18b2f67fae7f4963a4b4c562003df2fe (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:17:24.928139 12768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18b2f67fae7f4963a4b4c562003df2fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.929708 12766 tablet_service.cc:1511] Processing CreateTablet for tablet 18205d974a7c4592aec0153ef3e44c97 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:17:24.925276 12618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4e4ae9d38ba4fd0a700998e4f4750be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.930949 12766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18205d974a7c4592aec0153ef3e44c97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.936270 12624 tablet_service.cc:1511] Processing CreateTablet for tablet 3b98bb10e3e04334b7fcf7f781593682 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:17:24.936441 12696 tablet_service.cc:1511] Processing CreateTablet for tablet 3b98bb10e3e04334b7fcf7f781593682 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:17:24.937580 12624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b98bb10e3e04334b7fcf7f781593682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.937747 12696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b98bb10e3e04334b7fcf7f781593682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.939705 12691 tablet_service.cc:1511] Processing CreateTablet for tablet 8f91059867c444d89718e83475c089cd (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:17:24.940936 12691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f91059867c444d89718e83475c089cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.940048 12695 tablet_service.cc:1511] Processing CreateTablet for tablet 98a43c5d78c945bb8c5541bc892f4865 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:17:24.942534 12695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98a43c5d78c945bb8c5541bc892f4865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.919865 12824 raft_consensus.cc:740] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.948396 12832 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.953658 12832 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:24.956377 12615 tablet_service.cc:1511] Processing CreateTablet for tablet 6609d5f93ab048558ea46e511ef3b2ec (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:17:24.957676 12832 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:24.959450 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:24.959513 12619 tablet_service.cc:1511] Processing CreateTablet for tablet 8f91059867c444d89718e83475c089cd (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:17:24.959630 12615 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6609d5f93ab048558ea46e511ef3b2ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.960188 12709 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 0.
I20260709 13:17:24.960919 12619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f91059867c444d89718e83475c089cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.959513 12773 tablet_service.cc:1511] Processing CreateTablet for tablet bb2fc32fdf0a4cd981b02de0a47a7f96 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:17:24.961352 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:24.961942 12773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb2fc32fdf0a4cd981b02de0a47a7f96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.962363 12623 tablet_service.cc:1511] Processing CreateTablet for tablet 98a43c5d78c945bb8c5541bc892f4865 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:17:24.963047 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:24.963507 12623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98a43c5d78c945bb8c5541bc892f4865. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:17:24.963928 12747 leader_election.cc:343] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.964753 12832 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:24.965124 12832 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.965440 12832 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.970094 12771 tablet_service.cc:1511] Processing CreateTablet for tablet 6609d5f93ab048558ea46e511ef3b2ec (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:17:24.971446 12771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6609d5f93ab048558ea46e511ef3b2ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.971789 12841 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.972245 12841 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:24.974452 12841 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:24.974819 12832 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:24.978423 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:17:24.980280 12747 leader_election.cc:343] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.989744 12824 consensus_queue.cc:260] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:24.995057 12824 ts_tablet_manager.cc:1434] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.108s	user 0.006s	sys 0.001s
I20260709 13:17:25.004370 12832 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:25.009927 12696 tablet_service.cc:1511] Processing CreateTablet for tablet 4c13fa2aff324f65ad0bf6191ea68fab (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:17:25.019832 12696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c13fa2aff324f65ad0bf6191ea68fab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.021390 12622 tablet_service.cc:1511] Processing CreateTablet for tablet 18b2f67fae7f4963a4b4c562003df2fe (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:17:25.022722 12622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18b2f67fae7f4963a4b4c562003df2fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:24.997511 12698 tablet_service.cc:1511] Processing CreateTablet for tablet c4e4ae9d38ba4fd0a700998e4f4750be (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:17:25.023981 12832 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.024291 12698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4e4ae9d38ba4fd0a700998e4f4750be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.024487 12832 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } }
I20260709 13:17:24.999097 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:25.026178 12709 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.007776 12620 tablet_service.cc:1511] Processing CreateTablet for tablet 4c13fa2aff324f65ad0bf6191ea68fab (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:17:25.031163 12620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c13fa2aff324f65ad0bf6191ea68fab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.010639 12694 tablet_service.cc:1511] Processing CreateTablet for tablet 6609d5f93ab048558ea46e511ef3b2ec (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:17:25.034397 12772 tablet_service.cc:1511] Processing CreateTablet for tablet 522bcf0a155f4da196cb33f1aae90a82 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:17:25.034932 12694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6609d5f93ab048558ea46e511ef3b2ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.036135 12765 tablet_service.cc:1511] Processing CreateTablet for tablet e390d5b6aea04f09ab5dbf11b20a7b98 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:17:25.038002 12824 tablet_bootstrap.cc:492] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:25.038090 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:25.038870 12708 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 0.
I20260709 13:17:25.040185 12747 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:25.041520 12832 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:17:25.042085 12832 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:25.042412 12832 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.042732 12832 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.052927 12709 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:17:25.054945 12693 tablet_service.cc:1511] Processing CreateTablet for tablet 18b2f67fae7f4963a4b4c562003df2fe (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:17:25.059759 12769 tablet_service.cc:1511] Processing CreateTablet for tablet 981fb2f435e24d06a81fc02cf4591cc5 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:17:25.064235 12824 tablet_bootstrap.cc:654] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.026276 12770 tablet_service.cc:1511] Processing CreateTablet for tablet eb7fca96394f4ba9b4fe3d7a6e278878 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:17:25.072410 12832 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } }
I20260709 13:17:25.076045 12697 tablet_service.cc:1511] Processing CreateTablet for tablet 18205d974a7c4592aec0153ef3e44c97 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:17:25.077081 12699 tablet_service.cc:1511] Processing CreateTablet for tablet eb7fca96394f4ba9b4fe3d7a6e278878 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:17:25.077347 12617 tablet_service.cc:1511] Processing CreateTablet for tablet bb2fc32fdf0a4cd981b02de0a47a7f96 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:17:25.087698 12618 tablet_service.cc:1511] Processing CreateTablet for tablet 18205d974a7c4592aec0153ef3e44c97 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:17:25.078812 12765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e390d5b6aea04f09ab5dbf11b20a7b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.090734 12618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18205d974a7c4592aec0153ef3e44c97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.077807 12621 tablet_service.cc:1511] Processing CreateTablet for tablet eb7fca96394f4ba9b4fe3d7a6e278878 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:17:25.094501 12621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb7fca96394f4ba9b4fe3d7a6e278878. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.094841 12773 tablet_service.cc:1511] Processing CreateTablet for tablet 9155962ca7f34a8c83a1780ce0018a59 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:17:25.098091 12693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18b2f67fae7f4963a4b4c562003df2fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.100046 12617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb2fc32fdf0a4cd981b02de0a47a7f96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.093279 12616 tablet_service.cc:1511] Processing CreateTablet for tablet 522bcf0a155f4da196cb33f1aae90a82 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:17:25.102865 12616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 522bcf0a155f4da196cb33f1aae90a82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.105643 12766 tablet_service.cc:1511] Processing CreateTablet for tablet ccf3f471aa044537a9492a86bb48b0aa (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:17:25.106194 12697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18205d974a7c4592aec0153ef3e44c97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.110102 12699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb7fca96394f4ba9b4fe3d7a6e278878. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.109961 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:25.109747 12696 tablet_service.cc:1511] Processing CreateTablet for tablet bb2fc32fdf0a4cd981b02de0a47a7f96 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:17:25.110630 12709 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:25.110692 12690 tablet_service.cc:1511] Processing CreateTablet for tablet 522bcf0a155f4da196cb33f1aae90a82 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:17:25.111331 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:25.112074 12690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 522bcf0a155f4da196cb33f1aae90a82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.112677 12696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb2fc32fdf0a4cd981b02de0a47a7f96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.114362 12618 tablet_service.cc:1511] Processing CreateTablet for tablet e390d5b6aea04f09ab5dbf11b20a7b98 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:17:25.116343 12618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e390d5b6aea04f09ab5dbf11b20a7b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.123494 12620 tablet_service.cc:1511] Processing CreateTablet for tablet ccf3f471aa044537a9492a86bb48b0aa (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:17:25.124912 12620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccf3f471aa044537a9492a86bb48b0aa. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:17:25.088667 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 48531.1us
I20260709 13:17:25.155589 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:25.156126 12598 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:25.156273 12634 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 0.
I20260709 13:17:25.157472 12841 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:25.157917 12841 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.158269 12841 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.159152 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:17:25.160238 12853 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:25.158285 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:25.162352 12708 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.166463 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:25.171872 12825 tablet_bootstrap.cc:492] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:25.172442 12825 ts_tablet_manager.cc:1403] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.330s	user 0.026s	sys 0.043s
I20260709 13:17:25.174330 12823 tablet_bootstrap.cc:492] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:25.174809 12823 ts_tablet_manager.cc:1403] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.343s	user 0.013s	sys 0.002s
I20260709 13:17:25.175062 12841 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } }
I20260709 13:17:25.176187 12634 raft_consensus.cc:2393] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 1: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:17:25.175850 12825 raft_consensus.cc:359] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.176618 12825 raft_consensus.cc:385] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.176937 12825 raft_consensus.cc:740] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
W20260709 13:17:25.177661 12517 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 61263.7us
I20260709 13:17:25.178987 12825 consensus_queue.cc:260] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.178665 12823 raft_consensus.cc:359] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.180006 12823 raft_consensus.cc:385] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.180392 12823 raft_consensus.cc:740] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.181397 12823 consensus_queue.cc:260] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.184158 12823 ts_tablet_manager.cc:1434] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:17:25.189849 12767 tablet_service.cc:1511] Processing CreateTablet for tablet 15d86ce6de4b43719dc715ab6f4de742 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:17:25.136999 12619 tablet_service.cc:1511] Processing CreateTablet for tablet 9155962ca7f34a8c83a1780ce0018a59 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:17:25.142194 12772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 522bcf0a155f4da196cb33f1aae90a82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.145991 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:25.191201 12782 raft_consensus.cc:2393] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 1: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:17:25.147503 12692 tablet_service.cc:1511] Processing CreateTablet for tablet 981fb2f435e24d06a81fc02cf4591cc5 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:17:25.193893 12692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 981fb2f435e24d06a81fc02cf4591cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.147920 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:25.151892 12832 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
W20260709 13:17:25.185864 12598 leader_election.cc:343] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.188319 12708 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:17:25.132998 12624 tablet_service.cc:1511] Processing CreateTablet for tablet 981fb2f435e24d06a81fc02cf4591cc5 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:17:25.198184 12696 tablet_service.cc:1511] Processing CreateTablet for tablet e390d5b6aea04f09ab5dbf11b20a7b98 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:17:25.194063 12823 tablet_bootstrap.cc:492] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:25.203888 12622 tablet_service.cc:1511] Processing CreateTablet for tablet 15d86ce6de4b43719dc715ab6f4de742 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:17:25.206813 12696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e390d5b6aea04f09ab5dbf11b20a7b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.196388 12619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9155962ca7f34a8c83a1780ce0018a59. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:17:25.221961 12597 leader_election.cc:343] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.222441 12597 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:25.235576 12747 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:17:25.236127 12765 tablet_service.cc:1511] Processing CreateTablet for tablet 8e0d0566acd7465f87e20053c75e28fa (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:17:25.236617 12695 tablet_service.cc:1511] Processing CreateTablet for tablet ccf3f471aa044537a9492a86bb48b0aa (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:17:25.238131 12695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccf3f471aa044537a9492a86bb48b0aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.228951 12691 tablet_service.cc:1511] Processing CreateTablet for tablet 9155962ca7f34a8c83a1780ce0018a59 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:17:25.232113 12770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb7fca96394f4ba9b4fe3d7a6e278878. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.226331 12853 raft_consensus.cc:697] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:17:25.259183 12853 consensus_queue.cc:237] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.263087 12764 tablet_service.cc:1511] Processing CreateTablet for tablet e2a417d650a44658bfd5e2f5305185ac (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:17:25.250039 12694 tablet_service.cc:1511] Processing CreateTablet for tablet 15d86ce6de4b43719dc715ab6f4de742 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:17:25.264966 12617 tablet_service.cc:1511] Processing CreateTablet for tablet 8e0d0566acd7465f87e20053c75e28fa (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:17:25.248474 12696 tablet_service.cc:1511] Processing CreateTablet for tablet 8e0d0566acd7465f87e20053c75e28fa (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:17:25.254153 12765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e0d0566acd7465f87e20053c75e28fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.270668 12767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15d86ce6de4b43719dc715ab6f4de742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.248505 12624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 981fb2f435e24d06a81fc02cf4591cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.258296 12691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9155962ca7f34a8c83a1780ce0018a59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.274549 12824 tablet_bootstrap.cc:492] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:25.275156 12824 ts_tablet_manager.cc:1403] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.237s	user 0.012s	sys 0.014s
I20260709 13:17:25.275468 12617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e0d0566acd7465f87e20053c75e28fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.284137 12841 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:25.284814 12841 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:25.285602 12856 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:25.286194 12856 raft_consensus.cc:697] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:17:25.287138 12856 consensus_queue.cc:237] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } }
I20260709 13:17:25.288079 12620 tablet_service.cc:1511] Processing CreateTablet for tablet e2a417d650a44658bfd5e2f5305185ac (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:17:25.289815 12696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e0d0566acd7465f87e20053c75e28fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.292210 12823 tablet_bootstrap.cc:654] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.272063 12769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 981fb2f435e24d06a81fc02cf4591cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.293609 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:25.294193 12825 ts_tablet_manager.cc:1434] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.121s	user 0.013s	sys 0.007s
I20260709 13:17:25.296743 12708 raft_consensus.cc:2393] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 1: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:17:25.297719 12825 tablet_bootstrap.cc:492] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:25.298581 12693 tablet_service.cc:1511] Processing CreateTablet for tablet e2a417d650a44658bfd5e2f5305185ac (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:17:25.301000 12824 raft_consensus.cc:359] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.301734 12824 raft_consensus.cc:385] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.302043 12824 raft_consensus.cc:740] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.302870 12824 consensus_queue.cc:260] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.303898 12764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2a417d650a44658bfd5e2f5305185ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.313510 12766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccf3f471aa044537a9492a86bb48b0aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.316078 12623 tablet_service.cc:1511] Processing CreateTablet for tablet 37e99a6b09d744a9b056b83be1033820 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:17:25.319024 12768 tablet_service.cc:1511] Processing CreateTablet for tablet 37e99a6b09d744a9b056b83be1033820 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:17:25.328609 12773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9155962ca7f34a8c83a1780ce0018a59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.329308 12768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37e99a6b09d744a9b056b83be1033820. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.329691 12622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15d86ce6de4b43719dc715ab6f4de742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.330407 12623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37e99a6b09d744a9b056b83be1033820. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.333892 12620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2a417d650a44658bfd5e2f5305185ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.334748 12692 tablet_service.cc:1511] Processing CreateTablet for tablet 37e99a6b09d744a9b056b83be1033820 (DEFAULT_TABLE table=test-workload [id=0a30ba58723d445b9932a2283c60055a]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:17:25.335165 12694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15d86ce6de4b43719dc715ab6f4de742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.340932 12693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2a417d650a44658bfd5e2f5305185ac. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:17:25.294226 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 72301.6us
I20260709 13:17:25.347034 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:25.348402 12530 catalog_manager.cc:5697] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:25.350178 12692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37e99a6b09d744a9b056b83be1033820. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:25.349100 12597 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:25.364688 12841 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:17:25.369122 12857 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.369396 12825 tablet_bootstrap.cc:654] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.369936 12857 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.372859 12857 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:25.373064 12853 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.373641 12853 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.374200 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:25.375021 12783 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:25.375799 12853 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:25.376354 12597 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:17:25.376847 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:25.377094 12857 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:25.377213 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:25.377470 12857 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.377844 12857 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:25.386467 12747 leader_election.cc:343] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.386685 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:25.377496 12708 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 0.
I20260709 13:17:25.389842 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:17:25.390938 12853 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:25.391299 12853 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.391633 12853 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.398895 12853 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:17:25.402527 12598 leader_election.cc:343] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.404035 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:25.409756 12824 ts_tablet_manager.cc:1434] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.134s	user 0.009s	sys 0.000s
I20260709 13:17:25.409948 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:25.410609 12708 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.411446 12824 tablet_bootstrap.cc:492] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:25.423027 12824 tablet_bootstrap.cc:654] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.426054 12857 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:17:25.426954 12747 leader_election.cc:343] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.414951 12853 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:25.444443 12823 tablet_bootstrap.cc:492] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:25.445417 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:17:25.451469 12598 leader_election.cc:343] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.451964 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:25.452606 12783 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.460244 12825 tablet_bootstrap.cc:492] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:25.460785 12825 ts_tablet_manager.cc:1403] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.163s	user 0.018s	sys 0.018s
I20260709 13:17:25.458407 12857 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:25.469666 12823 ts_tablet_manager.cc:1403] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.276s	user 0.034s	sys 0.017s
I20260709 13:17:25.474109 12828 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.474592 12828 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.476670 12828 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:25.478366 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:25.478660 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:25.478930 12708 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
W20260709 13:17:25.480031 12673 leader_election.cc:343] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.480466 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
W20260709 13:17:25.481705 12672 leader_election.cc:343] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.482148 12672 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:25.483027 12823 raft_consensus.cc:359] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.483471 12853 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:25.483801 12823 raft_consensus.cc:385] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.483958 12853 raft_consensus.cc:697] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:17:25.484140 12823 raft_consensus.cc:740] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.484838 12853 consensus_queue.cc:237] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.486743 12828 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:25.487291 12823 consensus_queue.cc:260] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.488590 12825 raft_consensus.cc:359] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.492728 12825 raft_consensus.cc:385] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.501616 12825 raft_consensus.cc:740] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.502475 12825 consensus_queue.cc:260] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.503072 12823 ts_tablet_manager.cc:1434] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.001s
I20260709 13:17:25.504202 12823 tablet_bootstrap.cc:492] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:25.504513 12825 ts_tablet_manager.cc:1434] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.000s
I20260709 13:17:25.505645 12825 tablet_bootstrap.cc:492] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:25.511059 12783 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:17:25.512041 12824 tablet_bootstrap.cc:492] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:25.512194 12823 tablet_bootstrap.cc:654] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.512590 12824 ts_tablet_manager.cc:1403] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.101s	user 0.016s	sys 0.012s
I20260709 13:17:25.512559 12597 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:25.513944 12857 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:25.514501 12857 raft_consensus.cc:697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:25.515951 12857 consensus_queue.cc:237] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.516640 12824 raft_consensus.cc:359] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.517433 12824 raft_consensus.cc:385] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.517799 12824 raft_consensus.cc:740] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.518519 12824 consensus_queue.cc:260] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.527137 12824 ts_tablet_manager.cc:1434] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.005s
I20260709 13:17:25.528563 12824 tablet_bootstrap.cc:492] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:25.534409 12825 tablet_bootstrap.cc:654] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.543756 12824 tablet_bootstrap.cc:654] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.547116 12823 tablet_bootstrap.cc:492] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:25.547744 12823 ts_tablet_manager.cc:1403] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.044s	user 0.018s	sys 0.017s
I20260709 13:17:25.552652 12823 raft_consensus.cc:359] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.553681 12823 raft_consensus.cc:385] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.554186 12823 raft_consensus.cc:740] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.555364 12823 consensus_queue.cc:260] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.559626 12823 ts_tablet_manager.cc:1434] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.005s
I20260709 13:17:25.561121 12823 tablet_bootstrap.cc:492] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:25.565521 12824 tablet_bootstrap.cc:492] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:25.566097 12824 ts_tablet_manager.cc:1403] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.038s	user 0.008s	sys 0.014s
I20260709 13:17:25.569108 12824 raft_consensus.cc:359] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.569873 12824 raft_consensus.cc:385] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.570173 12824 raft_consensus.cc:740] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.571171 12853 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.570974 12824 consensus_queue.cc:260] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.571630 12853 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.573649 12824 ts_tablet_manager.cc:1434] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 13:17:25.573859 12853 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:25.574662 12824 tablet_bootstrap.cc:492] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:25.575487 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:25.576871 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:25.577594 12708 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 0.
I20260709 13:17:25.579093 12747 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:25.580140 12853 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:25.580585 12853 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.580999 12853 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.581135 12823 tablet_bootstrap.cc:654] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.581135 12824 tablet_bootstrap.cc:654] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:25.582175 12747 leader_election.cc:343] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.592965 12857 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.593997 12823 tablet_bootstrap.cc:492] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:25.593477 12857 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.594509 12823 ts_tablet_manager.cc:1403] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.034s	user 0.008s	sys 0.011s
I20260709 13:17:25.593447 12532 catalog_manager.cc:5697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 0 to 1, leader changed from <none> to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:25.595791 12857 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:25.597441 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:25.597338 12823 raft_consensus.cc:359] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.598127 12823 raft_consensus.cc:385] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.598457 12823 raft_consensus.cc:740] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.599852 12828 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.600317 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:25.599577 12531 catalog_manager.cc:5697] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:25.600317 12828 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.601199 12708 raft_consensus.cc:2393] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 1: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:17:25.556113 12825 tablet_bootstrap.cc:492] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:25.601974 12825 ts_tablet_manager.cc:1403] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.097s	user 0.010s	sys 0.012s
I20260709 13:17:25.602275 12598 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:25.602638 12828 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:25.603528 12857 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.605218 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:25.605893 12634 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:25.606876 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:17:25.608306 12672 leader_election.cc:343] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.608065 12853 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.609447 12673 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:25.610584 12828 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:25.611001 12828 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.611368 12828 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.611651 12824 tablet_bootstrap.cc:492] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:25.612136 12824 ts_tablet_manager.cc:1403] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.038s	user 0.008s	sys 0.009s
I20260709 13:17:25.599422 12823 consensus_queue.cc:260] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.615489 12857 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:25.618217 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:17:25.620067 12747 leader_election.cc:343] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.612928 12531 catalog_manager.cc:5697] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:25.622452 12825 raft_consensus.cc:359] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.623217 12825 raft_consensus.cc:385] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.623525 12825 raft_consensus.cc:740] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.624333 12825 consensus_queue.cc:260] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.626683 12825 ts_tablet_manager.cc:1434] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:17:25.627771 12825 tablet_bootstrap.cc:492] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:25.630162 12853 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:25.632738 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:25.633599 12708 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.636336 12824 raft_consensus.cc:359] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.637141 12824 raft_consensus.cc:385] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.642395 12828 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.643901 12708 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:17:25.645718 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:25.652632 12634 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.652650 12825 tablet_bootstrap.cc:654] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.661077 12634 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:25.637485 12824 raft_consensus.cc:740] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.648488 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:17:25.662671 12672 leader_election.cc:343] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.663760 12824 consensus_queue.cc:260] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.666033 12824 ts_tablet_manager.cc:1434] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.054s	user 0.009s	sys 0.000s
I20260709 13:17:25.649012 12828 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:25.668301 12824 tablet_bootstrap.cc:492] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:25.670509 12823 ts_tablet_manager.cc:1434] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.076s	user 0.020s	sys 0.021s
I20260709 13:17:25.671482 12823 tablet_bootstrap.cc:492] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:25.672494 12825 tablet_bootstrap.cc:492] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:25.672974 12825 ts_tablet_manager.cc:1403] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.045s	user 0.015s	sys 0.001s
I20260709 13:17:25.674700 12747 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:17:25.676151 12853 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:25.676666 12853 raft_consensus.cc:697] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:17:25.677206 12823 tablet_bootstrap.cc:654] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.677515 12853 consensus_queue.cc:237] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.679145 12825 raft_consensus.cc:359] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.681592 12673 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:25.683076 12875 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:25.683593 12875 raft_consensus.cc:697] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:25.684440 12875 consensus_queue.cc:237] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.689007 12823 tablet_bootstrap.cc:492] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:25.689527 12823 ts_tablet_manager.cc:1403] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.004s
I20260709 13:17:25.692096 12823 raft_consensus.cc:359] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.692754 12823 raft_consensus.cc:385] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.693050 12823 raft_consensus.cc:740] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.693871 12823 consensus_queue.cc:260] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.696105 12823 ts_tablet_manager.cc:1434] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.001s
I20260709 13:17:25.696998 12823 tablet_bootstrap.cc:492] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:25.698629 12824 tablet_bootstrap.cc:654] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.679865 12825 raft_consensus.cc:385] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.703907 12825 raft_consensus.cc:740] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.702364 12823 tablet_bootstrap.cc:654] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.704769 12825 consensus_queue.cc:260] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.707639 12825 ts_tablet_manager.cc:1434] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.034s	user 0.002s	sys 0.006s
I20260709 13:17:25.709268 12825 tablet_bootstrap.cc:492] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:25.711822 12531 catalog_manager.cc:5697] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:25.716212 12823 tablet_bootstrap.cc:492] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:25.716699 12823 ts_tablet_manager.cc:1403] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.004s
I20260709 13:17:25.717123 12825 tablet_bootstrap.cc:654] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.720957 12823 raft_consensus.cc:359] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.721951 12823 raft_consensus.cc:385] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.722319 12823 raft_consensus.cc:740] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.723168 12823 consensus_queue.cc:260] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.727768 12823 ts_tablet_manager.cc:1434] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:17:25.729476 12823 tablet_bootstrap.cc:492] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:25.730806 12853 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:25.737545 12823 tablet_bootstrap.cc:654] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.749423 12824 tablet_bootstrap.cc:492] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:25.749950 12824 ts_tablet_manager.cc:1403] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.082s	user 0.012s	sys 0.002s
I20260709 13:17:25.749621 12853 consensus_queue.cc:1048] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:25.752806 12824 raft_consensus.cc:359] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.753505 12824 raft_consensus.cc:385] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.753856 12824 raft_consensus.cc:740] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.758881 12824 consensus_queue.cc:260] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.761322 12824 ts_tablet_manager.cc:1434] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:17:25.765614 12824 tablet_bootstrap.cc:492] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:25.768858 12853 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:25.778021 12824 tablet_bootstrap.cc:654] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.777645 12532 catalog_manager.cc:5697] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:25.786996 12825 tablet_bootstrap.cc:492] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:25.787631 12825 ts_tablet_manager.cc:1403] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.079s	user 0.019s	sys 0.018s
I20260709 13:17:25.791958 12823 tablet_bootstrap.cc:492] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:25.792475 12823 ts_tablet_manager.cc:1403] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.063s	user 0.028s	sys 0.012s
I20260709 13:17:25.795158 12823 raft_consensus.cc:359] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.795850 12823 raft_consensus.cc:385] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.796128 12823 raft_consensus.cc:740] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.796838 12823 consensus_queue.cc:260] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.798283 12853 consensus_queue.cc:1048] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:25.799163 12823 ts_tablet_manager.cc:1434] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:17:25.800210 12823 tablet_bootstrap.cc:492] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:25.805790 12823 tablet_bootstrap.cc:654] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.807248 12825 raft_consensus.cc:359] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.808151 12825 raft_consensus.cc:385] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.808460 12825 raft_consensus.cc:740] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.809376 12825 consensus_queue.cc:260] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.819846 12825 ts_tablet_manager.cc:1434] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.001s
I20260709 13:17:25.820811 12825 tablet_bootstrap.cc:492] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:25.838201 12825 tablet_bootstrap.cc:654] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.843465 12824 tablet_bootstrap.cc:492] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:25.845073 12824 ts_tablet_manager.cc:1403] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.080s	user 0.008s	sys 0.007s
I20260709 13:17:25.846529 12823 tablet_bootstrap.cc:492] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:25.847081 12823 ts_tablet_manager.cc:1403] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.047s	user 0.009s	sys 0.005s
I20260709 13:17:25.847972 12824 raft_consensus.cc:359] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.850015 12823 raft_consensus.cc:359] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.850786 12823 raft_consensus.cc:385] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.851065 12823 raft_consensus.cc:740] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.851732 12823 consensus_queue.cc:260] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.853840 12823 ts_tablet_manager.cc:1434] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:17:25.854631 12823 tablet_bootstrap.cc:492] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:25.855540 12825 tablet_bootstrap.cc:492] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:25.855973 12825 ts_tablet_manager.cc:1403] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.000s
I20260709 13:17:25.858641 12825 raft_consensus.cc:359] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.859310 12825 raft_consensus.cc:385] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.859458 12823 tablet_bootstrap.cc:654] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.859618 12825 raft_consensus.cc:740] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.860394 12825 consensus_queue.cc:260] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.862360 12825 ts_tablet_manager.cc:1434] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:17:25.863466 12825 tablet_bootstrap.cc:492] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:25.869195 12825 tablet_bootstrap.cc:654] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.870990 12824 raft_consensus.cc:385] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.871280 12823 tablet_bootstrap.cc:492] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:25.871438 12824 raft_consensus.cc:740] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.871814 12823 ts_tablet_manager.cc:1403] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.000s
I20260709 13:17:25.872627 12824 consensus_queue.cc:260] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.874668 12823 raft_consensus.cc:359] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.875396 12823 raft_consensus.cc:385] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.875666 12823 raft_consensus.cc:740] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.876469 12823 consensus_queue.cc:260] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.878683 12823 ts_tablet_manager.cc:1434] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:17:25.880455 12823 tablet_bootstrap.cc:492] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:25.883061 12824 ts_tablet_manager.cc:1434] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.038s	user 0.004s	sys 0.003s
I20260709 13:17:25.884160 12825 tablet_bootstrap.cc:492] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:25.884673 12825 ts_tablet_manager.cc:1403] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.000s
I20260709 13:17:25.884263 12824 tablet_bootstrap.cc:492] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:25.886116 12875 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.886577 12875 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.887001 12823 tablet_bootstrap.cc:654] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.888618 12875 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:25.890642 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:25.893697 12824 tablet_bootstrap.cc:654] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.896411 12823 tablet_bootstrap.cc:492] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:25.896703 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:25.896943 12823 ts_tablet_manager.cc:1403] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.003s
I20260709 13:17:25.897444 12633 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:25.898833 12673 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:25.899852 12875 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:25.900220 12875 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.900573 12875 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.900372 12823 raft_consensus.cc:359] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.901271 12823 raft_consensus.cc:385] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.901654 12823 raft_consensus.cc:740] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.902634 12823 consensus_queue.cc:260] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.904925 12823 ts_tablet_manager.cc:1434] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:17:25.905431 12824 tablet_bootstrap.cc:492] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:25.905790 12823 tablet_bootstrap.cc:492] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:25.905872 12824 ts_tablet_manager.cc:1403] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.010s
I20260709 13:17:25.908466 12824 raft_consensus.cc:359] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.909166 12824 raft_consensus.cc:385] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.909466 12824 raft_consensus.cc:740] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.910358 12824 consensus_queue.cc:260] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.912149 12823 tablet_bootstrap.cc:654] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.907582 12875 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.915171 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:25.915741 12633 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.916662 12825 raft_consensus.cc:359] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.917497 12825 raft_consensus.cc:385] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.917847 12825 raft_consensus.cc:740] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.918705 12825 consensus_queue.cc:260] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.920210 12783 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:25.920536 12824 ts_tablet_manager.cc:1434] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:17:25.921149 12825 ts_tablet_manager.cc:1434] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 13:17:25.922016 12824 tablet_bootstrap.cc:492] T a4d7de736149496b86a37ba2ca92670d P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:25.923759 12825 tablet_bootstrap.cc:492] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:25.925549 12823 tablet_bootstrap.cc:492] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:25.926218 12823 ts_tablet_manager.cc:1403] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.007s
I20260709 13:17:25.928262 12853 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.928866 12853 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:17:25.929327 12579 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:25.929845 12857 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.928906 12823 raft_consensus.cc:359] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.934558 12823 raft_consensus.cc:385] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.934294 12857 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.934868 12823 raft_consensus.cc:740] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.935801 12823 consensus_queue.cc:260] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.936342 12857 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:25.930950 12853 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:25.931828 12825 tablet_bootstrap.cc:654] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.933079 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:25.937753 12634 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 0.
I20260709 13:17:25.937877 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:25.938439 12708 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:25.939421 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:17:25.939685 12824 tablet_bootstrap.cc:654] T a4d7de736149496b86a37ba2ca92670d P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.933100 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:25.940462 12709 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 0.
I20260709 13:17:25.940575 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:25.941140 12783 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:25.941745 12823 ts_tablet_manager.cc:1434] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:17:25.942701 12823 tablet_bootstrap.cc:492] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:25.939796 12598 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:25.951782 12824 tablet_bootstrap.cc:492] T a4d7de736149496b86a37ba2ca92670d P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:25.946798 12633 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:25.961805 12824 ts_tablet_manager.cc:1403] T a4d7de736149496b86a37ba2ca92670d P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.040s	user 0.024s	sys 0.004s
I20260709 13:17:25.962289 12828 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.962363 12673 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:25.962869 12828 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.947325 12853 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:25.964926 12853 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.965014 12828 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:25.965479 12853 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.965730 12828 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:25.966358 12828 raft_consensus.cc:697] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:25.966574 12841 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:25.967319 12828 consensus_queue.cc:237] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.964938 12824 raft_consensus.cc:359] T a4d7de736149496b86a37ba2ca92670d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.971899 12824 raft_consensus.cc:385] T a4d7de736149496b86a37ba2ca92670d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.972340 12824 raft_consensus.cc:740] T a4d7de736149496b86a37ba2ca92670d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.973471 12824 consensus_queue.cc:260] T a4d7de736149496b86a37ba2ca92670d P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.974685 12853 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.955369 12823 tablet_bootstrap.cc:654] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.975986 12856 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:25.976727 12853 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:25.978338 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:25.979393 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:25.980106 12709 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.985446 12824 ts_tablet_manager.cc:1434] T a4d7de736149496b86a37ba2ca92670d P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.001s
I20260709 13:17:25.991624 12856 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.992306 12856 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.995016 12856 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:25.995532 12902 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.996100 12902 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:25.997470 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:25.998347 12632 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 0.
I20260709 13:17:25.998880 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:25.999686 12824 tablet_bootstrap.cc:492] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:26.003618 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:25.947554 12857 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:26.004498 12857 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.004868 12823 tablet_bootstrap.cc:492] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:26.003940 12828 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.999439 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:26.005336 12823 ts_tablet_manager.cc:1403] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.063s	user 0.012s	sys 0.003s
I20260709 13:17:26.005452 12707 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 0.
I20260709 13:17:26.004897 12857 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.005501 12673 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:26.000799 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:17:26.007007 12856 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:26.007563 12828 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.007411 12856 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.008551 12856 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.010109 12828 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:26.010596 12828 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:26.010952 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:17:26.014178 12673 leader_election.cc:343] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:26.014746 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:17:26.016410 12672 leader_election.cc:343] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:26.016873 12824 tablet_bootstrap.cc:654] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.016988 12672 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:26.017953 12902 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:26.018465 12902 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:26.018544 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:26.019304 12632 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:26.019734 12875 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:26.007954 12823 raft_consensus.cc:359] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.020606 12823 raft_consensus.cc:385] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.020569 12908 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.021008 12823 raft_consensus.cc:740] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.021093 12673 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:26.021306 12908 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.021733 12823 consensus_queue.cc:260] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.023815 12823 ts_tablet_manager.cc:1434] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 13:17:26.024873 12825 tablet_bootstrap.cc:492] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:26.026059 12823 tablet_bootstrap.cc:492] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:26.026544 12825 ts_tablet_manager.cc:1403] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.103s	user 0.019s	sys 0.015s
I20260709 13:17:26.024809 12908 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:26.025049 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:26.029336 12709 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:17:26.030606 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:26.031172 12825 raft_consensus.cc:359] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.025902 12901 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.031602 12855 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:26.024659 12902 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:26.031960 12825 raft_consensus.cc:385] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.032167 12902 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.032290 12825 raft_consensus.cc:740] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.032624 12902 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.033051 12825 consensus_queue.cc:260] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.034658 12707 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.036171 12901 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.043565 12901 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:26.046458 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:26.047314 12633 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:26.048792 12673 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:26.049962 12901 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:26.050448 12901 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.051426 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:26.052861 12823 tablet_bootstrap.cc:654] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.053579 12901 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.054292 12855 raft_consensus.cc:697] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:17:26.057899 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:26.058511 12783 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.058712 12855 consensus_queue.cc:237] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.055516 12902 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.071374 12857 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.075237 12857 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:26.072743 12853 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.077514 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:26.077850 12853 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.080117 12634 raft_consensus.cc:2393] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 1: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:17:26.083835 12875 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.084394 12875 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.081523 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:26.087285 12825 ts_tablet_manager.cc:1434] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.060s	user 0.004s	sys 0.022s
I20260709 13:17:26.088716 12825 tablet_bootstrap.cc:492] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:26.066064 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:26.104929 12828 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.104633 12856 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.105398 12828 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.105829 12857 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:26.111649 12824 tablet_bootstrap.cc:492] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:26.165194 12824 ts_tablet_manager.cc:1403] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.167s	user 0.008s	sys 0.030s
I20260709 13:17:26.135131 12857 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.107568 12828 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:26.109186 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:26.145170 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:26.142304 12912 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.146294 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:26.170408 12853 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:26.186627 12907 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.187093 12633 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:26.111323 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:26.170073 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:26.183861 12782 raft_consensus.cc:2393] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 1: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
W20260709 13:17:26.170368 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 59908.3us
W20260709 13:17:26.182691 12672 leader_election.cc:343] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:26.200590 12707 raft_consensus.cc:2393] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 1: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:17:26.184289 12857 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.206025 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:26.206790 12631 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.208449 12857 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:26.197702 12875 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:26.205178 12902 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:26.209760 12783 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:26.205070 12907 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.200366 12912 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.218210 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:17:26.218437 12673 leader_election.cc:343] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:26.209748 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:26.198361 12781 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:26.195175 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:26.193760 12856 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:26.231858 12907 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:26.232431 12912 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:26.234751 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:26.238188 12632 raft_consensus.cc:2393] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 1: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:17:26.240248 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:26.209507 12901 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.243603 12825 tablet_bootstrap.cc:654] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.231314 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:26.475615 12707 raft_consensus.cc:2393] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 1: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:17:26.545981 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:26.547020 12705 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 0.
I20260709 13:17:26.671922 12824 raft_consensus.cc:359] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.673003 12824 raft_consensus.cc:385] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.673300 12824 raft_consensus.cc:740] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.686430 12824 consensus_queue.cc:260] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.689368 12824 ts_tablet_manager.cc:1434] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.524s	user 0.008s	sys 0.000s
W20260709 13:17:26.689780 12824 ts_tablet_manager.cc:1467] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e: Tablet startup took 524ms
I20260709 13:17:26.697872 12823 tablet_bootstrap.cc:492] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:26.698545 12823 ts_tablet_manager.cc:1403] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.674s	user 0.006s	sys 0.029s
I20260709 13:17:26.704107 12823 raft_consensus.cc:359] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.704639 12823 raft_consensus.cc:385] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:17:26.690117 12824 ts_tablet_manager.cc:1469] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e: Trace:
0709 13:17:23.630059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:23.630231 (+   172us) service_pool.cc:224] Handling call
0709 13:17:23.633053 (+  2822us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:17:23.633150 (+    97us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:17:23.700576 (+ 67426us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:23.830524 (+129948us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:23.830965 (+   441us) inbound_call.cc:177] Queueing success response
0709 13:17:25.998503 (+2167538us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:17:26.165828 (+167325us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:17:26.246816 (+ 80988us) tablet_replica.cc:261] Starting instrumentation
0709 13:17:26.247319 (+   503us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":106,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":1531,"tablet-open.queue_time_us":2167585}
I20260709 13:17:26.704908 12823 raft_consensus.cc:740] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.706542 12823 consensus_queue.cc:260] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.708879 12823 ts_tablet_manager.cc:1434] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 13:17:26.709856 12823 tablet_bootstrap.cc:492] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:26.709728 12825 tablet_bootstrap.cc:492] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:26.710719 12825 ts_tablet_manager.cc:1403] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.622s	user 0.169s	sys 0.184s
I20260709 13:17:26.717370 12823 tablet_bootstrap.cc:654] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.756716 12823 tablet_bootstrap.cc:492] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:26.757159 12823 ts_tablet_manager.cc:1403] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.048s	user 0.005s	sys 0.007s
I20260709 13:17:26.759712 12823 raft_consensus.cc:359] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.760555 12823 raft_consensus.cc:385] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.760807 12823 raft_consensus.cc:740] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.761595 12823 consensus_queue.cc:260] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.763557 12823 ts_tablet_manager.cc:1434] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 13:17:26.764459 12823 tablet_bootstrap.cc:492] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:26.774734 12823 tablet_bootstrap.cc:654] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.779460 12828 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.779953 12828 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.780869 12823 tablet_bootstrap.cc:492] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:26.781452 12823 ts_tablet_manager.cc:1403] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.017s	user 0.003s	sys 0.008s
I20260709 13:17:26.786319 12823 raft_consensus.cc:359] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.786899 12823 raft_consensus.cc:385] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.787184 12823 raft_consensus.cc:740] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.787827 12823 consensus_queue.cc:260] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.790441 12823 ts_tablet_manager.cc:1434] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.004s
I20260709 13:17:26.791493 12823 tablet_bootstrap.cc:492] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:26.797183 12823 tablet_bootstrap.cc:654] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.802098 12823 tablet_bootstrap.cc:492] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:26.802561 12823 ts_tablet_manager.cc:1403] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 13:17:26.805662 12823 raft_consensus.cc:359] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.806242 12823 raft_consensus.cc:385] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.806469 12823 raft_consensus.cc:740] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.807137 12823 consensus_queue.cc:260] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.809067 12823 ts_tablet_manager.cc:1434] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:17:26.810027 12823 tablet_bootstrap.cc:492] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:26.815464 12823 tablet_bootstrap.cc:654] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.821161 12823 tablet_bootstrap.cc:492] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:26.821482 12823 ts_tablet_manager.cc:1403] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.012s	user 0.007s	sys 0.003s
I20260709 13:17:26.824034 12823 raft_consensus.cc:359] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.824604 12823 raft_consensus.cc:385] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.824833 12823 raft_consensus.cc:740] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.825453 12823 consensus_queue.cc:260] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.828155 12823 ts_tablet_manager.cc:1434] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:17:26.829089 12823 tablet_bootstrap.cc:492] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:26.835498 12823 tablet_bootstrap.cc:654] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.842300 12823 tablet_bootstrap.cc:492] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:26.842780 12823 ts_tablet_manager.cc:1403] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.003s
I20260709 13:17:26.843429 12922 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.843926 12922 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.845359 12823 raft_consensus.cc:359] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.846041 12823 raft_consensus.cc:385] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.846102 12922 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:17:26.234365 12672 leader_election.cc:343] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:26.846350 12823 raft_consensus.cc:740] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.846724 12672 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:26.847086 12823 consensus_queue.cc:260] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.849303 12926 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:17:26.852576 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 618189us
I20260709 13:17:26.858443 12672 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:26.859041 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:26.859710 12926 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.860217 12926 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.860569 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:26.864040 12782 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.864830 12926 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:26.866039 12926 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:26.866413 12926 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.866788 12926 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.874874 12825 raft_consensus.cc:359] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.876173 12825 raft_consensus.cc:385] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.876551 12825 raft_consensus.cc:740] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.877009 12901 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:26.877498 12825 consensus_queue.cc:260] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.880636 12825 ts_tablet_manager.cc:1434] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.170s	user 0.008s	sys 0.000s
I20260709 13:17:26.881750 12825 tablet_bootstrap.cc:492] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:26.884512 12926 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.889269 12926 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:26.889840 12828 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:26.892184 12912 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:26.889406 12747 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:17:26.895188 12672 leader_election.cc:343] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:26.895401 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:17:26.896714 12672 leader_election.cc:343] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:26.897365 12672 leader_election.cc:343] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:26.898325 12922 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:17:26.904458 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:26.905354 12626 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.908663 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:26.909793 12708 raft_consensus.cc:2393] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 1: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:17:26.903510 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:26.911450 12779 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.920725 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:26.928872 12779 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:26.929798 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:26.930262 12672 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:17:26.931838 12672 leader_election.cc:343] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:26.931913 12928 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:26.933061 12928 raft_consensus.cc:697] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:26.934405 12928 consensus_queue.cc:237] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:17:26.942672 12579 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:26.943655 12626 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
W20260709 13:17:26.955206 12631 consensus_meta.cc:287] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.748s	user 0.013s	sys 0.000s
I20260709 13:17:26.955642 12631 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:26.974464 12855 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:26.974951 12855 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.975255 12855 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.981887 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:26.982434 12855 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:26.983799 12853 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:26.984553 12855 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:26.985960 12853 consensus_queue.cc:1048] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:26.986410 12626 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.987879 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:26.988634 12631 raft_consensus.cc:2393] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 1: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:17:26.990069 12634 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.991135 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:26.991484 12598 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:26.991711 12706 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.992702 12598 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:26.995523 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:26.996783 12625 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.995206 12857 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:17:27.010270 12598 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:27.010075 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:27.015962 12706 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:17:27.017757 12747 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:27.018703 12855 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:27.019196 12855 raft_consensus.cc:697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:17:27.020095 12855 consensus_queue.cc:237] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.021380 12823 ts_tablet_manager.cc:1434] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.178s	user 0.038s	sys 0.041s
I20260709 13:17:27.022234 12823 tablet_bootstrap.cc:492] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:27.027161 12934 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:27.029446 12857 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:27.031873 12625 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
W20260709 13:17:27.035734 12673 leader_election.cc:343] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.036355 12673 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:27.037694 12823 tablet_bootstrap.cc:654] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.041110 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:27.042820 12825 tablet_bootstrap.cc:654] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:27.044008 12673 leader_election.cc:343] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.050631 12915 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:27.052522 12673 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:27.056066 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:27.057961 12915 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:27.058696 12673 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:27.059770 12915 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:27.060248 12915 raft_consensus.cc:697] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:27.061071 12915 consensus_queue.cc:237] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:17:27.062270 12673 leader_election.cc:343] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.062732 12673 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:27.063833 12922 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:17:27.063846 12673 leader_election.cc:343] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:27.072932 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 62797.7us
I20260709 13:17:27.077980 12927 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:17:26.897531 12901 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.184s
I20260709 13:17:27.083431 12824 tablet_bootstrap.cc:492] T 866523bf2d2f442f971d504267ed200f P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:27.088121 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:27.075614 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:27.087498 12928 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:17:27.092404 12913 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:17:27.088589 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:27.093699 12912 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:27.090370 12630 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.096454 12913 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:17:27.102152 12823 tablet_bootstrap.cc:492] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:27.102629 12823 ts_tablet_manager.cc:1403] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.081s	user 0.010s	sys 0.006s
I20260709 13:17:27.104012 12825 tablet_bootstrap.cc:492] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:27.104481 12825 ts_tablet_manager.cc:1403] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.223s	user 0.023s	sys 0.003s
I20260709 13:17:27.109205 12824 tablet_bootstrap.cc:654] T 866523bf2d2f442f971d504267ed200f P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.111297 12823 raft_consensus.cc:359] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.112236 12823 raft_consensus.cc:385] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.112576 12823 raft_consensus.cc:740] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.116420 12823 consensus_queue.cc:260] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.119310 12823 ts_tablet_manager.cc:1434] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:17:27.119833 12825 raft_consensus.cc:359] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.120213 12823 tablet_bootstrap.cc:492] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:27.120498 12825 raft_consensus.cc:385] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.120604 12913 consensus_queue.cc:1048] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:27.120814 12825 raft_consensus.cc:740] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.119746 12630 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:27.123567 12825 consensus_queue.cc:260] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.128795 12673 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:27.130323 12915 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:27.130820 12915 raft_consensus.cc:697] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:27.131596 12915 consensus_queue.cc:237] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.138931 12825 ts_tablet_manager.cc:1434] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.034s	user 0.003s	sys 0.003s
I20260709 13:17:27.140164 12825 tablet_bootstrap.cc:492] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:27.178256 12823 tablet_bootstrap.cc:654] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.186044 12927 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.186595 12927 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:17:27.186669 12672 leader_election.cc:343] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.194831 12532 catalog_manager.cc:5697] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:27.195727 12927 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:27.197481 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:27.200029 12856 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:27.200487 12673 leader_election.cc:343] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.147507 12529 catalog_manager.cc:5697] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:27.194954 12672 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:27.206619 12901 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:27.207198 12922 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.207700 12922 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.208118 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:27.207234 12901 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.209797 12901 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.215174 12901 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:27.200536 12856 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.213515 12922 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:17:27.213485 12672 leader_election.cc:343] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.218346 12672 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:27.219314 12922 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:27.220165 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:27.220945 12628 raft_consensus.cc:2393] T a4d7de736149496b86a37ba2ca92670d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 1: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:17:27.220976 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:27.221277 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:17:27.222534 12673 leader_election.cc:343] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.222795 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:27.226071 12824 tablet_bootstrap.cc:492] T 866523bf2d2f442f971d504267ed200f P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:27.226574 12824 ts_tablet_manager.cc:1403] T 866523bf2d2f442f971d504267ed200f P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.143s	user 0.023s	sys 0.019s
I20260709 13:17:27.230325 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:17:27.232132 12673 leader_election.cc:343] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.234279 12856 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:27.235210 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:27.237769 12747 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:27.239410 12856 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.244637 12823 tablet_bootstrap.cc:492] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:27.245085 12823 ts_tablet_manager.cc:1403] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.125s	user 0.019s	sys 0.008s
W20260709 13:17:27.236562 12672 leader_election.cc:343] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.246836 12672 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:27.247709 12928 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:17:27.249140 12672 leader_election.cc:343] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.249620 12672 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:27.250371 12928 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:27.251374 12825 tablet_bootstrap.cc:654] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.246192 12824 raft_consensus.cc:359] T 866523bf2d2f442f971d504267ed200f P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.252866 12824 raft_consensus.cc:385] T 866523bf2d2f442f971d504267ed200f P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.253244 12824 raft_consensus.cc:740] T 866523bf2d2f442f971d504267ed200f P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.254180 12824 consensus_queue.cc:260] T 866523bf2d2f442f971d504267ed200f P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.255527 12856 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:27.256966 12824 ts_tablet_manager.cc:1434] T 866523bf2d2f442f971d504267ed200f P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 13:17:27.258252 12824 tablet_bootstrap.cc:492] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:27.263276 12823 raft_consensus.cc:359] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.264067 12823 raft_consensus.cc:385] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.264417 12823 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.265281 12823 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.267616 12824 tablet_bootstrap.cc:654] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.268039 12823 ts_tablet_manager.cc:1434] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:17:27.269238 12823 tablet_bootstrap.cc:492] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:27.273496 12824 tablet_bootstrap.cc:492] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:27.274048 12824 ts_tablet_manager.cc:1403] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.000s
I20260709 13:17:27.275256 12825 tablet_bootstrap.cc:492] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:27.275780 12825 ts_tablet_manager.cc:1403] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.136s	user 0.012s	sys 0.008s
I20260709 13:17:27.279575 12825 raft_consensus.cc:359] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.280274 12825 raft_consensus.cc:385] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.280548 12825 raft_consensus.cc:740] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.281356 12825 consensus_queue.cc:260] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.277113 12824 raft_consensus.cc:359] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.283668 12824 raft_consensus.cc:385] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.283699 12825 ts_tablet_manager.cc:1434] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:17:27.284096 12824 raft_consensus.cc:740] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.284852 12825 tablet_bootstrap.cc:492] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:27.284793 12824 consensus_queue.cc:260] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.287134 12824 ts_tablet_manager.cc:1434] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:17:27.288045 12824 tablet_bootstrap.cc:492] T 059429c0c4024dfb941d1ff325fe8f98 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:27.291560 12825 tablet_bootstrap.cc:654] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.294744 12824 tablet_bootstrap.cc:654] T 059429c0c4024dfb941d1ff325fe8f98 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.355376 12823 tablet_bootstrap.cc:654] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.364197 12529 catalog_manager.cc:5697] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:27.368693 12825 tablet_bootstrap.cc:492] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:27.368960 12823 tablet_bootstrap.cc:492] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:27.369180 12825 ts_tablet_manager.cc:1403] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.085s	user 0.024s	sys 0.044s
I20260709 13:17:27.369858 12823 ts_tablet_manager.cc:1403] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.101s	user 0.007s	sys 0.005s
I20260709 13:17:27.370813 12824 tablet_bootstrap.cc:492] T 059429c0c4024dfb941d1ff325fe8f98 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:27.371261 12824 ts_tablet_manager.cc:1403] T 059429c0c4024dfb941d1ff325fe8f98 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.083s	user 0.037s	sys 0.037s
I20260709 13:17:27.371922 12825 raft_consensus.cc:359] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.372807 12825 raft_consensus.cc:385] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.372921 12928 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.373106 12825 raft_consensus.cc:740] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.373384 12928 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.373814 12825 consensus_queue.cc:260] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.374059 12824 raft_consensus.cc:359] T 059429c0c4024dfb941d1ff325fe8f98 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.374776 12824 raft_consensus.cc:385] T 059429c0c4024dfb941d1ff325fe8f98 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.375255 12824 raft_consensus.cc:740] T 059429c0c4024dfb941d1ff325fe8f98 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.375515 12928 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:27.376144 12825 ts_tablet_manager.cc:1434] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:17:27.376046 12824 consensus_queue.cc:260] T 059429c0c4024dfb941d1ff325fe8f98 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.377084 12825 tablet_bootstrap.cc:492] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:27.377700 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:27.378067 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:27.378322 12630 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:27.378683 12824 ts_tablet_manager.cc:1434] T 059429c0c4024dfb941d1ff325fe8f98 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:17:27.379536 12673 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:27.380244 12824 tablet_bootstrap.cc:492] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
W20260709 13:17:27.380429 12672 leader_election.cc:343] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.381142 12529 catalog_manager.cc:5697] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:27.383150 12928 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:27.372306 12823 raft_consensus.cc:359] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.383584 12823 raft_consensus.cc:385] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.383549 12928 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.384316 12928 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.390170 12825 tablet_bootstrap.cc:654] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.383958 12823 raft_consensus.cc:740] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.392625 12824 tablet_bootstrap.cc:654] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.393172 12823 consensus_queue.cc:260] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.391845 12928 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.396938 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:27.397699 12630 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.401228 12824 tablet_bootstrap.cc:492] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:27.401734 12824 ts_tablet_manager.cc:1403] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.022s	user 0.008s	sys 0.007s
I20260709 13:17:27.404378 12824 raft_consensus.cc:359] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.405053 12824 raft_consensus.cc:385] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.405341 12824 raft_consensus.cc:740] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.391314 12529 catalog_manager.cc:5697] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:27.407927 12824 consensus_queue.cc:260] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.410506 12824 ts_tablet_manager.cc:1434] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:17:27.411319 12825 tablet_bootstrap.cc:492] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:27.411577 12824 tablet_bootstrap.cc:492] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:27.414574 12915 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:27.418093 12934 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.418934 12915 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.418717 12934 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.419397 12915 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.422436 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:27.422851 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:27.423460 12708 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
W20260709 13:17:27.423940 12597 leader_election.cc:343] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.426154 12598 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:27.426992 12857 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:27.427197 12824 tablet_bootstrap.cc:654] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.427378 12857 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.427772 12857 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.429960 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:27.431107 12825 ts_tablet_manager.cc:1403] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.054s	user 0.011s	sys 0.011s
I20260709 13:17:27.431924 12934 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:17:27.433350 12673 leader_election.cc:343] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.435714 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:17:27.437427 12672 leader_election.cc:343] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.437919 12672 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:27.438648 12922 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:27.439548 12915 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:27.438649 12857 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.441454 12857 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:27.442214 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:27.442818 12707 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.443228 12825 raft_consensus.cc:359] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.444149 12825 raft_consensus.cc:385] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.444522 12825 raft_consensus.cc:740] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.444640 12824 tablet_bootstrap.cc:492] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:27.445176 12824 ts_tablet_manager.cc:1403] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.004s
I20260709 13:17:27.445441 12825 consensus_queue.cc:260] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.446841 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:27.447784 12825 ts_tablet_manager.cc:1434] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
W20260709 13:17:27.448262 12597 leader_election.cc:343] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.448724 12825 tablet_bootstrap.cc:492] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:27.448318 12824 raft_consensus.cc:359] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.449164 12824 raft_consensus.cc:385] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.449465 12824 raft_consensus.cc:740] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.450402 12824 consensus_queue.cc:260] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.452876 12824 ts_tablet_manager.cc:1434] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:17:27.453850 12824 tablet_bootstrap.cc:492] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:27.453387 12707 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:17:27.455763 12825 tablet_bootstrap.cc:654] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.456329 12598 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:27.457648 12857 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:27.458182 12857 raft_consensus.cc:697] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:27.459137 12857 consensus_queue.cc:237] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.460311 12824 tablet_bootstrap.cc:654] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.467855 12823 ts_tablet_manager.cc:1434] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.098s	user 0.004s	sys 0.002s
I20260709 13:17:27.478147 12823 tablet_bootstrap.cc:492] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:27.471264 12825 tablet_bootstrap.cc:492] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:27.471873 12824 tablet_bootstrap.cc:492] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:27.481240 12824 ts_tablet_manager.cc:1403] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.005s
I20260709 13:17:27.473246 12630 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:27.475785 12857 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.476016 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:27.476639 12928 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:27.480731 12825 ts_tablet_manager.cc:1403] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.032s	user 0.019s	sys 0.003s
W20260709 13:17:27.483474 12672 leader_election.cc:343] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.484275 12857 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.484548 12824 raft_consensus.cc:359] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.485275 12824 raft_consensus.cc:385] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.499751 12824 raft_consensus.cc:740] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.485404 12825 raft_consensus.cc:359] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.500360 12825 raft_consensus.cc:385] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.500795 12825 raft_consensus.cc:740] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.500656 12824 consensus_queue.cc:260] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.496949 12532 catalog_manager.cc:5697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:27.501717 12825 consensus_queue.cc:260] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.504949 12824 ts_tablet_manager.cc:1434] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.023s	user 0.003s	sys 0.003s
I20260709 13:17:27.505998 12824 tablet_bootstrap.cc:492] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:27.497918 12823 tablet_bootstrap.cc:654] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.507910 12901 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:27.506031 12825 ts_tablet_manager.cc:1434] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.023s	user 0.011s	sys 0.000s
I20260709 13:17:27.510025 12825 tablet_bootstrap.cc:492] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:27.513922 12824 tablet_bootstrap.cc:654] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.491783 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:17:27.516166 12597 leader_election.cc:343] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.524652 12825 tablet_bootstrap.cc:654] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.525892 12928 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.526383 12928 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.531546 12824 tablet_bootstrap.cc:492] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:27.531559 12823 tablet_bootstrap.cc:492] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:27.532158 12824 ts_tablet_manager.cc:1403] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.004s
I20260709 13:17:27.532222 12823 ts_tablet_manager.cc:1403] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.054s	user 0.012s	sys 0.002s
I20260709 13:17:27.534718 12928 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:27.535558 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:27.535699 12824 raft_consensus.cc:359] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.536396 12824 raft_consensus.cc:385] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.536715 12824 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
W20260709 13:17:27.536916 12672 leader_election.cc:343] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.537420 12824 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.496701 12857 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:27.539090 12673 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:27.539528 12824 ts_tablet_manager.cc:1434] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.002s
I20260709 13:17:27.534729 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:27.540544 12824 tablet_bootstrap.cc:492] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
W20260709 13:17:27.540941 12673 leader_election.cc:343] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.541409 12673 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:27.541965 12928 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:27.542165 12927 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:27.542464 12928 raft_consensus.cc:697] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:27.487228 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:27.543292 12928 consensus_queue.cc:237] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.544216 12707 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:27.545073 12823 raft_consensus.cc:359] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.545782 12823 raft_consensus.cc:385] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.546082 12823 raft_consensus.cc:740] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.545461 12598 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:27.555229 12856 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:27.562606 12934 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:27.563221 12934 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.563663 12934 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.570204 12823 consensus_queue.cc:260] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.571491 12934 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.572448 12823 ts_tablet_manager.cc:1434] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.039s	user 0.002s	sys 0.003s
I20260709 13:17:27.573485 12823 tablet_bootstrap.cc:492] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:27.573796 12824 tablet_bootstrap.cc:654] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.574379 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:27.574940 12707 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.576835 12530 catalog_manager.cc:5697] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 0 to 1, leader changed from <none> to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:27.594487 12856 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.606376 12707 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:17:27.624527 12824 tablet_bootstrap.cc:492] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:27.625048 12824 ts_tablet_manager.cc:1403] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.085s	user 0.007s	sys 0.006s
I20260709 13:17:27.625123 12825 tablet_bootstrap.cc:492] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:27.625982 12825 ts_tablet_manager.cc:1403] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.116s	user 0.028s	sys 0.021s
I20260709 13:17:27.630607 12825 raft_consensus.cc:359] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.631536 12825 raft_consensus.cc:385] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.631929 12825 raft_consensus.cc:740] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.632829 12825 consensus_queue.cc:260] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.626514 12598 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:27.634258 12857 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:27.595036 12856 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.637042 12825 ts_tablet_manager.cc:1434] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.003s
I20260709 13:17:27.639955 12825 tablet_bootstrap.cc:492] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:27.634713 12857 raft_consensus.cc:697] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:27.641526 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:27.622584 12913 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.643891 12928 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:27.648325 12913 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.649269 12928 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.636294 12934 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:27.649837 12928 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.651098 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:27.651737 12632 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 0.
I20260709 13:17:27.652031 12913 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:17:27.645349 12597 leader_election.cc:343] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.653478 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:27.645610 12856 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:27.656723 12708 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 0.
I20260709 13:17:27.647680 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:27.652179 12928 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:27.653118 12747 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:17:27.644095 12857 consensus_queue.cc:237] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.672545 12825 tablet_bootstrap.cc:654] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.673130 12529 catalog_manager.cc:5697] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:27.647239 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:27.660859 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:27.661021 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:27.662009 12901 consensus_queue.cc:1048] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:27.679068 12927 consensus_queue.cc:1048] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:27.655822 12824 raft_consensus.cc:359] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.685653 12824 raft_consensus.cc:385] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.685971 12824 raft_consensus.cc:740] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.686120 12707 raft_consensus.cc:2393] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 1: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:17:27.686614 12672 leader_election.cc:343] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.686748 12824 consensus_queue.cc:260] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.688321 12747 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:27.689366 12856 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.689390 12824 ts_tablet_manager.cc:1434] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.064s	user 0.003s	sys 0.003s
I20260709 13:17:27.673454 12979 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:27.695025 12979 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.695439 12979 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.674157 12823 tablet_bootstrap.cc:654] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:27.698727 12673 leader_election.cc:343] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.699234 12673 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:27.700706 12901 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:27.704057 12824 tablet_bootstrap.cc:492] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:27.706202 12913 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:27.711090 12856 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:27.718971 12824 tablet_bootstrap.cc:654] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.718199 12901 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:17:27.716522 12979 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.729077 12825 tablet_bootstrap.cc:492] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:27.729809 12825 ts_tablet_manager.cc:1403] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.090s	user 0.012s	sys 0.002s
I20260709 13:17:27.733305 12979 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:27.741186 12825 raft_consensus.cc:359] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.741724 12915 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.741963 12825 raft_consensus.cc:385] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.742251 12825 raft_consensus.cc:740] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.743055 12825 consensus_queue.cc:260] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.745126 12825 ts_tablet_manager.cc:1434] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.000s
I20260709 13:17:27.742220 12915 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.761339 12856 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.761688 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:27.762109 12856 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:17:27.763563 12673 leader_election.cc:343] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.764813 12856 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:27.772755 12825 tablet_bootstrap.cc:492] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
W20260709 13:17:27.767645 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 62870.2us
I20260709 13:17:27.770896 12915 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:27.780694 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:27.781656 12782 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:27.785312 12857 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.792302 12857 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.805539 12672 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:17:27.816411 12915 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:27.816813 12915 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.817952 12915 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.819424 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:27.820256 12708 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:27.827136 12598 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:27.830821 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:27.831496 12629 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.831413 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:27.832350 12708 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.833101 12934 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:27.835526 12936 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.835881 12934 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.836102 12857 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:27.836266 12936 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.835500 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:27.842180 12936 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:17:27.842378 12597 leader_election.cc:343] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.843197 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:27.843870 12706 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:27.844444 12825 tablet_bootstrap.cc:654] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.844480 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:27.845091 12598 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:17:27.846894 12597 leader_election.cc:343] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.836302 12934 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.848165 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:27.849311 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:27.849668 12984 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.850011 12706 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 0.
I20260709 13:17:27.850158 12984 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.850864 12936 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:27.851455 12936 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.849299 12532 catalog_manager.cc:5697] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 0 to 1, leader changed from <none> to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:27.852285 12936 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:27.853744 12747 leader_election.cc:343] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.854981 12984 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:27.856622 12823 tablet_bootstrap.cc:492] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:27.859315 12823 ts_tablet_manager.cc:1403] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.286s	user 0.026s	sys 0.046s
I20260709 13:17:27.862035 12708 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:17:27.861937 12823 raft_consensus.cc:359] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.862632 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:27.865164 12824 tablet_bootstrap.cc:492] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:27.862672 12823 raft_consensus.cc:385] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.865697 12824 ts_tablet_manager.cc:1403] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.162s	user 0.012s	sys 0.006s
I20260709 13:17:27.865664 12823 raft_consensus.cc:740] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.869252 12824 raft_consensus.cc:359] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.871737 12824 raft_consensus.cc:385] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.872072 12824 raft_consensus.cc:740] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.873664 12824 consensus_queue.cc:260] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.876220 12823 consensus_queue.cc:260] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.879549 12823 ts_tablet_manager.cc:1434] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:17:27.880808 12934 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.888278 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:17:27.889756 12597 leader_election.cc:343] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.877447 12857 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.880831 12824 ts_tablet_manager.cc:1434] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:17:27.861402 12915 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.890488 12857 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.892009 12823 tablet_bootstrap.cc:492] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:27.893174 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:27.891297 12824 tablet_bootstrap.cc:492] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
W20260709 13:17:27.895669 12673 leader_election.cc:343] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.896301 12857 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:27.876986 12629 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:17:27.895864 12934 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:27.897524 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:27.900089 12915 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:27.901396 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:17:27.898111 12804 debug-util.cc:398] Leaking SignalData structure 0x7b08001c8280 after lost signal to thread 11520
I20260709 13:17:27.905901 12780 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:27.906740 12804 debug-util.cc:398] Leaking SignalData structure 0x7b0800099460 after lost signal to thread 12566
W20260709 13:17:27.908586 12804 debug-util.cc:398] Leaking SignalData structure 0x7b08001cc200 after lost signal to thread 12658
W20260709 13:17:27.909637 12804 debug-util.cc:398] Leaking SignalData structure 0x7b080017b200 after lost signal to thread 12733
W20260709 13:17:27.910667 12804 debug-util.cc:398] Leaking SignalData structure 0x7b08001ac3c0 after lost signal to thread 12807
I20260709 13:17:27.920506 12824 tablet_bootstrap.cc:654] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.902184 12936 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.895372 12747 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:17:27.923950 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:27.924481 12984 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:27.924566 12708 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
W20260709 13:17:27.924676 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 49368.9us
I20260709 13:17:27.922482 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:27.925503 12707 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.926047 12825 tablet_bootstrap.cc:492] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:27.926488 12825 ts_tablet_manager.cc:1403] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.154s	user 0.012s	sys 0.024s
I20260709 13:17:27.926843 12747 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:27.927685 12912 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:27.898223 12782 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:27.928135 12912 raft_consensus.cc:697] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:17:27.928967 12912 consensus_queue.cc:237] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.929207 12825 raft_consensus.cc:359] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.929826 12825 raft_consensus.cc:385] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.930090 12825 raft_consensus.cc:740] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.930701 12825 consensus_queue.cc:260] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.931792 12598 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:27.924892 12984 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.933426 12984 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.937080 12991 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:27.937670 12991 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.938102 12991 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.935027 12780 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:27.940016 12672 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
W20260709 13:17:27.940819 12747 leader_election.cc:343] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.945391 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:27.922676 12823 tablet_bootstrap.cc:654] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.941725 12915 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:27.942952 12825 ts_tablet_manager.cc:1434] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 13:17:27.943392 12707 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:17:27.944217 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:17:27.948261 12597 leader_election.cc:343] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.949386 12706 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.954290 12825 tablet_bootstrap.cc:492] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:27.944418 12936 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:27.946007 12984 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.941354 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:27.947834 12915 raft_consensus.cc:697] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:27.958285 12708 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 0.
I20260709 13:17:27.959061 12915 consensus_queue.cc:237] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.960718 12984 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:27.963587 12598 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:27.963795 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:27.964509 12708 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.964883 12936 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:27.965682 12936 raft_consensus.cc:697] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:17:27.967597 12579 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:27.967252 12936 consensus_queue.cc:237] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.968613 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:27.969233 12629 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.985486 12706 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:17:27.993357 12598 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:27.993080 12529 catalog_manager.cc:5697] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 0 to 1, leader changed from <none> to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:27.994925 12936 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:27.995249 12825 tablet_bootstrap.cc:654] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.995814 12936 raft_consensus.cc:697] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:27.998078 12936 consensus_queue.cc:237] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:27.999117 12629 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:17:27.999728 12530 catalog_manager.cc:5697] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:28.002545 12532 catalog_manager.cc:5697] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:28.007227 12823 tablet_bootstrap.cc:492] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:28.007758 12823 ts_tablet_manager.cc:1403] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.116s	user 0.006s	sys 0.011s
I20260709 13:17:27.972076 12747 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:17:28.011030 12708 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:17:28.013976 12912 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:28.014397 12912 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.014770 12912 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.003756 12991 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:17:28.020620 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 48637.6us
I20260709 13:17:28.023852 12824 tablet_bootstrap.cc:492] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:28.024395 12824 ts_tablet_manager.cc:1403] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.133s	user 0.013s	sys 0.025s
I20260709 13:17:28.024835 12936 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:28.027725 12823 raft_consensus.cc:359] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.027940 12824 raft_consensus.cc:359] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.028764 12824 raft_consensus.cc:385] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.029096 12824 raft_consensus.cc:740] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.029908 12824 consensus_queue.cc:260] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.030299 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:28.031090 12708 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.032636 12824 ts_tablet_manager.cc:1434] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:17:28.033533 12824 tablet_bootstrap.cc:492] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:28.028414 12823 raft_consensus.cc:385] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.034222 12823 raft_consensus.cc:740] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.035699 12991 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:28.024812 12747 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:17:28.037009 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:28.035471 12912 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.037688 12782 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.042594 12991 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.043308 12991 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.040688 12984 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:28.046041 12984 raft_consensus.cc:697] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:17:28.046888 12984 consensus_queue.cc:237] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.047873 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:17:28.050521 12598 leader_election.cc:343] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.053738 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:28.055460 12781 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:28.057230 12597 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:28.058490 12936 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:28.058981 12936 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.059417 12936 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.059665 12708 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:17:28.055022 12823 consensus_queue.cc:260] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.071406 12598 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:28.072348 12934 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:28.072530 12927 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.072865 12934 raft_consensus.cc:697] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:28.073134 12927 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.073768 12934 consensus_queue.cc:237] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.075841 12927 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:28.077230 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:17:28.078650 12672 leader_election.cc:343] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.079399 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:28.060827 12991 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:28.091092 12823 ts_tablet_manager.cc:1434] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.083s	user 0.007s	sys 0.003s
I20260709 13:17:28.097395 12912 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:28.097878 12825 tablet_bootstrap.cc:492] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:28.099063 12825 ts_tablet_manager.cc:1403] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.145s	user 0.024s	sys 0.018s
I20260709 13:17:28.104650 12823 tablet_bootstrap.cc:492] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:28.111353 12782 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:17:28.094795 12927 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.112893 12927 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.114356 12991 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:28.115057 12927 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:28.117054 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:17:28.117632 12673 leader_election.cc:343] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.118155 12673 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:28.123229 12825 raft_consensus.cc:359] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.125515 12825 raft_consensus.cc:385] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.125819 12825 raft_consensus.cc:740] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.126751 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:17:28.127552 12747 leader_election.cc:343] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:28.128262 12673 leader_election.cc:343] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.129745 12928 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:28.130879 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:28.129163 12825 consensus_queue.cc:260] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.131450 12708 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.124549 12936 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.126727 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:28.138203 12825 ts_tablet_manager.cc:1434] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 13:17:28.139027 12825 tablet_bootstrap.cc:492] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
W20260709 13:17:28.137233 12672 leader_election.cc:343] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.139948 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:28.141281 12928 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:28.135443 12530 catalog_manager.cc:5697] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:28.133339 12780 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:28.134285 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 53471.6us
I20260709 13:17:28.140081 12936 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:28.154727 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:28.156327 12824 tablet_bootstrap.cc:654] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:28.166112 12598 leader_election.cc:343] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.168965 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:28.170245 12781 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.176592 12708 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:17:28.178417 12747 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:17:28.179663 12856 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:28.174667 12915 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:17:28.180716 12856 raft_consensus.cc:697] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:17:28.181664 12856 consensus_queue.cc:237] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.188218 12936 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.187981 12915 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:28.188704 12936 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:17:28.207652 12597 consensus_peers.cc:597] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:28.212800 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:28.213513 12706 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:28.214986 12598 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:28.215175 12936 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:28.215910 12936 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:28.216251 12823 tablet_bootstrap.cc:654] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.216924 12936 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.223198 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:28.224634 12782 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:28.226584 12529 catalog_manager.cc:5697] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 0 to 1, leader changed from <none> to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:28.228492 12781 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:17:28.219169 12936 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.232010 12597 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:28.233891 13002 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:28.234355 13002 raft_consensus.cc:697] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:28.235148 13002 consensus_queue.cc:237] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.238446 12825 tablet_bootstrap.cc:654] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.235733 12934 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:17:28.240535 12530 catalog_manager.cc:5697] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:28.278771 12823 tablet_bootstrap.cc:492] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:28.279291 12823 ts_tablet_manager.cc:1403] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.175s	user 0.008s	sys 0.010s
I20260709 13:17:28.297223 12824 tablet_bootstrap.cc:492] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:28.297793 12824 ts_tablet_manager.cc:1403] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.264s	user 0.004s	sys 0.017s
I20260709 13:17:28.306875 12824 raft_consensus.cc:359] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.308027 12824 raft_consensus.cc:385] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.308410 12824 raft_consensus.cc:740] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.309928 12824 consensus_queue.cc:260] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.313100 12824 ts_tablet_manager.cc:1434] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.001s
I20260709 13:17:28.314313 12824 tablet_bootstrap.cc:492] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:28.314677 12936 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.317864 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:28.321405 12708 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.323509 12936 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:28.323710 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:28.324261 12781 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.330009 12708 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:17:28.334511 12598 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:28.330811 12781 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:17:28.335551 13002 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:28.336030 13002 raft_consensus.cc:697] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:28.336817 13002 consensus_queue.cc:237] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.340718 12824 tablet_bootstrap.cc:654] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.341533 12823 raft_consensus.cc:359] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.354864 12823 raft_consensus.cc:385] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.343972 12825 tablet_bootstrap.cc:492] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:28.355888 12825 ts_tablet_manager.cc:1403] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.217s	user 0.016s	sys 0.022s
I20260709 13:17:28.355196 12823 raft_consensus.cc:740] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.358275 12823 consensus_queue.cc:260] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.358858 12825 raft_consensus.cc:359] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.359510 12825 raft_consensus.cc:385] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.359771 12825 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.360510 12823 ts_tablet_manager.cc:1434] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.081s	user 0.002s	sys 0.003s
I20260709 13:17:28.360443 12825 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.359871 12529 catalog_manager.cc:5697] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 0 to 1, leader changed from <none> to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:28.361828 12823 tablet_bootstrap.cc:492] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:28.363329 12825 ts_tablet_manager.cc:1434] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:17:28.365372 12825 tablet_bootstrap.cc:492] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:28.367959 12823 tablet_bootstrap.cc:654] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.371688 12825 tablet_bootstrap.cc:654] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.373731 12824 tablet_bootstrap.cc:492] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:28.374337 12824 ts_tablet_manager.cc:1403] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.060s	user 0.013s	sys 0.002s
I20260709 13:17:28.377835 12824 raft_consensus.cc:359] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.378562 12824 raft_consensus.cc:385] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.378890 12824 raft_consensus.cc:740] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.380530 12824 consensus_queue.cc:260] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.382556 12824 ts_tablet_manager.cc:1434] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:17:28.386235 12823 tablet_bootstrap.cc:492] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:28.392608 12984 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:28.394870 12824 tablet_bootstrap.cc:492] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:28.388921 12529 catalog_manager.cc:5697] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 0 to 1, leader changed from <none> to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:28.397136 12825 tablet_bootstrap.cc:492] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:28.397707 12825 ts_tablet_manager.cc:1403] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.014s
I20260709 13:17:28.399828 12823 ts_tablet_manager.cc:1403] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.039s	user 0.012s	sys 0.009s
I20260709 13:17:28.400779 12825 raft_consensus.cc:359] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.401503 12824 tablet_bootstrap.cc:654] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.401623 12825 raft_consensus.cc:385] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.402117 12825 raft_consensus.cc:740] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.402971 12825 consensus_queue.cc:260] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.406347 12823 raft_consensus.cc:359] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.410085 12825 ts_tablet_manager.cc:1434] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:17:28.418521 12825 tablet_bootstrap.cc:492] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:28.419499 12824 tablet_bootstrap.cc:492] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:28.419936 12824 ts_tablet_manager.cc:1403] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.025s	user 0.010s	sys 0.000s
I20260709 13:17:28.422511 12824 raft_consensus.cc:359] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.423125 12824 raft_consensus.cc:385] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.423379 12824 raft_consensus.cc:740] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.425135 12825 tablet_bootstrap.cc:654] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.425729 12823 raft_consensus.cc:385] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.428675 12823 raft_consensus.cc:740] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.428577 12856 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:28.423970 12824 consensus_queue.cc:260] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.433887 12824 ts_tablet_manager.cc:1434] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 13:17:28.434844 12824 tablet_bootstrap.cc:492] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:28.435881 12532 catalog_manager.cc:5697] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 0 to 1, leader changed from <none> to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:28.436782 12823 consensus_queue.cc:260] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.441668 12823 ts_tablet_manager.cc:1434] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.001s
I20260709 13:17:28.443523 12823 tablet_bootstrap.cc:492] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:28.447181 12781 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.447504 12928 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.448016 12928 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.450433 12928 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:28.452446 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:28.453576 12824 tablet_bootstrap.cc:654] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:28.458484 12597 consensus_peers.cc:597] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:28.466281 12934 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:28.466471 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:28.468838 12912 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.469345 12912 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.459558 13002 consensus_queue.cc:1048] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:17:28.471541 12912 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:17:28.466375 12673 leader_election.cc:343] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.473387 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:17:28.482981 12747 leader_election.cc:343] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.484905 12632 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:28.487637 12672 leader_election.cc:343] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.488095 12672 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:28.491530 13012 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:28.497668 12934 consensus_queue.cc:1048] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:28.501236 13012 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:28.496220 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:28.508824 12825 tablet_bootstrap.cc:492] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:28.508737 12936 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:28.509374 12825 ts_tablet_manager.cc:1403] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.091s	user 0.011s	sys 0.005s
W20260709 13:17:28.508677 12747 leader_election.cc:343] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.510215 12747 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:28.510813 13012 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.511466 12912 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:28.511317 13012 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.512159 12825 raft_consensus.cc:359] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.512835 12825 raft_consensus.cc:385] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.513131 12825 raft_consensus.cc:740] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.513811 12825 consensus_queue.cc:260] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.508662 12824 tablet_bootstrap.cc:492] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:28.518095 12825 ts_tablet_manager.cc:1434] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:17:28.519066 12825 tablet_bootstrap.cc:492] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:28.516531 12824 ts_tablet_manager.cc:1403] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.082s	user 0.009s	sys 0.010s
I20260709 13:17:28.514392 13012 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:28.527889 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:28.529438 13012 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:28.521979 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:17:28.534670 12672 leader_election.cc:343] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:28.541716 12673 leader_election.cc:343] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.542303 12673 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:28.556285 13012 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:28.547233 12936 consensus_queue.cc:1048] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:17:28.550691 12934 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.560672 12934 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.554561 13002 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.563827 13002 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.565168 12934 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:28.565775 13002 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:28.547931 12856 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:17:28.566398 12912 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.566902 12912 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.568984 12912 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:28.574417 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:28.576514 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:28.577078 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:28.577441 12707 raft_consensus.cc:2393] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 1: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:17:28.578584 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:17:28.578626 12747 leader_election.cc:343] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.579773 12708 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:28.581105 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:28.582268 12598 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:28.582562 12597 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:28.583869 12936 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.584335 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:28.586324 12934 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:17:28.589447 12597 leader_election.cc:343] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.590199 12934 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.590618 12934 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.602293 12823 tablet_bootstrap.cc:654] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:28.609481 12747 leader_election.cc:343] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.610025 12747 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:28.611532 12856 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:28.612332 12825 tablet_bootstrap.cc:654] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.620289 12856 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:28.621389 12824 raft_consensus.cc:359] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.623154 12824 raft_consensus.cc:385] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.623594 12824 raft_consensus.cc:740] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.624459 12824 consensus_queue.cc:260] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.628744 12824 ts_tablet_manager.cc:1434] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.108s	user 0.004s	sys 0.004s
I20260709 13:17:28.629462 12934 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.630010 12824 tablet_bootstrap.cc:492] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:28.636081 12934 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:28.628870 12936 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:28.638818 12934 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.650040 12934 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.655282 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:28.661783 12934 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:28.665468 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:17:28.666955 12598 leader_election.cc:343] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:28.668661 12597 leader_election.cc:343] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.664240 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:28.672197 12823 tablet_bootstrap.cc:492] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:28.672621 12823 ts_tablet_manager.cc:1403] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.229s	user 0.016s	sys 0.013s
I20260709 13:17:28.674407 12709 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.677408 12824 tablet_bootstrap.cc:654] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.682554 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:28.683233 12781 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:28.685809 12825 tablet_bootstrap.cc:492] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:28.686259 12825 ts_tablet_manager.cc:1403] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.167s	user 0.011s	sys 0.005s
I20260709 13:17:28.686594 12597 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:28.693010 12825 raft_consensus.cc:359] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.693962 12825 raft_consensus.cc:385] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.694296 12825 raft_consensus.cc:740] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.695748 12825 consensus_queue.cc:260] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.702409 12823 raft_consensus.cc:359] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.703262 12823 raft_consensus.cc:385] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.703588 12823 raft_consensus.cc:740] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.706149 12936 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:28.706594 12936 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.707000 12936 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.707612 12823 consensus_queue.cc:260] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.710042 12823 ts_tablet_manager.cc:1434] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.001s
I20260709 13:17:28.711014 12823 tablet_bootstrap.cc:492] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:28.706930 12709 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:17:28.713869 12598 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:28.715080 12934 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:28.715590 12934 raft_consensus.cc:697] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:28.715947 12825 ts_tablet_manager.cc:1434] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:17:28.717151 12825 tablet_bootstrap.cc:492] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:28.716488 12934 consensus_queue.cc:237] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.734313 12936 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.746179 12824 tablet_bootstrap.cc:492] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:28.746879 12824 ts_tablet_manager.cc:1403] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.117s	user 0.023s	sys 0.021s
I20260709 13:17:28.751974 12824 raft_consensus.cc:359] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.752647 12824 raft_consensus.cc:385] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.752944 12824 raft_consensus.cc:740] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.753594 12824 consensus_queue.cc:260] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:17:28.757911 12598 consensus_peers.cc:597] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:28.759017 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:28.759665 12781 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.760028 12824 ts_tablet_manager.cc:1434] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:17:28.759579 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:28.761740 12824 tablet_bootstrap.cc:492] T 58239dd301f64758846dd758e449e4d8 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
W20260709 13:17:28.762212 12598 leader_election.cc:343] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.761147 12936 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:28.766592 12825 tablet_bootstrap.cc:654] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.763688 12823 tablet_bootstrap.cc:654] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.770404 12912 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:28.776501 12530 catalog_manager.cc:5697] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 0 to 1, leader changed from <none> to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:28.789026 12824 tablet_bootstrap.cc:654] T 58239dd301f64758846dd758e449e4d8 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.799773 12934 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:28.795743 12936 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.800899 13035 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:28.801829 12936 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.803895 12936 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:28.805811 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:28.806576 12709 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:28.807809 12598 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:28.810783 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:17:28.812249 12597 leader_election.cc:343] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.809775 12936 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:28.815876 12936 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.816227 12936 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.828096 12781 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:17:28.837982 12936 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.853178 12597 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:28.878715 13002 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:28.879375 13002 raft_consensus.cc:697] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:28.880460 13002 consensus_queue.cc:237] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.927944 13012 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.928718 13012 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.940124 13012 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:28.960040 12824 tablet_bootstrap.cc:492] T 58239dd301f64758846dd758e449e4d8 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:28.960595 12824 ts_tablet_manager.cc:1403] T 58239dd301f64758846dd758e449e4d8 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.199s	user 0.015s	sys 0.008s
I20260709 13:17:28.962076 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:28.962885 13012 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:28.968732 12672 leader_election.cc:343] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.972321 12825 tablet_bootstrap.cc:492] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:28.972950 12825 ts_tablet_manager.cc:1403] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.256s	user 0.011s	sys 0.015s
I20260709 13:17:28.981580 12823 tablet_bootstrap.cc:492] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:28.982277 12823 ts_tablet_manager.cc:1403] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.271s	user 0.027s	sys 0.038s
W20260709 13:17:28.959685 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 106659us
I20260709 13:17:28.987015 13002 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:28.963552 13012 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.990584 13012 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:28.993872 12824 raft_consensus.cc:359] T 58239dd301f64758846dd758e449e4d8 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:28.994853 12824 raft_consensus.cc:385] T 58239dd301f64758846dd758e449e4d8 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.995193 12824 raft_consensus.cc:740] T 58239dd301f64758846dd758e449e4d8 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.995925 12824 consensus_queue.cc:260] T 58239dd301f64758846dd758e449e4d8 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.010650 12936 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:29.018556 12823 raft_consensus.cc:359] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.019627 12823 raft_consensus.cc:385] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.020051 12823 raft_consensus.cc:740] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.021549 12823 consensus_queue.cc:260] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.023594 12825 raft_consensus.cc:359] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.024544 12825 raft_consensus.cc:385] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.024915 12825 raft_consensus.cc:740] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.030301 13035 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.030802 13035 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.032799 13035 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:29.034632 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:29.035149 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:29.035887 12707 raft_consensus.cc:2393] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 1: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:17:29.035913 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:29.036567 12708 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.036751 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:29.037395 12780 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:29.028141 12825 consensus_queue.cc:260] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.038913 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:29.039623 13012 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:29.039947 13012 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.040263 13012 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.040864 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:29.041425 12825 ts_tablet_manager.cc:1434] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.068s	user 0.005s	sys 0.005s
I20260709 13:17:29.042724 12825 tablet_bootstrap.cc:492] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:29.047719 12747 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:29.049546 13035 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.050001 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:17:29.050911 12673 leader_election.cc:343] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:29.051466 12673 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:29.053390 12922 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:29.050778 12630 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:29.056264 12824 ts_tablet_manager.cc:1434] T 58239dd301f64758846dd758e449e4d8 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.095s	user 0.007s	sys 0.002s
I20260709 13:17:29.057529 12824 tablet_bootstrap.cc:492] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:29.064427 12823 ts_tablet_manager.cc:1434] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.082s	user 0.010s	sys 0.000s
I20260709 13:17:29.065773 12823 tablet_bootstrap.cc:492] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:29.077813 12922 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.078634 12922 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.105938 12708 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:17:29.107185 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:17:29.113901 12597 leader_election.cc:343] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:29.115212 12857 consensus_queue.cc:1048] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:29.117180 13035 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:29.131181 12824 tablet_bootstrap.cc:654] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.133536 12823 tablet_bootstrap.cc:654] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.121438 13012 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.125921 13045 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.135249 13045 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.088333 12530 catalog_manager.cc:5697] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 0 to 1, leader changed from <none> to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.138113 12922 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:29.144373 13045 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:29.145937 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:29.146960 13012 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:17:29.147850 12672 leader_election.cc:343] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:29.148649 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:29.149209 12780 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.150832 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:17:29.152611 12673 leader_election.cc:343] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:29.152767 12579 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:29.153107 12673 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:29.154434 12598 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:29.155279 12915 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:29.158787 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:29.159497 12632 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.153765 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:29.170423 12857 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:29.173162 12857 raft_consensus.cc:697] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:29.174664 12780 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:29.174450 12857 consensus_queue.cc:237] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.176930 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:29.188846 12824 tablet_bootstrap.cc:492] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:29.189414 12824 ts_tablet_manager.cc:1403] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.132s	user 0.000s	sys 0.014s
I20260709 13:17:29.196401 12824 raft_consensus.cc:359] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.197177 12824 raft_consensus.cc:385] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.197465 12824 raft_consensus.cc:740] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.199944 12824 consensus_queue.cc:260] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.186271 13012 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:29.201884 13012 raft_consensus.cc:697] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:29.202803 13012 consensus_queue.cc:237] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.207337 12632 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:29.211220 12825 tablet_bootstrap.cc:654] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.215016 12824 ts_tablet_manager.cc:1434] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.002s
I20260709 13:17:29.215997 12824 tablet_bootstrap.cc:492] T ab5ddded6fd34f278dade99f2086c4e6 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:29.223219 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:29.223767 12823 tablet_bootstrap.cc:492] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:29.224185 12823 ts_tablet_manager.cc:1403] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.159s	user 0.026s	sys 0.009s
I20260709 13:17:29.241641 12825 tablet_bootstrap.cc:492] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:29.242102 12825 ts_tablet_manager.cc:1403] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.200s	user 0.015s	sys 0.024s
I20260709 13:17:29.240806 12532 catalog_manager.cc:5697] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 0 to 1, leader changed from <none> to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.244621 12825 raft_consensus.cc:359] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.245218 12825 raft_consensus.cc:385] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.246748 12823 raft_consensus.cc:359] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.247433 12823 raft_consensus.cc:385] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.247704 12823 raft_consensus.cc:740] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.253270 12529 catalog_manager.cc:5697] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.255951 12824 tablet_bootstrap.cc:654] T ab5ddded6fd34f278dade99f2086c4e6 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.257014 12747 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:29.258783 12823 consensus_queue.cc:260] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.259208 13035 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:29.260883 12823 ts_tablet_manager.cc:1434] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.001s
I20260709 13:17:29.245484 12825 raft_consensus.cc:740] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.269352 12857 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:29.270533 12825 consensus_queue.cc:260] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.272305 12823 tablet_bootstrap.cc:492] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:29.273595 12824 tablet_bootstrap.cc:492] T ab5ddded6fd34f278dade99f2086c4e6 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:29.274104 12824 ts_tablet_manager.cc:1403] T ab5ddded6fd34f278dade99f2086c4e6 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.058s	user 0.006s	sys 0.004s
I20260709 13:17:29.277019 12824 raft_consensus.cc:359] T ab5ddded6fd34f278dade99f2086c4e6 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.277766 12824 raft_consensus.cc:385] T ab5ddded6fd34f278dade99f2086c4e6 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.278075 12824 raft_consensus.cc:740] T ab5ddded6fd34f278dade99f2086c4e6 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.278302 12823 tablet_bootstrap.cc:654] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.282418 12825 ts_tablet_manager.cc:1434] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.000s
I20260709 13:17:29.282150 12824 consensus_queue.cc:260] T ab5ddded6fd34f278dade99f2086c4e6 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.283492 12825 tablet_bootstrap.cc:492] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:29.285647 12824 ts_tablet_manager.cc:1434] T ab5ddded6fd34f278dade99f2086c4e6 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:17:29.286561 12857 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:29.301715 12824 tablet_bootstrap.cc:492] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:29.302080 12825 tablet_bootstrap.cc:654] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.303011 12779 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.309437 12823 tablet_bootstrap.cc:492] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:29.309945 12823 ts_tablet_manager.cc:1403] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.038s	user 0.011s	sys 0.001s
I20260709 13:17:29.314568 12825 tablet_bootstrap.cc:492] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:29.315091 12825 ts_tablet_manager.cc:1403] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.032s	user 0.007s	sys 0.004s
I20260709 13:17:29.316782 12824 tablet_bootstrap.cc:654] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.317701 12825 raft_consensus.cc:359] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.318416 12825 raft_consensus.cc:385] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.318684 12825 raft_consensus.cc:740] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.319037 12823 raft_consensus.cc:359] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.319468 12825 consensus_queue.cc:260] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.341166 12824 tablet_bootstrap.cc:492] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:29.347461 12936 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:17:29.354115 12984 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.354583 12984 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.359462 12984 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:29.361789 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:29.365880 12824 ts_tablet_manager.cc:1403] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.064s	user 0.006s	sys 0.007s
I20260709 13:17:29.368184 12825 ts_tablet_manager.cc:1434] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.000s
I20260709 13:17:29.368580 12824 raft_consensus.cc:359] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.369168 12825 tablet_bootstrap.cc:492] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:29.369266 12824 raft_consensus.cc:385] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.372628 12824 raft_consensus.cc:740] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.346926 12823 raft_consensus.cc:385] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.373420 12823 raft_consensus.cc:740] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.374423 12823 consensus_queue.cc:260] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.375916 12824 consensus_queue.cc:260] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.376679 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:29.379374 12823 ts_tablet_manager.cc:1434] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.069s	user 0.007s	sys 0.000s
I20260709 13:17:29.380982 12823 tablet_bootstrap.cc:492] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:29.388801 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:29.390971 12824 ts_tablet_manager.cc:1434] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 13:17:29.391867 12824 tablet_bootstrap.cc:492] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:29.391052 12984 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:29.395517 12823 tablet_bootstrap.cc:654] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.411628 13012 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.412189 13012 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.414295 13012 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:29.417038 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:29.418380 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:29.420706 12672 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:29.421839 13012 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:29.423144 12825 tablet_bootstrap.cc:654] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.424132 12706 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.437116 12824 tablet_bootstrap.cc:654] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.443810 12984 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.444362 12984 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.447255 12984 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:29.448813 13012 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.449293 13012 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.451470 13012 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:29.452194 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:29.452819 12708 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 0.
I20260709 13:17:29.455295 12823 tablet_bootstrap.cc:492] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:29.441649 12857 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:17:29.455574 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:29.455981 12823 ts_tablet_manager.cc:1403] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.076s	user 0.012s	sys 0.011s
W20260709 13:17:29.457432 12672 leader_election.cc:343] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:29.458106 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:29.459949 12934 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.460153 12747 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:29.460592 12934 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.461670 12630 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 0.
I20260709 13:17:29.466032 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:29.466509 12984 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:29.467667 12984 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.468007 12632 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:29.468706 12984 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.472819 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:29.473309 12824 tablet_bootstrap.cc:492] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:29.469946 12934 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:29.470724 12673 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:29.471089 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:29.474400 13012 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:29.473464 12706 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:29.473811 12824 ts_tablet_manager.cc:1403] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.082s	user 0.019s	sys 0.005s
I20260709 13:17:29.477017 13012 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.478341 13012 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:29.478235 12597 leader_election.cc:343] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:29.479327 12823 raft_consensus.cc:359] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.480114 12823 raft_consensus.cc:385] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.480396 12823 raft_consensus.cc:740] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.481788 12598 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:29.481990 12825 tablet_bootstrap.cc:492] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:29.482874 12825 ts_tablet_manager.cc:1403] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.114s	user 0.016s	sys 0.006s
I20260709 13:17:29.481029 12823 consensus_queue.cc:260] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.484263 12934 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:29.485035 12934 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.485373 12934 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.487689 12922 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.488134 12922 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.490262 12922 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:29.491705 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:29.492154 12934 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.487008 13012 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.493372 12825 raft_consensus.cc:359] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.494398 12825 raft_consensus.cc:385] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.494731 12825 raft_consensus.cc:740] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.495205 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:29.495420 12825 consensus_queue.cc:260] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.496740 12673 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:29.497437 12922 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:29.498134 12824 raft_consensus.cc:359] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.501006 12823 ts_tablet_manager.cc:1434] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.045s	user 0.009s	sys 0.000s
I20260709 13:17:29.501199 12824 raft_consensus.cc:385] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.500115 13012 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:29.501523 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:29.502476 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:29.513821 12824 raft_consensus.cc:740] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.514199 12632 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.506253 12934 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:29.514609 12824 consensus_queue.cc:260] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.516727 12824 ts_tablet_manager.cc:1434] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.039s	user 0.004s	sys 0.000s
I20260709 13:17:29.517653 12824 tablet_bootstrap.cc:492] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:29.522333 12823 tablet_bootstrap.cc:492] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
W20260709 13:17:29.522603 12672 leader_election.cc:343] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:29.506615 12825 ts_tablet_manager.cc:1434] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.002s
I20260709 13:17:29.508194 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:29.529407 12825 tablet_bootstrap.cc:492] T 1b13b87a5cea4263937041d123c5889d P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
W20260709 13:17:29.529798 12597 leader_election.cc:343] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:29.509212 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:29.530601 12708 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.509044 12984 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.532437 12824 tablet_bootstrap.cc:654] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.532902 13035 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.540908 13035 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.543093 13035 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:29.528110 12632 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:29.534919 12984 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:29.536115 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:29.544281 12630 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.537628 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:29.539217 12823 tablet_bootstrap.cc:654] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.544307 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:29.539448 12708 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:17:29.545959 12706 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.546468 12709 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 1: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:17:29.540400 12825 tablet_bootstrap.cc:654] T 1b13b87a5cea4263937041d123c5889d P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.560559 12598 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:29.563822 12934 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:29.564561 12934 raft_consensus.cc:697] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:29.569803 12934 consensus_queue.cc:237] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.572546 12673 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:29.573549 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:29.574352 13012 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:29.578766 12747 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:29.581435 13035 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.575924 13012 raft_consensus.cc:697] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:29.573407 12630 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:17:29.610061 12530 catalog_manager.cc:5697] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 0 to 1, leader changed from <none> to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.591216 12706 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:17:29.592718 13012 consensus_queue.cc:237] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.620846 12747 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:29.623929 12984 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:29.633208 12779 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.624439 12984 raft_consensus.cc:697] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:17:29.636988 13035 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:29.639250 12984 consensus_queue.cc:237] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.646215 12823 tablet_bootstrap.cc:492] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:29.646731 12823 ts_tablet_manager.cc:1403] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.125s	user 0.013s	sys 0.006s
I20260709 13:17:29.647780 12824 tablet_bootstrap.cc:492] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:29.648176 12824 ts_tablet_manager.cc:1403] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.131s	user 0.046s	sys 0.028s
I20260709 13:17:29.649758 12823 raft_consensus.cc:359] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.650477 12823 raft_consensus.cc:385] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.650188 12824 raft_consensus.cc:359] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.650869 12823 raft_consensus.cc:740] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.651122 12824 raft_consensus.cc:385] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.651473 12824 raft_consensus.cc:740] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.651643 12823 consensus_queue.cc:260] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.651964 12824 consensus_queue.cc:260] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.654749 12823 ts_tablet_manager.cc:1434] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:17:29.654918 12825 tablet_bootstrap.cc:492] T 1b13b87a5cea4263937041d123c5889d P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:29.655440 12825 ts_tablet_manager.cc:1403] T 1b13b87a5cea4263937041d123c5889d P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.126s	user 0.015s	sys 0.002s
I20260709 13:17:29.657893 12825 raft_consensus.cc:359] T 1b13b87a5cea4263937041d123c5889d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.658978 12824 ts_tablet_manager.cc:1434] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:17:29.659041 12825 raft_consensus.cc:385] T 1b13b87a5cea4263937041d123c5889d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.659439 12825 raft_consensus.cc:740] T 1b13b87a5cea4263937041d123c5889d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.660205 12824 tablet_bootstrap.cc:492] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:29.660084 12825 consensus_queue.cc:260] T 1b13b87a5cea4263937041d123c5889d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.662179 12825 ts_tablet_manager.cc:1434] T 1b13b87a5cea4263937041d123c5889d P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:17:29.663297 12825 tablet_bootstrap.cc:492] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:29.715348 12529 catalog_manager.cc:5697] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:29.721776 12530 catalog_manager.cc:5697] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.738394 12922 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.739563 12922 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.742815 12922 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:29.748472 12936 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:29.750797 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:29.764544 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:29.751885 12780 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:29.765691 12632 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:29.779748 12934 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.780359 12934 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.785805 12934 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:29.778842 12672 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:29.788262 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:29.790941 12857 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:17:29.792050 13074 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:17:29.791818 12597 leader_election.cc:343] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:29.792596 13074 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.802451 13074 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.808640 12824 tablet_bootstrap.cc:654] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.814334 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:29.818513 12709 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:29.823393 12825 tablet_bootstrap.cc:654] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.826581 12915 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:29.827554 12598 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:29.830842 12915 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:29.829599 12936 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:29.831913 12936 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.832367 12936 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.856822 13074 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.878937 12823 tablet_bootstrap.cc:492] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:29.882265 13074 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:29.883786 12936 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.893414 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:17:29.895701 12597 leader_election.cc:343] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:29.902304 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:29.903803 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:29.922909 12709 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.930023 12780 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.903715 12936 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:29.955484 12709 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:17:29.968536 12823 tablet_bootstrap.cc:654] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.968839 12824 tablet_bootstrap.cc:492] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:29.961539 12780 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:29.971036 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:29.973145 12825 tablet_bootstrap.cc:492] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:29.972412 12824 ts_tablet_manager.cc:1403] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.312s	user 0.010s	sys 0.015s
I20260709 13:17:29.973884 12630 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.982321 12709 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.987017 12672 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:29.989377 13012 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:29.989969 13012 raft_consensus.cc:697] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:29.990974 13012 consensus_queue.cc:237] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:29.996078 12630 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:30.000592 12922 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.001087 12922 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.002801 12824 raft_consensus.cc:359] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.003244 12922 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:30.003566 12824 raft_consensus.cc:385] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:30.003909 12824 raft_consensus.cc:740] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.005194 12825 ts_tablet_manager.cc:1403] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.342s	user 0.014s	sys 0.008s
I20260709 13:17:30.005287 12824 consensus_queue.cc:260] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.010907 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:30.011966 12922 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.012732 12922 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:17:30.013535 12673 leader_election.cc:343] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:30.014590 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:30.010953 12825 raft_consensus.cc:359] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.022290 12824 ts_tablet_manager.cc:1434] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.000s
I20260709 13:17:30.021095 12825 raft_consensus.cc:385] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:30.023695 12825 raft_consensus.cc:740] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.024425 12825 consensus_queue.cc:260] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.026695 12922 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:30.029016 12824 tablet_bootstrap.cc:492] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:30.029927 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:17:30.030457 12672 leader_election.cc:343] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:30.030902 12672 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:17:30.031975 12672 leader_election.cc:343] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:30.032804 12922 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:30.015012 12598 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:30.036098 12936 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:30.036607 12936 raft_consensus.cc:697] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:30.037501 12936 consensus_queue.cc:237] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.049134 12825 ts_tablet_manager.cc:1434] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 13:17:30.054742 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:30.055464 12630 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:30.063750 12673 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:30.068094 12825 tablet_bootstrap.cc:492] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
W20260709 13:17:30.068411 12597 consensus_peers.cc:597] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:30.068819 12922 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:30.070492 12922 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:30.069275 12532 catalog_manager.cc:5697] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:30.070847 12922 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.074153 12823 tablet_bootstrap.cc:492] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:30.074697 12823 ts_tablet_manager.cc:1403] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.196s	user 0.026s	sys 0.017s
I20260709 13:17:30.082273 12823 raft_consensus.cc:359] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.084134 12823 raft_consensus.cc:385] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:30.084414 12823 raft_consensus.cc:740] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.085134 12823 consensus_queue.cc:260] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.087405 12824 tablet_bootstrap.cc:654] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:30.089330 12529 catalog_manager.cc:5697] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 0 to 1, leader changed from <none> to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:30.097219 12934 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:30.102677 12823 ts_tablet_manager.cc:1434] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:17:30.106468 12823 tablet_bootstrap.cc:492] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:30.112790 12824 tablet_bootstrap.cc:492] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:30.113366 12824 ts_tablet_manager.cc:1403] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.085s	user 0.017s	sys 0.005s
I20260709 13:17:30.115224 12825 tablet_bootstrap.cc:654] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:30.115577 12922 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.118395 12823 tablet_bootstrap.cc:654] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:30.119462 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:30.119740 12922 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:17:30.122102 12672 leader_election.cc:343] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:30.122605 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:30.123240 12628 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.131484 12628 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:30.132686 12673 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:30.133715 12922 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:30.133754 13076 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.133643 12824 raft_consensus.cc:359] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.134420 13076 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.134976 12824 raft_consensus.cc:385] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:30.135257 12824 raft_consensus.cc:740] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.135902 12824 consensus_queue.cc:260] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.136929 13076 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:30.138262 12824 ts_tablet_manager.cc:1434] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.025s	user 0.003s	sys 0.003s
I20260709 13:17:30.138331 12922 raft_consensus.cc:697] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:30.139536 12824 tablet_bootstrap.cc:492] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:30.139448 12922 consensus_queue.cc:237] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.140867 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:17:30.141700 12672 consensus_peers.cc:597] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:30.141734 12705 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 0.
I20260709 13:17:30.143473 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:30.144232 13076 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:30.144593 13076 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:30.144923 13076 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.147943 12824 tablet_bootstrap.cc:654] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:30.157012 12823 tablet_bootstrap.cc:492] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:30.157598 12823 ts_tablet_manager.cc:1403] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.051s	user 0.008s	sys 0.002s
I20260709 13:17:30.174286 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:30.175076 12628 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 0.
I20260709 13:17:30.179209 12915 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:30.181066 12984 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:30.187445 12984 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:30.188681 12823 raft_consensus.cc:359] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.193903 13076 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.199620 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:30.200387 12632 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.207216 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:30.207862 12705 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.211823 12530 catalog_manager.cc:5697] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:30.213663 12632 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:17:30.217620 12824 tablet_bootstrap.cc:492] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:30.218091 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:17:30.189407 12823 raft_consensus.cc:385] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:30.218868 12984 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:30.221125 12823 raft_consensus.cc:740] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.218118 12824 ts_tablet_manager.cc:1403] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.079s	user 0.015s	sys 0.009s
I20260709 13:17:30.221303 12984 raft_consensus.cc:697] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:17:30.222761 12984 consensus_queue.cc:237] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.223174 12823 consensus_queue.cc:260] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.226444 12824 raft_consensus.cc:359] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.227210 12824 raft_consensus.cc:385] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:30.227567 12824 raft_consensus.cc:740] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.228294 12824 consensus_queue.cc:260] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.230654 12824 ts_tablet_manager.cc:1434] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:17:30.231665 12824 tablet_bootstrap.cc:492] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:30.236588 12915 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.237109 12915 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.237990 12824 tablet_bootstrap.cc:654] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:30.239791 12915 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:30.241551 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:30.242494 12632 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:30.242352 13076 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:30.243808 12673 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:30.245042 12922 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:30.245431 12922 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:30.245862 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:30.246204 12922 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.225488 12705 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
W20260709 13:17:30.253957 12672 leader_election.cc:343] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:30.254571 12824 tablet_bootstrap.cc:492] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:30.255017 12824 ts_tablet_manager.cc:1403] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.024s	user 0.016s	sys 0.000s
I20260709 13:17:30.257550 12824 raft_consensus.cc:359] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.258234 12824 raft_consensus.cc:385] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:30.258553 12824 raft_consensus.cc:740] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.259322 12824 consensus_queue.cc:260] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.261418 12824 ts_tablet_manager.cc:1434] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:17:30.262521 12824 tablet_bootstrap.cc:492] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:30.264469 12825 tablet_bootstrap.cc:492] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:30.264976 12825 ts_tablet_manager.cc:1403] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.199s	user 0.008s	sys 0.015s
I20260709 13:17:30.270423 12825 raft_consensus.cc:359] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.271924 12824 tablet_bootstrap.cc:654] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:30.271163 12825 raft_consensus.cc:385] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:30.273809 12825 raft_consensus.cc:740] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.274814 12825 consensus_queue.cc:260] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.275434 12922 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.277340 12825 ts_tablet_manager.cc:1434] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 13:17:30.278455 12825 tablet_bootstrap.cc:492] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:30.278411 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:30.279153 12628 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.227659 12823 ts_tablet_manager.cc:1434] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.070s	user 0.006s	sys 0.000s
I20260709 13:17:30.286656 12823 tablet_bootstrap.cc:492] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:30.289786 12825 tablet_bootstrap.cc:654] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:30.308060 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:17:30.309499 12672 leader_election.cc:343] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:30.313192 12922 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:30.320104 12824 tablet_bootstrap.cc:492] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:30.317272 12530 catalog_manager.cc:5697] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:30.320577 12824 ts_tablet_manager.cc:1403] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.058s	user 0.024s	sys 0.027s
I20260709 13:17:30.326599 12628 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:30.327682 12823 tablet_bootstrap.cc:654] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:30.342047 13002 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:30.342767 12579 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:30.342548 13002 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.343569 12824 raft_consensus.cc:359] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.344031 12824 raft_consensus.cc:385] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:30.344183 12824 raft_consensus.cc:740] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.344589 12824 consensus_queue.cc:260] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.345162 13002 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:30.345991 12673 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:30.351478 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:30.354722 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:30.356125 12705 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:30.346392 12824 ts_tablet_manager.cc:1434] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 13:17:30.357002 12922 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:30.357280 12824 tablet_bootstrap.cc:492] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:30.357527 12922 raft_consensus.cc:697] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:30.360617 12922 consensus_queue.cc:237] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.363399 12824 tablet_bootstrap.cc:654] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:30.366922 12597 leader_election.cc:343] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:30.369927 12598 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:30.372747 12824 tablet_bootstrap.cc:492] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:30.373200 12824 ts_tablet_manager.cc:1403] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.003s
I20260709 13:17:30.374713 13002 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:30.375161 13002 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:30.375555 13002 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.375790 12824 raft_consensus.cc:359] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.376500 12824 raft_consensus.cc:385] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:30.376791 12824 raft_consensus.cc:740] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.377611 12824 consensus_queue.cc:260] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.379413 12825 tablet_bootstrap.cc:492] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:30.379830 12825 ts_tablet_manager.cc:1403] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.102s	user 0.008s	sys 0.004s
I20260709 13:17:30.382846 13002 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.384999 13002 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:30.386195 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:30.385684 12529 catalog_manager.cc:5697] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:30.386327 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:30.386737 12705 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.386888 12824 ts_tablet_manager.cc:1434] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.004s
W20260709 13:17:30.389513 12597 leader_election.cc:343] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:30.389693 12824 tablet_bootstrap.cc:492] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:30.396704 12825 raft_consensus.cc:359] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.397357 12825 raft_consensus.cc:385] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:30.397717 12824 tablet_bootstrap.cc:654] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:30.404196 12705 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:17:30.405922 12598 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:30.406446 12823 tablet_bootstrap.cc:492] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:30.406895 12823 ts_tablet_manager.cc:1403] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.120s	user 0.016s	sys 0.013s
I20260709 13:17:30.406689 12936 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:30.407373 12936 raft_consensus.cc:697] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:30.409138 12936 consensus_queue.cc:237] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.409577 12823 raft_consensus.cc:359] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.410187 12823 raft_consensus.cc:385] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:30.410473 12823 raft_consensus.cc:740] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.411044 12823 consensus_queue.cc:260] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.413035 12823 ts_tablet_manager.cc:1434] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:17:30.397681 12825 raft_consensus.cc:740] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.416571 12825 consensus_queue.cc:260] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.421226 12825 ts_tablet_manager.cc:1434] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 13:17:30.425034 12825 tablet_bootstrap.cc:492] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:30.427269 12823 tablet_bootstrap.cc:492] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:30.428922 12922 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:30.438699 12823 tablet_bootstrap.cc:654] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:30.445751 12825 tablet_bootstrap.cc:654] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:30.438444 13002 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:30.450215 12824 tablet_bootstrap.cc:492] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:30.450721 12824 ts_tablet_manager.cc:1403] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.061s	user 0.013s	sys 0.021s
I20260709 13:17:30.452677 12823 tablet_bootstrap.cc:492] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:30.453114 12823 ts_tablet_manager.cc:1403] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.026s	user 0.005s	sys 0.005s
I20260709 13:17:30.454806 12824 raft_consensus.cc:359] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.455466 12824 raft_consensus.cc:385] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:30.455714 12824 raft_consensus.cc:740] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.455757 12823 raft_consensus.cc:359] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.456419 12823 raft_consensus.cc:385] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:30.456341 12824 consensus_queue.cc:260] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.458511 12824 ts_tablet_manager.cc:1434] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:17:30.459020 12922 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:30.459532 12824 tablet_bootstrap.cc:492] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:30.456693 12823 raft_consensus.cc:740] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.462926 12823 consensus_queue.cc:260] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.465814 12823 ts_tablet_manager.cc:1434] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.005s
I20260709 13:17:30.471024 12824 tablet_bootstrap.cc:654] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:30.474459 12823 tablet_bootstrap.cc:492] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:30.474807 12922 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.475399 12922 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.475803 12529 catalog_manager.cc:5697] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 0 to 1, leader changed from <none> to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:30.478883 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:17:30.481050 12672 leader_election.cc:343] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:30.481971 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:30.477418 12922 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:30.502631 13076 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.503319 13076 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.631619 12823 tablet_bootstrap.cc:700] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530: Time spent opening tablet: real 0.150s	user 0.027s	sys 0.041s
I20260709 13:17:30.632437 12823 tablet_bootstrap.cc:654] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:30.642745 12628 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
W20260709 13:17:30.716763 13110 log.cc:927] Time spent T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:17:30.717657 13115 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.718211 13115 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.720283 13115 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:30.714651 13076 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:30.728137 12824 tablet_bootstrap.cc:492] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:30.728583 12824 ts_tablet_manager.cc:1403] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.269s	user 0.027s	sys 0.025s
I20260709 13:17:30.733704 12824 raft_consensus.cc:359] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.734395 12824 raft_consensus.cc:385] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:30.735704 13117 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.735996 12673 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:30.736207 13117 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.736791 13012 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:30.737118 12825 tablet_bootstrap.cc:492] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:30.737546 12825 ts_tablet_manager.cc:1403] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.313s	user 0.016s	sys 0.012s
I20260709 13:17:30.734647 12824 raft_consensus.cc:740] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.738874 12824 consensus_queue.cc:260] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.740338 13117 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:30.737142 13012 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:30.741376 13012 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.741942 12915 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:30.747848 12825 raft_consensus.cc:359] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.749053 12825 raft_consensus.cc:385] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:30.749397 12825 raft_consensus.cc:740] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.754508 12825 consensus_queue.cc:260] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.756801 12936 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.757323 12936 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.762730 12936 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:30.764180 12824 ts_tablet_manager.cc:1434] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.001s
I20260709 13:17:30.765496 12825 ts_tablet_manager.cc:1434] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:17:30.766921 12824 tablet_bootstrap.cc:492] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:30.774405 12825 tablet_bootstrap.cc:492] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:30.780381 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:30.781042 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:30.786036 12823 tablet_bootstrap.cc:492] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:30.787899 12922 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.795478 12922 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.788131 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:30.791556 13074 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.802903 12823 ts_tablet_manager.cc:1403] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.329s	user 0.038s	sys 0.043s
I20260709 13:17:30.807072 12747 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:30.807956 13078 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:30.801420 13074 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.813274 13074 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:30.797536 12922 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:30.830557 12781 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.831687 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:30.835484 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:30.839514 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:30.839987 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:30.842674 12782 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.841926 12631 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:30.813782 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:30.850355 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:30.859715 13035 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:17:30.854892 12673 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:17:30.894371 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 44948.9us
I20260709 13:17:30.896107 13074 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:30.896601 13074 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:30.896996 13074 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.901046 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:30.902024 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:30.934743 12824 tablet_bootstrap.cc:654] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:30.931257 12673 leader_election.cc:343] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:30.945758 13118 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:30.943117 13012 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:30.953343 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:30.955781 13012 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:30.958048 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:30.959154 12625 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:30.986971 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 73967.8us
I20260709 13:17:30.978053 13118 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
W20260709 13:17:30.997630 12672 consensus_peers.cc:597] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:31.009016 12823 raft_consensus.cc:359] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.009747 12823 raft_consensus.cc:385] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:31.010040 12823 raft_consensus.cc:740] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:31.044605 12823 consensus_queue.cc:260] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.051115 13128 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.976291 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:17:31.057699 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 48112.9us
I20260709 13:17:31.059856 13074 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.062151 13074 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:31.062418 12709 raft_consensus.cc:2393] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 1: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
W20260709 13:17:31.063570 12672 leader_election.cc:343] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.064059 12672 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:31.077550 12825 tablet_bootstrap.cc:654] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:31.052019 13128 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:17:31.099153 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 47289.1us
W20260709 13:17:31.066340 12597 leader_election.cc:343] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.108065 13125 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:31.112808 12824 tablet_bootstrap.cc:492] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:31.113283 12824 ts_tablet_manager.cc:1403] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.347s	user 0.020s	sys 0.031s
W20260709 13:17:31.113523 12598 leader_election.cc:343] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.118403 13128 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:31.121068 12776 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:31.132880 12672 leader_election.cc:343] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.133384 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:31.133701 13076 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.133992 12628 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:31.134212 13076 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.136359 13076 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:17:31.138046 12672 consensus_peers.cc:597] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:31.133013 12824 raft_consensus.cc:359] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.139232 12824 raft_consensus.cc:385] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:31.139523 12824 raft_consensus.cc:740] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:31.139647 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:31.140986 12824 consensus_queue.cc:260] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.143709 12824 ts_tablet_manager.cc:1434] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.000s
I20260709 13:17:31.144790 12824 tablet_bootstrap.cc:492] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:31.146116 12625 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
W20260709 13:17:31.113528 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 47222.3us
I20260709 13:17:31.146207 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:31.152876 12922 consensus_queue.cc:1048] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:31.154491 12598 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:17:31.155613 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 85367.6us
W20260709 13:17:31.155717 12672 leader_election.cc:343] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.157136 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:31.156761 12673 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:31.160655 12823 ts_tablet_manager.cc:1434] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.357s	user 0.012s	sys 0.026s
I20260709 13:17:31.166900 12823 tablet_bootstrap.cc:492] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:31.180119 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:17:31.162870 12597 leader_election.cc:343] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.219689 13123 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:31.220283 13123 raft_consensus.cc:697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:31.179571 13129 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:31.187588 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:31.256388 13132 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.208104 12628 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:31.257169 13132 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.221115 13123 consensus_queue.cc:237] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:17:31.211275 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 49369.5us
W20260709 13:17:31.293111 12672 leader_election.cc:343] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.300673 12824 tablet_bootstrap.cc:654] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:31.306308 12823 tablet_bootstrap.cc:654] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:31.308804 12598 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:17:31.309520 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 118171us
W20260709 13:17:31.316830 12598 leader_election.cc:343] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.316758 12936 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:31.316283 13116 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:31.320111 12747 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:31.323230 12825 tablet_bootstrap.cc:492] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:31.323640 12825 ts_tablet_manager.cc:1403] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.550s	user 0.062s	sys 0.057s
I20260709 13:17:31.344195 13076 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:31.366734 13129 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:31.376794 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:31.374435 13132 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:31.379446 12825 raft_consensus.cc:359] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.380128 12825 raft_consensus.cc:385] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:31.380422 12825 raft_consensus.cc:740] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:31.381083 12825 consensus_queue.cc:260] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:17:31.385576 12672 leader_election.cc:343] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.388022 12673 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:31.389511 12915 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:31.390041 12915 raft_consensus.cc:697] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:31.391067 12915 consensus_queue.cc:237] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.392602 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:31.395442 12825 ts_tablet_manager.cc:1434] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.072s	user 0.006s	sys 0.002s
I20260709 13:17:31.402423 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:31.403165 12628 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:31.404536 12673 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:31.408855 13127 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:31.409600 12825 tablet_bootstrap.cc:492] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:31.409396 13127 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:31.411281 13127 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:31.420311 12597 consensus_peers.cc:597] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:31.482870 12824 tablet_bootstrap.cc:492] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:31.483433 12824 ts_tablet_manager.cc:1403] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.339s	user 0.009s	sys 0.019s
I20260709 13:17:31.483736 12823 tablet_bootstrap.cc:492] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:31.484243 12823 ts_tablet_manager.cc:1403] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.323s	user 0.021s	sys 0.010s
W20260709 13:17:31.485116 12597 leader_election.cc:343] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.485656 12597 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:31.487090 12823 raft_consensus.cc:359] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.487882 12823 raft_consensus.cc:385] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:31.488183 12823 raft_consensus.cc:740] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:31.488893 12823 consensus_queue.cc:260] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.491654 12823 ts_tablet_manager.cc:1434] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:17:31.492825 12823 tablet_bootstrap.cc:492] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:31.495644 12825 tablet_bootstrap.cc:654] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:31.487151 13128 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:31.501809 12824 raft_consensus.cc:359] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.502951 12824 raft_consensus.cc:385] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:31.503176 12824 raft_consensus.cc:740] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:31.476891 13127 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.504163 12824 consensus_queue.cc:260] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.508891 12824 ts_tablet_manager.cc:1434] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:17:31.525692 12936 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.526242 12936 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.526726 12824 tablet_bootstrap.cc:492] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:31.529910 12936 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:31.531312 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:31.533463 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:17:31.535256 12598 leader_election.cc:343] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.535498 13127 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:17:31.541256 12579 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:17:31.542182 12672 leader_election.cc:343] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.528609 12532 catalog_manager.cc:5697] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:31.544284 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:31.548301 12532 catalog_manager.cc:5697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:31.550192 12823 tablet_bootstrap.cc:654] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:31.555217 12597 leader_election.cc:343] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.555785 12597 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:31.556537 13117 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:31.561523 12824 tablet_bootstrap.cc:654] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:31.572290 12915 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.572432 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:31.572909 12915 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.573091 12628 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:31.574399 12823 tablet_bootstrap.cc:492] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:31.574997 12823 ts_tablet_manager.cc:1403] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.082s	user 0.016s	sys 0.001s
I20260709 13:17:31.575238 12915 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:31.576583 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:17:31.578423 12672 leader_election.cc:343] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.580101 12823 raft_consensus.cc:359] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.581126 12823 raft_consensus.cc:385] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:31.581517 12823 raft_consensus.cc:740] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:31.589672 12823 consensus_queue.cc:260] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.592839 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:31.593555 12627 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:31.593984 12824 tablet_bootstrap.cc:492] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:31.596678 12824 ts_tablet_manager.cc:1403] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.070s	user 0.009s	sys 0.022s
I20260709 13:17:31.596864 12628 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:31.595151 12823 ts_tablet_manager.cc:1434] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:17:31.598958 12823 tablet_bootstrap.cc:492] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:31.595731 12673 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:31.601878 12824 raft_consensus.cc:359] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.602396 12673 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:31.602968 12824 raft_consensus.cc:385] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:31.603399 12915 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:31.603546 12824 raft_consensus.cc:740] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:31.604314 13127 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:31.604768 13127 raft_consensus.cc:697] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:31.604858 12824 consensus_queue.cc:260] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.605657 13127 consensus_queue.cc:237] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.603900 12915 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:31.607529 12915 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:31.608189 12824 ts_tablet_manager.cc:1434] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:17:31.609900 12824 tablet_bootstrap.cc:492] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:31.605433 12823 tablet_bootstrap.cc:654] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:31.617820 12824 tablet_bootstrap.cc:654] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:31.620047 12915 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.629128 12825 tablet_bootstrap.cc:492] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:31.641654 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:31.642035 12825 ts_tablet_manager.cc:1403] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.233s	user 0.044s	sys 0.044s
I20260709 13:17:31.642299 12628 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:31.645880 12825 raft_consensus.cc:359] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.646870 12825 raft_consensus.cc:385] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:31.647260 12825 raft_consensus.cc:740] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:31.648217 12825 consensus_queue.cc:260] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.656930 12915 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:31.660979 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:31.662200 12825 ts_tablet_manager.cc:1434] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.002s
W20260709 13:17:31.662343 12672 leader_election.cc:343] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.664748 13117 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.654635 12628 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:31.665989 12825 tablet_bootstrap.cc:492] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:31.665287 13117 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.669852 13117 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:31.670543 12823 tablet_bootstrap.cc:492] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:31.667054 12673 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:31.671048 12823 ts_tablet_manager.cc:1403] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.072s	user 0.016s	sys 0.000s
I20260709 13:17:31.674324 12823 raft_consensus.cc:359] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.676668 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:17:31.678313 12597 leader_election.cc:343] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.676962 12823 raft_consensus.cc:385] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:31.678869 12825 tablet_bootstrap.cc:654] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:31.679004 12823 raft_consensus.cc:740] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:31.679207 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:31.676587 12824 tablet_bootstrap.cc:492] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:31.679867 12706 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:31.680241 12824 ts_tablet_manager.cc:1403] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.071s	user 0.027s	sys 0.028s
I20260709 13:17:31.681752 13127 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:31.682369 13127 raft_consensus.cc:697] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:31.683372 12598 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:31.683282 13127 consensus_queue.cc:237] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.686610 13117 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:31.687029 13117 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:31.687381 13117 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:31.685245 12824 raft_consensus.cc:359] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.689656 12824 raft_consensus.cc:385] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:31.679888 12823 consensus_queue.cc:260] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.695672 13117 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.689986 12824 raft_consensus.cc:740] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:31.698215 12824 consensus_queue.cc:260] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.700452 12823 ts_tablet_manager.cc:1434] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 13:17:31.701483 12823 tablet_bootstrap.cc:492] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:31.703437 12824 ts_tablet_manager.cc:1434] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 13:17:31.703176 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:31.704442 12824 tablet_bootstrap.cc:492] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
W20260709 13:17:31.704993 12597 leader_election.cc:343] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.706081 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:31.706701 12709 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:31.707993 13117 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:31.715646 12915 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.717077 12824 tablet_bootstrap.cc:654] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:31.716439 12915 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.719868 12709 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:17:31.722398 12915 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:31.721062 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:31.727377 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:17:31.728101 12673 leader_election.cc:343] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.725987 12825 tablet_bootstrap.cc:492] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:31.728600 12598 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:31.729188 12825 ts_tablet_manager.cc:1403] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.063s	user 0.025s	sys 0.006s
I20260709 13:17:31.729645 13128 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:31.730327 13128 raft_consensus.cc:697] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:17:31.728770 12672 leader_election.cc:343] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.731177 12672 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:31.731499 13128 consensus_queue.cc:237] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.734380 12825 raft_consensus.cc:359] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.735117 12825 raft_consensus.cc:385] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:31.735234 12915 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:31.735512 12825 raft_consensus.cc:740] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:31.736657 12825 consensus_queue.cc:260] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.743981 12825 ts_tablet_manager.cc:1434] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:17:31.745504 12825 tablet_bootstrap.cc:492] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:31.756539 12824 tablet_bootstrap.cc:492] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:31.757045 12824 ts_tablet_manager.cc:1403] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.053s	user 0.020s	sys 0.016s
I20260709 13:17:31.758795 12825 tablet_bootstrap.cc:654] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:31.760157 12824 raft_consensus.cc:359] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.760874 12824 raft_consensus.cc:385] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:31.761134 12824 raft_consensus.cc:740] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:31.762391 12823 tablet_bootstrap.cc:654] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:31.761875 12824 consensus_queue.cc:260] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.766363 12824 ts_tablet_manager.cc:1434] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.009s	user 0.001s	sys 0.004s
I20260709 13:17:31.767287 12824 tablet_bootstrap.cc:492] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:31.773401 12824 tablet_bootstrap.cc:654] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:31.778453 13136 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:31.788862 13119 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.790505 13119 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.793411 13119 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:31.797663 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:31.803517 13117 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.804092 13117 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.806437 13117 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:31.812144 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:31.812950 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:17:31.813817 12597 leader_election.cc:343] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.815856 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:17:31.817345 12598 leader_election.cc:343] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.817799 12598 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:31.822750 12747 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:31.824188 13119 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:31.827001 12776 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:31.831239 12672 consensus_peers.cc:597] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:31.832476 13128 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:31.871526 12530 catalog_manager.cc:5697] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 0 to 1, leader changed from <none> to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:31.876688 13136 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:31.848541 12532 catalog_manager.cc:5697] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:31.909267 12672 consensus_peers.cc:597] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:31.912520 12532 catalog_manager.cc:5697] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:31.969658 12823 tablet_bootstrap.cc:492] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:31.969791 13035 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.970278 12823 ts_tablet_manager.cc:1403] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.269s	user 0.030s	sys 0.073s
I20260709 13:17:31.970310 13035 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.972798 13035 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:31.975430 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:31.978963 12823 raft_consensus.cc:359] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.979825 12823 raft_consensus.cc:385] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:31.980136 12823 raft_consensus.cc:740] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:31.980865 12823 consensus_queue.cc:260] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.981298 13076 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.981902 13076 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.983793 12823 ts_tablet_manager.cc:1434] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 13:17:31.984294 13076 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:31.984925 12823 tablet_bootstrap.cc:492] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:31.978971 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:31.984861 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:31.989939 12747 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:31.989872 13136 consensus_queue.cc:1048] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:31.991571 12824 tablet_bootstrap.cc:492] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:31.992067 12824 ts_tablet_manager.cc:1403] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.225s	user 0.018s	sys 0.078s
I20260709 13:17:31.994936 12824 raft_consensus.cc:359] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.995649 12824 raft_consensus.cc:385] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:31.995944 12824 raft_consensus.cc:740] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:31.996958 12824 consensus_queue.cc:260] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:31.997777 13117 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.998221 13117 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.002233 13117 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:32.003607 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:32.004530 12824 ts_tablet_manager.cc:1434] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:17:32.005164 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:32.005913 12824 tablet_bootstrap.cc:492] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:32.005509 13035 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:32.008163 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:17:32.012128 12597 leader_election.cc:343] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.010586 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:32.015894 13035 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:32.016321 12823 tablet_bootstrap.cc:654] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:32.020011 12824 tablet_bootstrap.cc:654] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:32.020139 12598 leader_election.cc:343] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.020622 12598 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:32.022199 13149 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:32.023531 13136 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.024048 13136 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.026782 13136 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:17:32.029417 12672 consensus_peers.cc:597] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:32.031102 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:32.031639 12824 tablet_bootstrap.cc:492] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:32.032166 12824 ts_tablet_manager.cc:1403] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.002s
W20260709 13:17:32.034360 12672 leader_election.cc:343] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.038919 12824 raft_consensus.cc:359] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.039733 12824 raft_consensus.cc:385] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.039950 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:32.042309 12627 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:32.040051 12824 raft_consensus.cc:740] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.045475 12824 consensus_queue.cc:260] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.045759 12673 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:32.046932 13136 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:32.047240 12824 ts_tablet_manager.cc:1434] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:17:32.047382 13136 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:32.048272 13136 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.056524 13136 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.059616 13136 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:32.048249 12824 tablet_bootstrap.cc:492] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:32.060631 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:17:32.061834 12672 leader_election.cc:343] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.063520 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:32.065313 12627 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.075999 12825 tablet_bootstrap.cc:492] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:32.076253 12824 tablet_bootstrap.cc:654] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:32.076501 12825 ts_tablet_manager.cc:1403] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.331s	user 0.061s	sys 0.084s
I20260709 13:17:32.081130 13132 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:32.085583 12823 tablet_bootstrap.cc:492] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:32.086100 12823 ts_tablet_manager.cc:1403] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.101s	user 0.015s	sys 0.009s
I20260709 13:17:32.086462 12825 raft_consensus.cc:359] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.103156 12825 raft_consensus.cc:385] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.103484 12825 raft_consensus.cc:740] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
W20260709 13:17:32.104099 12672 consensus_peers.cc:597] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:32.104302 12825 consensus_queue.cc:260] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.096925 13128 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.106681 13128 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.107678 12825 ts_tablet_manager.cc:1434] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.002s
I20260709 13:17:32.108646 12825 tablet_bootstrap.cc:492] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:32.114050 12823 raft_consensus.cc:359] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.114989 12823 raft_consensus.cc:385] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.115293 12823 raft_consensus.cc:740] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.115231 13132 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.116194 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:32.116277 13132 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.116721 13128 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:32.117813 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:32.119710 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:17:32.121158 12673 leader_election.cc:343] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:32.122402 12598 leader_election.cc:343] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.125288 13132 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:32.116065 12823 consensus_queue.cc:260] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:17:32.132016 12597 leader_election.cc:343] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.132575 12597 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:32.134446 13128 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:32.140780 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:17:32.145068 12672 leader_election.cc:343] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.145587 12672 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:32.146325 12915 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:17:32.157114 12597 consensus_peers.cc:597] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:32.158515 13117 consensus_queue.cc:1048] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:32.162047 12627 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:32.163590 12673 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:32.164755 13132 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:32.166326 13132 raft_consensus.cc:697] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:32.167279 13132 consensus_queue.cc:237] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.170648 12825 tablet_bootstrap.cc:654] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:32.177856 12775 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.180168 12823 ts_tablet_manager.cc:1434] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.094s	user 0.006s	sys 0.009s
I20260709 13:17:32.173265 12776 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.215108 12824 tablet_bootstrap.cc:492] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:32.215596 12824 ts_tablet_manager.cc:1403] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.168s	user 0.039s	sys 0.013s
I20260709 13:17:32.216346 12823 tablet_bootstrap.cc:492] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:32.223106 12824 raft_consensus.cc:359] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.223870 12824 raft_consensus.cc:385] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.224176 12824 raft_consensus.cc:740] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.225049 12824 consensus_queue.cc:260] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.232812 12824 ts_tablet_manager.cc:1434] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.002s
I20260709 13:17:32.257123 12823 tablet_bootstrap.cc:654] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:32.257555 12824 tablet_bootstrap.cc:492] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:32.268996 12532 catalog_manager.cc:5697] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:32.275669 12823 tablet_bootstrap.cc:492] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:32.276201 12823 ts_tablet_manager.cc:1403] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.093s	user 0.008s	sys 0.006s
I20260709 13:17:32.280022 12823 raft_consensus.cc:359] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.280664 12823 raft_consensus.cc:385] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.280936 12823 raft_consensus.cc:740] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.282114 12823 consensus_queue.cc:260] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.287771 12823 ts_tablet_manager.cc:1434] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.006s
I20260709 13:17:32.288882 12823 tablet_bootstrap.cc:492] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:32.293229 12825 tablet_bootstrap.cc:492] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:32.293673 12824 tablet_bootstrap.cc:654] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:32.295106 12825 ts_tablet_manager.cc:1403] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.187s	user 0.010s	sys 0.011s
I20260709 13:17:32.298718 12825 raft_consensus.cc:359] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.299564 12825 raft_consensus.cc:385] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.299832 12825 raft_consensus.cc:740] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.300479 12825 consensus_queue.cc:260] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.303484 12825 ts_tablet_manager.cc:1434] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:17:32.305117 12825 tablet_bootstrap.cc:492] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:32.313508 12825 tablet_bootstrap.cc:654] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:32.316555 12823 tablet_bootstrap.cc:654] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:32.327855 12824 tablet_bootstrap.cc:492] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:32.328323 12825 tablet_bootstrap.cc:492] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:32.328327 12824 ts_tablet_manager.cc:1403] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.071s	user 0.016s	sys 0.018s
I20260709 13:17:32.328809 12825 ts_tablet_manager.cc:1403] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.007s
I20260709 13:17:32.330037 13127 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.330513 13127 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.331290 12825 raft_consensus.cc:359] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.331970 12825 raft_consensus.cc:385] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.332231 12825 raft_consensus.cc:740] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.332931 12825 consensus_queue.cc:260] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.334331 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:32.332533 13127 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:32.335235 12825 ts_tablet_manager.cc:1434] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 13:17:32.336292 12825 tablet_bootstrap.cc:492] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:32.337054 12824 raft_consensus.cc:359] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.337728 12824 raft_consensus.cc:385] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.338716 12824 raft_consensus.cc:740] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.339129 12823 tablet_bootstrap.cc:492] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:32.339579 12823 ts_tablet_manager.cc:1403] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.051s	user 0.005s	sys 0.005s
I20260709 13:17:32.339409 12824 consensus_queue.cc:260] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.342044 12824 ts_tablet_manager.cc:1434] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:17:32.342415 12825 tablet_bootstrap.cc:654] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:32.342228 12823 raft_consensus.cc:359] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.342923 12823 raft_consensus.cc:385] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.342994 12824 tablet_bootstrap.cc:492] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:32.343237 12823 raft_consensus.cc:740] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
W20260709 13:17:32.339668 12673 leader_election.cc:343] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.349498 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:32.351671 12824 tablet_bootstrap.cc:654] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:32.353466 13035 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:32.354081 12672 leader_election.cc:343] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.353967 13035 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.354477 12672 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:32.359676 13035 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:32.360985 12825 tablet_bootstrap.cc:492] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:32.360895 12915 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:32.361518 12825 ts_tablet_manager.cc:1403] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.025s	user 0.019s	sys 0.003s
I20260709 13:17:32.361927 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:32.380111 12824 tablet_bootstrap.cc:492] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:32.381641 12824 ts_tablet_manager.cc:1403] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.039s	user 0.022s	sys 0.011s
I20260709 13:17:32.380815 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:32.382200 12825 raft_consensus.cc:359] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.384385 12824 raft_consensus.cc:359] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.385145 12824 raft_consensus.cc:385] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.385429 12824 raft_consensus.cc:740] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.382916 12825 raft_consensus.cc:385] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.385968 12825 raft_consensus.cc:740] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.386107 12824 consensus_queue.cc:260] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.386641 12825 consensus_queue.cc:260] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.388234 12824 ts_tablet_manager.cc:1434] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:17:32.388859 12825 ts_tablet_manager.cc:1434] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.000s
I20260709 13:17:32.389240 12824 tablet_bootstrap.cc:492] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:32.389806 12825 tablet_bootstrap.cc:492] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:32.343925 12823 consensus_queue.cc:260] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.391196 12747 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:32.391520 12915 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.392078 12915 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.392505 13117 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.392952 13117 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.393751 13076 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:32.395059 13117 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:32.396324 12824 tablet_bootstrap.cc:654] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:32.396456 12825 tablet_bootstrap.cc:654] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:32.397465 12915 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:32.402208 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:17:32.403605 12597 leader_election.cc:343] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.405682 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:32.406352 12628 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:32.409746 12673 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:32.410516 13127 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:32.411003 13127 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:32.411445 13127 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.421427 13127 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.423591 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:32.424288 12776 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:32.426394 13127 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:32.427129 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:32.428369 12823 ts_tablet_manager.cc:1434] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.088s	user 0.007s	sys 0.002s
I20260709 13:17:32.429787 12628 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.430035 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:32.430721 12706 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:32.432504 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:32.433182 12776 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.445531 12598 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:32.449602 13149 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:32.449980 13149 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:32.450297 13149 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.457818 13128 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.458349 13128 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:17:32.459043 12735 tablet.cc:2406] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:17:32.460470 13128 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:32.461628 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:32.462535 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:17:32.462983 12597 leader_election.cc:343] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.463315 12706 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:32.465440 12598 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:32.466625 13128 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:32.467015 13128 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:32.469609 13128 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.473804 13149 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.489459 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:32.430665 12823 tablet_bootstrap.cc:492] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:32.490213 12709 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.445813 12628 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:32.493283 12673 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:32.494120 13127 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:32.490079 13149 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:32.494763 13127 raft_consensus.cc:697] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:32.495522 13149 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.495999 13149 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.496279 13127 consensus_queue.cc:237] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.502692 12709 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:17:32.503950 13149 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:32.527860 12776 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:32.530104 13035 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.530680 13035 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.533528 13035 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:32.533908 12528 catalog_manager.cc:5697] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:32.535979 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:17:32.544116 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 75381.6us
I20260709 13:17:32.548982 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:17:32.554788 12597 leader_election.cc:343] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.556438 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:32.555846 13035 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.559303 13035 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.561542 13035 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:32.565712 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:17:32.567847 12597 leader_election.cc:343] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.573470 12598 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:32.574587 12776 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.574843 13117 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:32.575512 13117 raft_consensus.cc:697] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:32.576464 13117 consensus_queue.cc:237] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.583259 12775 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.585680 13132 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.586367 13132 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.589043 13132 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:32.591874 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:32.592846 12627 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:32.592991 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:32.594363 12673 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:32.594905 12747 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:32.595441 13132 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:32.595995 13132 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:32.596437 13132 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.603142 13035 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:32.605046 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:32.607095 12915 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:32.616362 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:32.618304 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:32.620648 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:32.621541 12706 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:32.626330 12824 tablet_bootstrap.cc:492] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:32.629886 12824 ts_tablet_manager.cc:1403] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.241s	user 0.081s	sys 0.098s
W20260709 13:17:32.638706 12672 leader_election.cc:343] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.639874 12823 tablet_bootstrap.cc:654] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:32.655485 12825 tablet_bootstrap.cc:492] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:32.656002 12825 ts_tablet_manager.cc:1403] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.266s	user 0.072s	sys 0.078s
I20260709 13:17:32.656301 12532 catalog_manager.cc:5697] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 0 to 1, leader changed from <none> to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:32.673609 12825 raft_consensus.cc:359] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.674386 12825 raft_consensus.cc:385] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.674659 12825 raft_consensus.cc:740] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.675316 12825 consensus_queue.cc:260] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.662081 12598 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:32.681936 12824 raft_consensus.cc:359] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.684177 12824 raft_consensus.cc:385] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.684473 12824 raft_consensus.cc:740] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.684692 12825 ts_tablet_manager.cc:1434] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:17:32.686022 12825 tablet_bootstrap.cc:492] T 8f91059867c444d89718e83475c089cd P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:32.686383 12778 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.681839 13117 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:32.688623 12824 consensus_queue.cc:260] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.685019 13132 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.694615 13128 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.695973 13132 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:17:32.697547 12672 consensus_peers.cc:597] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:32.688918 13117 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:32.701555 13117 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.703104 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:32.704720 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:32.705266 12627 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:32.705994 12672 leader_election.cc:343] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.709091 13128 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:32.709483 12824 ts_tablet_manager.cc:1434] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.079s	user 0.007s	sys 0.000s
I20260709 13:17:32.710491 12824 tablet_bootstrap.cc:492] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:32.713080 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:17:32.714844 12597 leader_election.cc:343] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.721228 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:32.721871 12706 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:32.723649 12579 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:32.730005 12706 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:17:32.731263 12598 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:32.732132 13128 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:32.732591 13128 raft_consensus.cc:697] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:32.749585 13128 consensus_queue.cc:237] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.754060 13035 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:32.757992 13188 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.760696 12627 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:32.739969 13117 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.760090 13188 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.769786 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:32.771304 12673 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:32.772339 13135 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:32.772799 13135 raft_consensus.cc:697] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:32.773494 13188 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:32.773609 13135 consensus_queue.cc:237] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.775023 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:17:32.776288 12597 leader_election.cc:343] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:32.780011 12598 leader_election.cc:343] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.780534 12598 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:32.781874 13193 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:32.789932 13117 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:32.791960 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:17:32.795704 12597 leader_election.cc:343] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.792263 12532 catalog_manager.cc:5697] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 0 to 1, leader changed from <none> to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:32.804566 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:32.805512 12703 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.813792 12823 tablet_bootstrap.cc:492] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:32.814328 12823 ts_tablet_manager.cc:1403] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.384s	user 0.016s	sys 0.005s
I20260709 13:17:32.820899 12824 tablet_bootstrap.cc:654] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:32.824383 12703 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:17:32.825750 12598 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:32.826570 13117 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:32.827246 12825 tablet_bootstrap.cc:654] T 8f91059867c444d89718e83475c089cd P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:32.827387 13117 raft_consensus.cc:697] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:32.828671 13117 consensus_queue.cc:237] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.817493 12823 raft_consensus.cc:359] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.830539 12823 raft_consensus.cc:385] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.830905 12823 raft_consensus.cc:740] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.831676 12823 consensus_queue.cc:260] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.834072 12823 ts_tablet_manager.cc:1434] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:17:32.835292 12823 tablet_bootstrap.cc:492] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:32.856527 12823 tablet_bootstrap.cc:654] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:32.858453 13035 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.876148 13035 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.879604 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:32.880606 13035 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:32.881414 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:32.885975 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:32.885896 12532 catalog_manager.cc:5697] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:32.904276 13076 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:32.890784 12824 tablet_bootstrap.cc:492] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:32.938522 12824 ts_tablet_manager.cc:1403] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.228s	user 0.019s	sys 0.004s
I20260709 13:17:32.942597 12778 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.949203 12823 tablet_bootstrap.cc:492] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:32.949796 12823 ts_tablet_manager.cc:1403] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.115s	user 0.013s	sys 0.004s
I20260709 13:17:32.953176 13136 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:32.957779 12823 raft_consensus.cc:359] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.958513 12823 raft_consensus.cc:385] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.958773 12823 raft_consensus.cc:740] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.961668 13117 consensus_queue.cc:1048] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:32.961642 12823 consensus_queue.cc:260] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.969147 12823 ts_tablet_manager.cc:1434] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:17:32.970130 12823 tablet_bootstrap.cc:492] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:32.976615 12532 catalog_manager.cc:5697] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 0 to 1, leader changed from <none> to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:32.977839 12824 raft_consensus.cc:359] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:32.979589 12824 raft_consensus.cc:385] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.980094 12824 raft_consensus.cc:740] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.997222 13135 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:33.002831 12824 consensus_queue.cc:260] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.020494 12825 tablet_bootstrap.cc:492] T 8f91059867c444d89718e83475c089cd P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:33.021051 12825 ts_tablet_manager.cc:1403] T 8f91059867c444d89718e83475c089cd P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.335s	user 0.038s	sys 0.053s
I20260709 13:17:33.033730 12823 tablet_bootstrap.cc:654] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:33.041285 12597 consensus_peers.cc:597] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:33.042505 12824 ts_tablet_manager.cc:1434] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.103s	user 0.006s	sys 0.002s
I20260709 13:17:33.043634 12824 tablet_bootstrap.cc:492] T 8e0d0566acd7465f87e20053c75e28fa P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:33.046679 12825 raft_consensus.cc:359] T 8f91059867c444d89718e83475c089cd P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.051959 12825 raft_consensus.cc:385] T 8f91059867c444d89718e83475c089cd P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:33.052358 12825 raft_consensus.cc:740] T 8f91059867c444d89718e83475c089cd P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:33.053164 12825 consensus_queue.cc:260] T 8f91059867c444d89718e83475c089cd P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.053890 12824 tablet_bootstrap.cc:654] T 8e0d0566acd7465f87e20053c75e28fa P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:33.055660 12825 ts_tablet_manager.cc:1434] T 8f91059867c444d89718e83475c089cd P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 13:17:33.056838 12825 tablet_bootstrap.cc:492] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:33.063345 12823 tablet_bootstrap.cc:492] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:33.063900 12823 ts_tablet_manager.cc:1403] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.094s	user 0.011s	sys 0.001s
I20260709 13:17:33.067957 12823 raft_consensus.cc:359] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.071395 12823 raft_consensus.cc:385] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:33.071794 12823 raft_consensus.cc:740] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:33.072532 12823 consensus_queue.cc:260] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.074874 12823 ts_tablet_manager.cc:1434] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:17:33.078678 12825 tablet_bootstrap.cc:654] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:33.091573 12823 tablet_bootstrap.cc:492] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:33.098753 12823 tablet_bootstrap.cc:654] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:33.103824 12824 tablet_bootstrap.cc:492] T 8e0d0566acd7465f87e20053c75e28fa P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:33.104357 12824 ts_tablet_manager.cc:1403] T 8e0d0566acd7465f87e20053c75e28fa P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.061s	user 0.019s	sys 0.012s
I20260709 13:17:33.106977 12824 raft_consensus.cc:359] T 8e0d0566acd7465f87e20053c75e28fa P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.107692 12824 raft_consensus.cc:385] T 8e0d0566acd7465f87e20053c75e28fa P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:33.108016 12824 raft_consensus.cc:740] T 8e0d0566acd7465f87e20053c75e28fa P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:33.108793 12824 consensus_queue.cc:260] T 8e0d0566acd7465f87e20053c75e28fa P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.132180 12825 tablet_bootstrap.cc:492] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:33.132696 12825 ts_tablet_manager.cc:1403] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.076s	user 0.011s	sys 0.000s
I20260709 13:17:33.136737 12825 raft_consensus.cc:359] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.137732 12825 raft_consensus.cc:385] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:33.138142 12825 raft_consensus.cc:740] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:33.139721 12825 consensus_queue.cc:260] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.154458 12824 ts_tablet_manager.cc:1434] T 8e0d0566acd7465f87e20053c75e28fa P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.050s	user 0.006s	sys 0.000s
I20260709 13:17:33.159863 12825 ts_tablet_manager.cc:1434] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.002s
I20260709 13:17:33.161000 12825 tablet_bootstrap.cc:492] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:33.166469 12824 tablet_bootstrap.cc:492] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:33.174288 12823 tablet_bootstrap.cc:492] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:33.174923 12823 ts_tablet_manager.cc:1403] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.084s	user 0.023s	sys 0.017s
I20260709 13:17:33.187512 12825 tablet_bootstrap.cc:654] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:33.189278 12824 tablet_bootstrap.cc:654] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:33.191891 12823 raft_consensus.cc:359] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.192374 12823 raft_consensus.cc:385] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:33.192644 12823 raft_consensus.cc:740] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:33.193466 12823 consensus_queue.cc:260] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.195580 12823 ts_tablet_manager.cc:1434] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.000s
I20260709 13:17:33.197032 12823 tablet_bootstrap.cc:492] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:33.208123 12823 tablet_bootstrap.cc:654] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:33.229084 12824 tablet_bootstrap.cc:492] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:33.229986 12824 ts_tablet_manager.cc:1403] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.063s	user 0.015s	sys 0.002s
I20260709 13:17:33.233848 13136 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.234243 12776 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:33.234392 13136 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.237040 13136 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:33.239300 13136 consensus_queue.cc:1048] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:33.240068 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:33.240800 12628 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:33.242481 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:33.242681 12673 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:33.243124 12823 tablet_bootstrap.cc:492] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:33.243574 12823 ts_tablet_manager.cc:1403] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.047s	user 0.015s	sys 0.009s
I20260709 13:17:33.244460 13132 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:33.244861 13132 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:33.245123 12825 tablet_bootstrap.cc:492] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:33.245213 13132 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:33.246340 12823 raft_consensus.cc:359] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.246964 12825 ts_tablet_manager.cc:1403] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.086s	user 0.016s	sys 0.007s
I20260709 13:17:33.247056 12823 raft_consensus.cc:385] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:33.247444 12823 raft_consensus.cc:740] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:33.248080 12823 consensus_queue.cc:260] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.260452 12824 raft_consensus.cc:359] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.261420 12824 raft_consensus.cc:385] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:33.261785 12824 raft_consensus.cc:740] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:33.262738 12824 consensus_queue.cc:260] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.265743 12824 ts_tablet_manager.cc:1434] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
W20260709 13:17:33.268858 12672 consensus_peers.cc:597] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:33.272521 12824 tablet_bootstrap.cc:492] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
W20260709 13:17:33.274621 12672 leader_election.cc:343] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:33.263227 13132 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.282686 12823 ts_tablet_manager.cc:1434] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.039s	user 0.014s	sys 0.007s
I20260709 13:17:33.280715 13214 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.293836 12823 tablet_bootstrap.cc:492] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:33.297977 13214 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.309837 13214 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:33.296665 13035 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.311120 13035 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.289722 13193 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 13:17:33.316488 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:33.318743 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:33.335268 12706 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 0.
I20260709 13:17:33.335151 12627 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:33.315419 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:33.340881 13132 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:33.313469 12825 raft_consensus.cc:359] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.316289 13127 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.313426 13117 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.314412 13035 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:33.343021 12825 raft_consensus.cc:385] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:33.343344 12825 raft_consensus.cc:740] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:33.344197 12825 consensus_queue.cc:260] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.317965 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:33.353039 12825 ts_tablet_manager.cc:1434] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.106s	user 0.004s	sys 0.002s
I20260709 13:17:33.344671 13117 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.355418 13117 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:33.355974 12630 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 0.
I20260709 13:17:33.357138 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:33.358875 12782 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:33.360091 12597 consensus_peers.cc:597] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:33.359732 12747 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:17:33.362572 13076 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:33.362919 13076 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:33.363215 13076 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:33.365777 12823 tablet_bootstrap.cc:654] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:33.344795 13127 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:17:33.371668 12672 leader_election.cc:343] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:33.359272 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:33.381772 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:33.382567 12706 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:33.367079 12629 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:33.390291 12673 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:33.391036 12825 tablet_bootstrap.cc:492] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:33.370704 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:33.397437 12629 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:33.399602 12673 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:33.400642 13136 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:33.401113 13136 raft_consensus.cc:697] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:33.401947 13136 consensus_queue.cc:237] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.404150 13132 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:33.405247 13132 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:33.393778 13117 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.406005 13076 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.407377 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:33.407997 12628 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:33.408818 13076 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:33.406509 13117 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.410043 12673 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:33.415547 13216 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:33.412688 12778 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 1: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:17:33.428962 12598 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:33.427397 13193 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:17:33.431221 13127 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:33.431290 13193 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:33.431777 13193 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:33.431938 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:33.432171 13193 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:33.433673 12672 leader_election.cc:343] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:33.420982 13132 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:33.438802 12824 tablet_bootstrap.cc:654] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:33.427799 13216 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:33.426857 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 58894.7us
I20260709 13:17:33.450088 13117 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:33.445950 13216 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:33.465471 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:33.465975 12709 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:33.467108 12672 leader_election.cc:343] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:33.462765 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:33.467125 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:17:33.501082 12597 leader_election.cc:343] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:33.477411 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:33.502765 13149 consensus_queue.cc:1048] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:17:33.524837 12823 tablet_bootstrap.cc:492] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:33.526110 12823 ts_tablet_manager.cc:1403] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.233s	user 0.018s	sys 0.001s
I20260709 13:17:33.527647 13229 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.528126 13229 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.529337 12709 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:17:33.540899 12825 tablet_bootstrap.cc:654] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:33.545409 13216 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.533593 13132 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:17:33.579528 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 103786us
W20260709 13:17:33.579865 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 57457.5us
W20260709 13:17:33.610256 12598 leader_election.cc:343] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:33.610684 12598 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:17:33.579344 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 123046us
I20260709 13:17:33.617834 13132 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:33.619472 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:33.614048 13238 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:17:33.666424 13227 log.cc:927] Time spent T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:17:33.693192 12824 tablet_bootstrap.cc:492] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:33.693746 12824 ts_tablet_manager.cc:1403] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.422s	user 0.018s	sys 0.005s
I20260709 13:17:33.699215 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:17:33.720245 12672 leader_election.cc:343] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:33.734443 13229 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:33.742857 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:33.743747 12625 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:33.727640 13216 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:33.762544 13193 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.763192 12625 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:33.769771 12823 raft_consensus.cc:359] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.770532 12823 raft_consensus.cc:385] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:33.770802 12823 raft_consensus.cc:740] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:33.786026 12823 consensus_queue.cc:260] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.785357 12629 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 1: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:17:33.791793 12825 tablet_bootstrap.cc:492] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:33.792306 12825 ts_tablet_manager.cc:1403] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.402s	user 0.041s	sys 0.036s
I20260709 13:17:33.795925 13193 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:33.797284 12825 raft_consensus.cc:359] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.801481 12825 raft_consensus.cc:385] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:33.801820 12825 raft_consensus.cc:740] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:33.802474 12825 consensus_queue.cc:260] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.804477 12825 ts_tablet_manager.cc:1434] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:17:33.806186 12825 tablet_bootstrap.cc:492] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:33.788940 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:33.807076 13216 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.807571 13216 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.810392 12824 raft_consensus.cc:359] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.811087 12824 raft_consensus.cc:385] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:33.811321 12824 raft_consensus.cc:740] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:33.811848 12824 consensus_queue.cc:260] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:17:33.819216 12579 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:17:33.840077 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 146391us
I20260709 13:17:33.840966 12747 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:33.842993 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:33.847595 13219 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:33.848029 12634 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:33.850734 12823 ts_tablet_manager.cc:1434] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.324s	user 0.007s	sys 0.005s
I20260709 13:17:33.850720 12824 ts_tablet_manager.cc:1434] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.157s	user 0.006s	sys 0.003s
I20260709 13:17:33.839627 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:33.852696 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:33.855085 12709 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 1: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:17:33.855439 13216 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:33.853928 13219 raft_consensus.cc:697] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:17:33.872536 12528 catalog_manager.cc:5697] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:33.881506 12824 tablet_bootstrap.cc:492] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:33.864735 13219 consensus_queue.cc:237] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:33.905771 12634 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:34.022882 12825 tablet_bootstrap.cc:654] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:34.024812 12824 tablet_bootstrap.cc:654] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:34.043470 12823 tablet_bootstrap.cc:492] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:34.029330 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:17:34.049038 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 140171us
W20260709 13:17:34.075583 12672 leader_election.cc:343] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:34.084385 12673 leader_election.cc:343] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:34.084858 12673 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:17:34.081363 12672 leader_election.cc:343] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:34.086371 13216 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:34.086910 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:17:34.088804 12672 leader_election.cc:343] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:34.091656 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:34.092298 12629 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:34.097795 12823 tablet_bootstrap.cc:654] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:34.114269 12597 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:34.115952 13193 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:17:34.120854 12672 consensus_peers.cc:597] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:34.123687 12825 tablet_bootstrap.cc:492] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:34.124241 12825 ts_tablet_manager.cc:1403] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.318s	user 0.017s	sys 0.008s
I20260709 13:17:34.124845 12673 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:34.126484 13234 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:34.127084 13234 raft_consensus.cc:697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:34.127632 12825 raft_consensus.cc:359] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.128474 12825 raft_consensus.cc:385] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:34.128840 12825 raft_consensus.cc:740] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:34.129671 12825 consensus_queue.cc:260] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.132018 12825 ts_tablet_manager.cc:1434] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:17:34.133505 12825 tablet_bootstrap.cc:492] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:34.134927 12777 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:34.135275 13242 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.135984 13242 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.128075 13234 consensus_queue.cc:237] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.149840 12673 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:34.151255 12825 tablet_bootstrap.cc:654] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:34.150998 13233 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:34.151885 13233 raft_consensus.cc:697] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:34.153412 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:34.156172 12673 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:34.160187 12824 tablet_bootstrap.cc:492] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:34.160708 12824 ts_tablet_manager.cc:1403] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.291s	user 0.021s	sys 0.004s
I20260709 13:17:34.151062 13242 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:34.154104 13233 consensus_queue.cc:237] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:17:34.170473 12597 leader_election.cc:343] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:34.175973 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:34.177232 12824 raft_consensus.cc:359] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.178493 12824 raft_consensus.cc:385] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:34.178946 12824 raft_consensus.cc:740] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:34.179827 12824 consensus_queue.cc:260] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.182551 13228 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:34.183076 13228 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:34.183508 13228 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:34.186817 12823 tablet_bootstrap.cc:492] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:34.187355 12823 ts_tablet_manager.cc:1403] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.144s	user 0.013s	sys 0.006s
W20260709 13:17:34.188654 12598 leader_election.cc:343] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:34.189180 12598 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:34.193761 13244 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:34.197438 12824 ts_tablet_manager.cc:1434] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 13:17:34.198733 12824 tablet_bootstrap.cc:492] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:34.197397 12823 raft_consensus.cc:359] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.201138 12823 raft_consensus.cc:385] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:34.201417 12823 raft_consensus.cc:740] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:34.202471 12823 consensus_queue.cc:260] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.204893 12823 ts_tablet_manager.cc:1434] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:17:34.209430 12825 tablet_bootstrap.cc:492] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:34.210000 12825 ts_tablet_manager.cc:1403] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.077s	user 0.011s	sys 0.004s
I20260709 13:17:34.206333 13228 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.206178 13234 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:34.214915 12825 raft_consensus.cc:359] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.215553 12825 raft_consensus.cc:385] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:34.215835 12825 raft_consensus.cc:740] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:34.216385 12825 consensus_queue.cc:260] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.221366 12823 tablet_bootstrap.cc:492] T 9155962ca7f34a8c83a1780ce0018a59 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:34.222738 12825 ts_tablet_manager.cc:1434] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.002s
I20260709 13:17:34.223667 12825 tablet_bootstrap.cc:492] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:34.226619 12778 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:34.241184 13228 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:34.241730 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:34.242554 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:34.244621 12630 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:34.245388 12672 leader_election.cc:343] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:34.248579 12823 tablet_bootstrap.cc:654] T 9155962ca7f34a8c83a1780ce0018a59 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:34.254915 12532 catalog_manager.cc:5697] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:34.263317 12825 tablet_bootstrap.cc:654] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:34.288650 12630 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:34.290766 12530 catalog_manager.cc:5697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:34.292989 12530 catalog_manager.cc:5697] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:34.297964 12673 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:34.303893 13229 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:34.304476 13229 raft_consensus.cc:697] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:34.305495 13229 consensus_queue.cc:237] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.317286 12824 tablet_bootstrap.cc:654] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:34.395853 13234 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.396381 13234 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.404675 13234 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:34.406280 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:34.433223 12825 tablet_bootstrap.cc:492] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:34.433821 12825 ts_tablet_manager.cc:1403] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.210s	user 0.032s	sys 0.039s
W20260709 13:17:34.433913 12672 leader_election.cc:343] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:34.421066 13266 log.cc:927] Time spent T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
W20260709 13:17:34.445832 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 83593.2us
I20260709 13:17:34.447356 12528 catalog_manager.cc:5697] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:34.452409 12823 tablet_bootstrap.cc:492] T 9155962ca7f34a8c83a1780ce0018a59 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:34.452888 12823 ts_tablet_manager.cc:1403] T 9155962ca7f34a8c83a1780ce0018a59 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.244s	user 0.020s	sys 0.016s
I20260709 13:17:34.454968 12823 raft_consensus.cc:359] T 9155962ca7f34a8c83a1780ce0018a59 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.455628 12823 raft_consensus.cc:385] T 9155962ca7f34a8c83a1780ce0018a59 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:34.455894 12823 raft_consensus.cc:740] T 9155962ca7f34a8c83a1780ce0018a59 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:34.458678 13244 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.459208 13244 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.462028 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:34.462738 12633 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:34.466336 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:34.467818 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:17:34.469081 12597 leader_election.cc:343] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:34.471067 12825 raft_consensus.cc:359] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.477715 12825 raft_consensus.cc:385] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:34.477792 12673 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:34.478116 12825 raft_consensus.cc:740] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:34.471140 12823 consensus_queue.cc:260] T 9155962ca7f34a8c83a1780ce0018a59 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.469141 13249 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.461410 13244 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:34.475036 13251 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:34.479466 12825 consensus_queue.cc:260] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.484743 12825 ts_tablet_manager.cc:1434] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.051s	user 0.004s	sys 0.003s
I20260709 13:17:34.480010 13249 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.480922 13136 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:34.492677 12823 ts_tablet_manager.cc:1434] T 9155962ca7f34a8c83a1780ce0018a59 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.001s
I20260709 13:17:34.481755 12703 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:34.496752 12824 tablet_bootstrap.cc:492] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:34.497294 12824 ts_tablet_manager.cc:1403] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.299s	user 0.017s	sys 0.004s
I20260709 13:17:34.499317 12598 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:34.505548 12823 tablet_bootstrap.cc:492] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:34.506699 13117 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:34.507097 13117 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:34.507412 13117 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:34.513833 12776 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:34.491536 13136 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:34.558233 13136 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:34.560808 12825 tablet_bootstrap.cc:492] T eb7fca96394f4ba9b4fe3d7a6e278878 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:34.641431 12824 raft_consensus.cc:359] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.642205 12824 raft_consensus.cc:385] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:34.642498 12824 raft_consensus.cc:740] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:34.643077 12824 consensus_queue.cc:260] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.644148 12823 tablet_bootstrap.cc:654] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:34.651789 13117 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.653884 13117 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:34.654323 12824 ts_tablet_manager.cc:1434] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.157s	user 0.021s	sys 0.015s
I20260709 13:17:34.655398 12824 tablet_bootstrap.cc:492] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e: Bootstrap starting.
I20260709 13:17:34.656111 12825 tablet_bootstrap.cc:654] T eb7fca96394f4ba9b4fe3d7a6e278878 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:34.656710 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:34.674823 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:34.678784 13249 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:34.680536 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:34.687664 13274 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:34.706683 13136 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.764459 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:34.769904 12782 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:34.776444 13229 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:34.777904 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:34.778610 12706 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:34.754550 13234 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.034s
W20260709 13:17:34.794430 12597 leader_election.cc:343] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:34.804279 12706 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:17:34.820290 13248 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:34.823570 12825 tablet_bootstrap.cc:492] T eb7fca96394f4ba9b4fe3d7a6e278878 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:34.824074 12825 ts_tablet_manager.cc:1403] T eb7fca96394f4ba9b4fe3d7a6e278878 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.309s	user 0.022s	sys 0.020s
I20260709 13:17:34.831421 12823 tablet_bootstrap.cc:492] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:34.831976 12823 ts_tablet_manager.cc:1403] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.327s	user 0.029s	sys 0.026s
I20260709 13:17:34.840021 13244 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.843186 13244 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.854024 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:34.854640 12630 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:34.855748 12825 raft_consensus.cc:359] T eb7fca96394f4ba9b4fe3d7a6e278878 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.856469 12825 raft_consensus.cc:385] T eb7fca96394f4ba9b4fe3d7a6e278878 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:34.856740 12825 raft_consensus.cc:740] T eb7fca96394f4ba9b4fe3d7a6e278878 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:34.857612 12825 consensus_queue.cc:260] T eb7fca96394f4ba9b4fe3d7a6e278878 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.861598 12825 ts_tablet_manager.cc:1434] T eb7fca96394f4ba9b4fe3d7a6e278878 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.000s
I20260709 13:17:34.863716 12825 tablet_bootstrap.cc:492] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:34.854759 13244 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:34.871501 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:17:34.874475 12597 leader_election.cc:343] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:34.875746 12630 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:34.877099 12673 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:34.877992 13229 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:34.879513 13229 raft_consensus.cc:697] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:34.880716 13229 consensus_queue.cc:237] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:17:34.886994 12579 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:34.882565 13136 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:34.907541 12824 tablet_bootstrap.cc:654] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:34.909140 12823 raft_consensus.cc:359] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.910006 12823 raft_consensus.cc:385] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:34.910274 12823 raft_consensus.cc:740] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:34.911067 12823 consensus_queue.cc:260] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.912436 13234 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:34.914229 12823 ts_tablet_manager.cc:1434] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.082s	user 0.008s	sys 0.000s
I20260709 13:17:34.924238 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:34.921867 12823 tablet_bootstrap.cc:492] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:34.932956 12598 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:34.934810 13248 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:17:34.934999 13117 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:34.935349 13248 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.936476 13117 raft_consensus.cc:697] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:34.937252 13117 consensus_queue.cc:237] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.939668 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:34.942132 13216 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:34.953426 12823 tablet_bootstrap.cc:654] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:34.948666 13216 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:34.959908 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:34.963027 12530 catalog_manager.cc:5697] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:34.942606 13248 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:34.961462 13216 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:17:34.991783 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 57508.2us
I20260709 13:17:34.995879 12825 tablet_bootstrap.cc:654] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:35.004448 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:35.005187 12626 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:35.020730 12673 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:35.024507 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:35.026816 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:17:35.028820 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 48216.8us
W20260709 13:17:35.029755 12672 consensus_peers.cc:597] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:35.029139 13135 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:35.035001 13248 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:35.041241 13135 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:35.065341 13135 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:35.084069 12824 tablet_bootstrap.cc:492] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e: No bootstrap required, opened a new log
I20260709 13:17:35.085613 12824 ts_tablet_manager.cc:1403] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e: Time spent bootstrapping tablet: real 0.430s	user 0.011s	sys 0.013s
I20260709 13:17:35.109717 12825 tablet_bootstrap.cc:492] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:35.110347 12825 ts_tablet_manager.cc:1403] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.247s	user 0.017s	sys 0.013s
I20260709 13:17:35.129218 12528 catalog_manager.cc:5697] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 0 to 1, leader changed from <none> to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:35.148947 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 82711.8us
W20260709 13:17:35.149739 12672 leader_election.cc:343] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:35.152575 12825 raft_consensus.cc:359] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.153295 12825 raft_consensus.cc:385] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:35.153855 12825 raft_consensus.cc:740] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:35.156440 12825 consensus_queue.cc:260] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.164575 12825 ts_tablet_manager.cc:1434] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.054s	user 0.005s	sys 0.002s
I20260709 13:17:35.165572 12825 tablet_bootstrap.cc:492] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:35.166026 12824 raft_consensus.cc:359] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.166679 12824 raft_consensus.cc:385] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:35.166911 12824 raft_consensus.cc:740] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Initialized, Role: FOLLOWER
I20260709 13:17:35.167482 12824 consensus_queue.cc:260] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.170382 13135 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.172575 13135 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:35.173543 12781 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:35.175024 12824 ts_tablet_manager.cc:1434] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e: Time spent starting tablet: real 0.089s	user 0.007s	sys 0.000s
I20260709 13:17:35.181869 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:35.182873 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:35.183514 12627 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:35.220009 12598 leader_election.cc:343] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:35.220497 12598 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:17:35.230911 12672 leader_election.cc:343] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:35.248610 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:35.266682 12823 tablet_bootstrap.cc:492] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:35.267279 12823 ts_tablet_manager.cc:1403] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.346s	user 0.013s	sys 0.024s
I20260709 13:17:35.267966 12627 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:35.233126 13288 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:35.270923 12673 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:17:35.273869 12672 consensus_peers.cc:597] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:35.276422 12672 leader_election.cc:343] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:35.272986 13305 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:35.278471 12823 raft_consensus.cc:359] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.280392 12823 raft_consensus.cc:385] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:35.280720 12823 raft_consensus.cc:740] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:35.281178 12823 consensus_queue.cc:260] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.283248 13305 raft_consensus.cc:697] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:35.283828 12825 tablet_bootstrap.cc:654] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:35.284112 13305 consensus_queue.cc:237] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.293723 12823 ts_tablet_manager.cc:1434] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:17:35.294868 12778 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:35.295004 12823 tablet_bootstrap.cc:492] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530: Bootstrap starting.
I20260709 13:17:35.307616 13303 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:35.319900 12530 catalog_manager.cc:5697] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:35.325942 12825 tablet_bootstrap.cc:492] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:35.326423 12825 ts_tablet_manager.cc:1403] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.161s	user 0.016s	sys 0.001s
I20260709 13:17:35.330123 12823 tablet_bootstrap.cc:654] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:35.334512 12825 raft_consensus.cc:359] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.335217 12825 raft_consensus.cc:385] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:35.335498 12825 raft_consensus.cc:740] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
W20260709 13:17:35.351194 12597 consensus_peers.cc:597] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:35.353677 12825 consensus_queue.cc:260] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.355432 12825 ts_tablet_manager.cc:1434] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.000s
I20260709 13:17:35.356592 12825 tablet_bootstrap.cc:492] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:35.360276 13305 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:35.368587 12825 tablet_bootstrap.cc:654] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:35.372313 12672 consensus_peers.cc:597] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:35.381969 12823 tablet_bootstrap.cc:492] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530: No bootstrap required, opened a new log
I20260709 13:17:35.382457 12823 ts_tablet_manager.cc:1403] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530: Time spent bootstrapping tablet: real 0.088s	user 0.018s	sys 0.000s
I20260709 13:17:35.385099 12823 raft_consensus.cc:359] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.385857 12823 raft_consensus.cc:385] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:35.386132 12823 raft_consensus.cc:740] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Initialized, Role: FOLLOWER
I20260709 13:17:35.387245 12823 consensus_queue.cc:260] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.389715 12823 ts_tablet_manager.cc:1434] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:17:35.414621 13303 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.415123 13303 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.416503 12825 tablet_bootstrap.cc:492] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:35.416985 12825 ts_tablet_manager.cc:1403] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.061s	user 0.014s	sys 0.021s
I20260709 13:17:35.417966 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:35.418615 12633 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
I20260709 13:17:35.419793 12673 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:35.420554 13290 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:35.420527 12825 raft_consensus.cc:359] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.421077 13290 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:35.421306 12825 raft_consensus.cc:385] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:35.421453 13290 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:35.421672 12825 raft_consensus.cc:740] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:35.422458 12825 consensus_queue.cc:260] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.424950 12825 ts_tablet_manager.cc:1434] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:17:35.425932 12825 tablet_bootstrap.cc:492] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:35.426467 13303 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:35.432245 12825 tablet_bootstrap.cc:654] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:35.432674 13290 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.433065 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:35.438591 13290 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:35.437832 12781 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:35.439632 12672 leader_election.cc:343] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:35.440331 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:17:35.444079 12672 leader_election.cc:343] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:35.444314 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:35.444867 12633 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:35.448350 12825 tablet_bootstrap.cc:492] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:35.448849 12825 ts_tablet_manager.cc:1403] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.002s
I20260709 13:17:35.451548 12825 raft_consensus.cc:359] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.452207 12825 raft_consensus.cc:385] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:35.452505 12825 raft_consensus.cc:740] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:35.453116 12825 consensus_queue.cc:260] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.455685 12825 ts_tablet_manager.cc:1434] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:17:35.456583 12825 tablet_bootstrap.cc:492] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:35.467087 12825 tablet_bootstrap.cc:654] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:35.479642 13248 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.480155 13248 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.482697 13248 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:35.486239 12633 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:35.488255 12673 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:35.488777 12825 tablet_bootstrap.cc:492] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:35.489378 13290 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:35.489602 12825 ts_tablet_manager.cc:1403] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.033s	user 0.004s	sys 0.011s
I20260709 13:17:35.490543 13290 raft_consensus.cc:697] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:35.491384 13290 consensus_queue.cc:237] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.492724 12825 raft_consensus.cc:359] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.493517 12825 raft_consensus.cc:385] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:35.493872 12825 raft_consensus.cc:740] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:35.494333 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:35.494748 12825 consensus_queue.cc:260] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.496136 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:35.498948 12747 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:35.499540 12825 ts_tablet_manager.cc:1434] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:17:35.500103 13248 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:35.503190 12825 tablet_bootstrap.cc:492] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:35.510751 12825 tablet_bootstrap.cc:654] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:35.515370 13290 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.515892 13290 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.516079 12528 catalog_manager.cc:5697] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:35.519254 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:35.519490 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:35.520241 12633 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 0.
W20260709 13:17:35.520515 12672 leader_election.cc:343] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:35.521256 13290 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:35.521744 12673 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:35.523859 13290 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:35.524250 13290 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:35.528395 13290 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:35.531936 12825 tablet_bootstrap.cc:492] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:35.532433 12825 ts_tablet_manager.cc:1403] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.010s
I20260709 13:17:35.535362 12825 raft_consensus.cc:359] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.535985 12825 raft_consensus.cc:385] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:35.536239 12825 raft_consensus.cc:740] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:35.536971 12825 consensus_queue.cc:260] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.542960 12825 ts_tablet_manager.cc:1434] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:17:35.543895 12825 tablet_bootstrap.cc:492] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:35.550552 12825 tablet_bootstrap.cc:654] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:35.557868 13149 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:35.572069 12825 tablet_bootstrap.cc:492] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:35.573889 12825 ts_tablet_manager.cc:1403] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.030s	user 0.024s	sys 0.000s
I20260709 13:17:35.575739 13290 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.579015 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:35.579715 12633 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:35.584450 12825 raft_consensus.cc:359] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.585191 12825 raft_consensus.cc:385] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:35.585462 12825 raft_consensus.cc:740] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:35.586133 12825 consensus_queue.cc:260] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.588160 12825 ts_tablet_manager.cc:1434] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.000s
I20260709 13:17:35.589040 12825 tablet_bootstrap.cc:492] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:35.593889 12777 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:35.603617 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:35.604192 12825 tablet_bootstrap.cc:654] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:35.605012 12672 leader_election.cc:343] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:35.605227 13290 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:35.617657 12633 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:17:35.628958 12825 tablet_bootstrap.cc:492] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:35.629477 12825 ts_tablet_manager.cc:1403] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.041s	user 0.014s	sys 0.002s
I20260709 13:17:35.630501 12673 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:35.631304 13290 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:35.631847 13290 raft_consensus.cc:697] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:35.634073 12825 raft_consensus.cc:359] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.636284 12825 raft_consensus.cc:385] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:35.635780 13290 consensus_queue.cc:237] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.636562 12825 raft_consensus.cc:740] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:35.637223 12825 consensus_queue.cc:260] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.639261 12825 ts_tablet_manager.cc:1434] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:17:35.640502 12825 tablet_bootstrap.cc:492] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2: Bootstrap starting.
I20260709 13:17:35.647305 13149 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.647825 13149 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.652443 12825 tablet_bootstrap.cc:654] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:35.652781 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:17:35.653451 12781 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:35.654634 12597 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:17:35.655145 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:35.655350 13149 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:35.655539 13117 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:35.655807 12704 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 0.
I20260709 13:17:35.656093 13117 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:35.656600 13117 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:35.679663 12825 tablet_bootstrap.cc:492] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2: No bootstrap required, opened a new log
I20260709 13:17:35.680161 12825 ts_tablet_manager.cc:1403] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2: Time spent bootstrapping tablet: real 0.040s	user 0.012s	sys 0.010s
I20260709 13:17:35.682602 12825 raft_consensus.cc:359] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.683249 12825 raft_consensus.cc:385] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:35.683537 12825 raft_consensus.cc:740] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Initialized, Role: FOLLOWER
I20260709 13:17:35.684161 12825 consensus_queue.cc:260] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.693984 12825 ts_tablet_manager.cc:1434] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:17:35.697535 13249 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.699067 12777 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:35.701995 13117 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.703778 13117 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:35.704181 13249 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.704465 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:35.704785 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:35.705163 12704 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:35.705389 12781 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:35.713519 12781 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:17:35.715057 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:35.715862 12627 raft_consensus.cc:2393] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 1: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:17:35.717067 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:35.717711 13249 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:35.721412 12528 catalog_manager.cc:5697] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:35.718532 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:35.729830 12704 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:17:35.730679 13248 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:35.721591 12597 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:17:35.728771 13290 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:35.734724 13128 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:35.735275 13128 raft_consensus.cc:697] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:35.736150 13128 consensus_queue.cc:237] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.737238 13248 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:35.746258 13290 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:35.772945 12529 catalog_manager.cc:5697] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 0 to 1, leader changed from <none> to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:35.826722 13328 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:35.827230 13328 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:35.831521 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:17:35.832336 12626 raft_consensus.cc:2393] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 1: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:17:35.833586 13328 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:35.836009 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:17:35.837198 12777 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:35.837858 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:17:35.838771 13328 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:35.845865 13328 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:35.848886 12781 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:36.009141 13290 consensus_queue.cc:1048] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:36.035771 13305 consensus_queue.cc:1048] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:36.100926 13305 consensus_queue.cc:1048] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:36.117734 13296 consensus_queue.cc:1048] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:36.288910 13288 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:36.289839 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260709 13:17:36.323653 13117 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:17:36.341646 13339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:36.343902 13338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:36.344982 13342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:36.345685 11517 server_base.cc:1061] running on GCE node
I20260709 13:17:36.347440 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:36.347704 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:36.347918 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603056347902 us; error 0 us; skew 500 ppm
I20260709 13:17:36.348534 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:36.354444 11517 webserver.cc:533] Webserver started at http://127.11.63.68:33643/ using document root <none> and password file <none>
I20260709 13:17:36.355067 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:36.355309 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:36.355710 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:36.357156 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-3-root/instance:
uuid: "522fd2ba0e9f402691b5ef29af8579cd"
format_stamp: "Formatted at 2026-07-09 13:17:36 on dist-test-slave-qv7r"
I20260709 13:17:36.370820 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.012s	user 0.009s	sys 0.001s
I20260709 13:17:36.376021 13347 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:36.382292 11517 fs_manager.cc:730] Time spent opening block manager: real 0.008s	user 0.001s	sys 0.000s
I20260709 13:17:36.382730 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-3-root
uuid: "522fd2ba0e9f402691b5ef29af8579cd"
format_stamp: "Formatted at 2026-07-09 13:17:36 on dist-test-slave-qv7r"
I20260709 13:17:36.383153 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:36.422667 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:36.424284 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:36.426124 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:36.429348 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:36.429536 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:36.429893 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:36.430107 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:36.623688 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.68:36021
I20260709 13:17:36.627861 13410 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.68:36021 every 8 connection(s)
I20260709 13:17:36.638121 11517 auto_leader_rebalancer-test.cc:384] add a tserver: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:36.654031 13411 heartbeater.cc:344] Connected to a master server at 127.11.63.126:38321
I20260709 13:17:36.654505 13411 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:36.655364 13411 heartbeater.cc:507] Master 127.11.63.126:38321 requested a full tablet report, sending...
I20260709 13:17:36.657541 12528 ts_manager.cc:194] Registered new tserver with Master: 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:17:36.659559 12528 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40528
I20260709 13:17:37.082504 12626 consensus_queue.cc:237] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:37.091897 12704 raft_consensus.cc:1275] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:37.093914 13128 consensus_queue.cc:1048] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:37.104583 13288 raft_consensus.cc:2955] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:37.106246 12781 raft_consensus.cc:1275] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:37.106195 12707 raft_consensus.cc:2955] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:37.109741 13117 consensus_queue.cc:1048] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:37.133100 12781 raft_consensus.cc:2955] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:37.136152 12528 catalog_manager.cc:5697] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:37.170270 12597 consensus_peers.cc:597] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d46747dede234934bea0f0c324106c28. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:37.184655 12704 consensus_queue.cc:237] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:37.208035 12633 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:37.209488 12781 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:37.209733 13296 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:37.214350 13229 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:37.225876 13296 raft_consensus.cc:2955] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:37.229182 12633 raft_consensus.cc:2955] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
W20260709 13:17:37.239903 12672 consensus_peers.cc:597] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1fa805ee07a7429abe6b520bbfb94cd9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:37.248845 12781 raft_consensus.cc:2955] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:37.253623 12707 consensus_queue.cc:237] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:37.271312 12529 catalog_manager.cc:5697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:37.296186 12777 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:17:37.297506 12672 consensus_peers.cc:597] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 059429c0c4024dfb941d1ff325fe8f98. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:37.299129 13305 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:37.309237 12627 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:37.311112 13296 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:37.341761 13229 raft_consensus.cc:2955] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:37.345118 12633 raft_consensus.cc:2955] T 059429c0c4024dfb941d1ff325fe8f98 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:37.348855 12781 raft_consensus.cc:2955] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:37.355026 12707 consensus_queue.cc:237] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:37.368383 12528 catalog_manager.cc:5697] T 059429c0c4024dfb941d1ff325fe8f98 P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:37.378213 12781 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:17:37.379123 12672 consensus_peers.cc:597] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 15d86ce6de4b43719dc715ab6f4de742. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:37.388906 13428 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:37.393250 12633 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:37.398408 13305 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:37.406263 13428 raft_consensus.cc:2955] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:37.420281 12781 raft_consensus.cc:2955] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:37.435895 12529 catalog_manager.cc:5697] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:37.442174 12627 raft_consensus.cc:2955] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:37.662555 13411 heartbeater.cc:499] Master 127.11.63.126:38321 was elected leader, sending a full tablet report...
I20260709 13:17:37.746793 13440 ts_tablet_manager.cc:933] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:17:37.751143 13439 ts_tablet_manager.cc:933] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:37.802063 13440 tablet_copy_client.cc:323] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.65:43603
I20260709 13:17:37.806442 13442 ts_tablet_manager.cc:933] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:37.819445 13439 tablet_copy_client.cc:323] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.66:45173
I20260709 13:17:37.824460 13442 tablet_copy_client.cc:323] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.66:45173
I20260709 13:17:37.858662 12644 tablet_copy_service.cc:140] P 19cd864c053744bebd96093a7982537e: Received BeginTabletCopySession request for tablet d46747dede234934bea0f0c324106c28 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:39590)
I20260709 13:17:37.859304 12644 tablet_copy_service.cc:161] P 19cd864c053744bebd96093a7982537e: Beginning new tablet copy session on tablet d46747dede234934bea0f0c324106c28 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:39590: session id = 522fd2ba0e9f402691b5ef29af8579cd-d46747dede234934bea0f0c324106c28
I20260709 13:17:37.878867 12719 tablet_copy_service.cc:140] P 639ab2db265f4747bb7afb93e833f530: Received BeginTabletCopySession request for tablet 1fa805ee07a7429abe6b520bbfb94cd9 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:38496)
I20260709 13:17:37.879536 12719 tablet_copy_service.cc:161] P 639ab2db265f4747bb7afb93e833f530: Beginning new tablet copy session on tablet 1fa805ee07a7429abe6b520bbfb94cd9 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:38496: session id = 522fd2ba0e9f402691b5ef29af8579cd-1fa805ee07a7429abe6b520bbfb94cd9
I20260709 13:17:37.887130 12719 tablet_copy_source_session.cc:215] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:37.889746 12718 tablet_copy_service.cc:140] P 639ab2db265f4747bb7afb93e833f530: Received BeginTabletCopySession request for tablet 059429c0c4024dfb941d1ff325fe8f98 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:38496)
I20260709 13:17:37.890362 12718 tablet_copy_service.cc:161] P 639ab2db265f4747bb7afb93e833f530: Beginning new tablet copy session on tablet 059429c0c4024dfb941d1ff325fe8f98 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:38496: session id = 522fd2ba0e9f402691b5ef29af8579cd-059429c0c4024dfb941d1ff325fe8f98
I20260709 13:17:37.898172 12718 tablet_copy_source_session.cc:215] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:37.903033 13439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fa805ee07a7429abe6b520bbfb94cd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.906404 12644 tablet_copy_source_session.cc:215] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:37.912017 13440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d46747dede234934bea0f0c324106c28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.918813 13442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 059429c0c4024dfb941d1ff325fe8f98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.944479 13442 tablet_copy_client.cc:806] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:37.945232 13442 tablet_copy_client.cc:670] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:37.947427 13440 tablet_copy_client.cc:806] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:37.948060 13440 tablet_copy_client.cc:670] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:37.948575 13439 tablet_copy_client.cc:806] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:37.949169 13439 tablet_copy_client.cc:670] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:37.952360 13442 tablet_copy_client.cc:538] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:37.964957 13440 tablet_copy_client.cc:538] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:37.973649 13439 tablet_copy_client.cc:538] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:37.981137 13442 tablet_bootstrap.cc:492] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:38.004189 13439 tablet_bootstrap.cc:492] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:38.010207 13440 tablet_bootstrap.cc:492] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:38.036096 13450 ts_tablet_manager.cc:933] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:38.065233 13450 tablet_copy_client.cc:323] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.66:45173
I20260709 13:17:38.068650 12718 tablet_copy_service.cc:140] P 639ab2db265f4747bb7afb93e833f530: Received BeginTabletCopySession request for tablet 15d86ce6de4b43719dc715ab6f4de742 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:38496)
I20260709 13:17:38.069372 12718 tablet_copy_service.cc:161] P 639ab2db265f4747bb7afb93e833f530: Beginning new tablet copy session on tablet 15d86ce6de4b43719dc715ab6f4de742 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:38496: session id = 522fd2ba0e9f402691b5ef29af8579cd-15d86ce6de4b43719dc715ab6f4de742
I20260709 13:17:38.098065 13442 tablet_bootstrap.cc:492] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:38.101467 13442 tablet_bootstrap.cc:492] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:17:38.107980 13442 ts_tablet_manager.cc:1403] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.127s	user 0.025s	sys 0.002s
I20260709 13:17:38.129972 12718 tablet_copy_source_session.cc:215] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:38.135797 13450 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15d86ce6de4b43719dc715ab6f4de742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.147917 13442 raft_consensus.cc:359] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:38.148937 13442 raft_consensus.cc:740] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:17:38.149969 13439 tablet_bootstrap.cc:492] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:38.159063 13440 tablet_bootstrap.cc:492] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:38.159957 13440 tablet_bootstrap.cc:492] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:17:38.160638 13440 ts_tablet_manager.cc:1403] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.151s	user 0.020s	sys 0.012s
I20260709 13:17:38.161770 13439 tablet_bootstrap.cc:492] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:17:38.149705 13442 consensus_queue.cc:260] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:38.162532 13439 ts_tablet_manager.cc:1403] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.159s	user 0.027s	sys 0.008s
I20260709 13:17:38.173863 13442 ts_tablet_manager.cc:1434] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.064s	user 0.016s	sys 0.001s
I20260709 13:17:38.176386 12718 tablet_copy_service.cc:342] P 639ab2db265f4747bb7afb93e833f530: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-059429c0c4024dfb941d1ff325fe8f98 received from {username='slave'} at 127.0.0.1:38496
I20260709 13:17:38.176932 12718 tablet_copy_service.cc:434] P 639ab2db265f4747bb7afb93e833f530: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-059429c0c4024dfb941d1ff325fe8f98 on tablet 059429c0c4024dfb941d1ff325fe8f98 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:38.182191 13440 raft_consensus.cc:359] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:38.190641 13440 raft_consensus.cc:740] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:17:38.191608 13440 consensus_queue.cc:260] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:38.182258 13439 raft_consensus.cc:359] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:38.193508 13439 raft_consensus.cc:740] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:17:38.194298 13439 consensus_queue.cc:260] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:38.196116 13439 ts_tablet_manager.cc:1434] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 13:17:38.198149 12718 tablet_copy_service.cc:342] P 639ab2db265f4747bb7afb93e833f530: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-1fa805ee07a7429abe6b520bbfb94cd9 received from {username='slave'} at 127.0.0.1:38496
I20260709 13:17:38.198570 12718 tablet_copy_service.cc:434] P 639ab2db265f4747bb7afb93e833f530: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-1fa805ee07a7429abe6b520bbfb94cd9 on tablet 1fa805ee07a7429abe6b520bbfb94cd9 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:38.200135 13440 ts_tablet_manager.cc:1434] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.039s	user 0.009s	sys 0.002s
I20260709 13:17:38.201951 12644 tablet_copy_service.cc:342] P 19cd864c053744bebd96093a7982537e: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-d46747dede234934bea0f0c324106c28 received from {username='slave'} at 127.0.0.1:39590
I20260709 13:17:38.202399 12644 tablet_copy_service.cc:434] P 19cd864c053744bebd96093a7982537e: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-d46747dede234934bea0f0c324106c28 on tablet d46747dede234934bea0f0c324106c28 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:38.204084 13450 tablet_copy_client.cc:806] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:38.204718 13450 tablet_copy_client.cc:670] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:38.211404 13450 tablet_copy_client.cc:538] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:38.220835 13450 tablet_bootstrap.cc:492] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:38.252197 13450 tablet_bootstrap.cc:492] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:38.253134 13450 tablet_bootstrap.cc:492] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:17:38.253859 13450 ts_tablet_manager.cc:1403] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.033s	user 0.018s	sys 0.012s
I20260709 13:17:38.256893 13450 raft_consensus.cc:359] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:38.257755 13450 raft_consensus.cc:740] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:17:38.258436 13450 consensus_queue.cc:260] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:38.264014 13385 raft_consensus.cc:1217] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:38.273871 13450 ts_tablet_manager.cc:1434] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:17:38.283103 12718 tablet_copy_service.cc:342] P 639ab2db265f4747bb7afb93e833f530: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-15d86ce6de4b43719dc715ab6f4de742 received from {username='slave'} at 127.0.0.1:38496
I20260709 13:17:38.283638 12718 tablet_copy_service.cc:434] P 639ab2db265f4747bb7afb93e833f530: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-15d86ce6de4b43719dc715ab6f4de742 on tablet 15d86ce6de4b43719dc715ab6f4de742 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:38.355069 13385 raft_consensus.cc:1217] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:38.428315 13385 raft_consensus.cc:1217] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:38.552835 13385 raft_consensus.cc:1217] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:38.817947 13455 raft_consensus.cc:1064] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:17:38.840512 13455 consensus_queue.cc:237] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:38.847739 13117 raft_consensus.cc:1064] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:17:38.852602 13117 consensus_queue.cc:237] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:38.880877 12627 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:38.884020 13385 raft_consensus.cc:1275] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:38.884285 12778 raft_consensus.cc:1275] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:38.887033 13460 consensus_queue.cc:1048] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:38.882930 13455 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 13:17:38.893651 13460 consensus_queue.cc:1048] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:38.898533 13384 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:38.899471 12781 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:38.905882 12707 raft_consensus.cc:1275] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:38.914600 13455 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:38.937979 13455 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:38.948589 13460 consensus_queue.cc:1048] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:39.000398 13117 raft_consensus.cc:2955] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.007704 12781 raft_consensus.cc:2955] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.013335 13229 raft_consensus.cc:1064] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:17:39.016428 13229 consensus_queue.cc:237] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:39.026331 13455 raft_consensus.cc:2955] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.033334 12529 catalog_manager.cc:5697] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.048841 13383 raft_consensus.cc:2955] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.086874 13385 raft_consensus.cc:2955] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.096817 12626 raft_consensus.cc:2955] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.107306 12781 raft_consensus.cc:2955] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.128321 12631 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:39.139322 12777 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:39.134779 12528 catalog_manager.cc:5697] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:39.148837 13470 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:17:39.150810 13455 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:39.154042 13384 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:39.154101 12702 raft_consensus.cc:2955] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.238758 13473 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:39.240037 13428 raft_consensus.cc:1064] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:17:39.253165 13428 consensus_queue.cc:237] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:39.287938 13473 raft_consensus.cc:2955] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.296856 12626 raft_consensus.cc:2955] T 059429c0c4024dfb941d1ff325fe8f98 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.300732 12633 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:39.302585 13428 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:39.322418 13385 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:39.323202 12776 raft_consensus.cc:2955] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.331284 12781 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:39.354228 13428 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:39.363229 12626 consensus_queue.cc:237] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:39.369786 13385 raft_consensus.cc:2955] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.384455 13473 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:17:39.415009 12777 raft_consensus.cc:1275] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:39.420364 13384 raft_consensus.cc:1275] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:39.430399 13128 consensus_queue.cc:1048] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:39.438390 13288 consensus_queue.cc:1048] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:39.443933 12707 consensus_queue.cc:237] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:39.450462 13117 raft_consensus.cc:2955] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.453476 13385 raft_consensus.cc:2955] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.456800 12783 raft_consensus.cc:2955] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.474574 13384 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:39.474732 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d46747dede234934bea0f0c324106c28 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:39.476428 12633 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:39.477880 13455 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:39.479610 13428 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:39.496742 12530 catalog_manager.cc:5697] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.515084 13296 raft_consensus.cc:2955] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.520303 12781 raft_consensus.cc:2955] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.536741 12633 raft_consensus.cc:2955] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.538918 12529 catalog_manager.cc:5697] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.569239 12529 catalog_manager.cc:5697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:17:39.577618 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 62492.5us
I20260709 13:17:39.578950 13471 raft_consensus.cc:2955] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.579457 13385 raft_consensus.cc:2955] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.589756 12626 raft_consensus.cc:2955] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.607235 12528 catalog_manager.cc:5697] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:39.622022 12695 tablet_service.cc:1558] Processing DeleteTablet for tablet d46747dede234934bea0f0c324106c28 with delete_type TABLET_DATA_TOMBSTONED (TS 639ab2db265f4747bb7afb93e833f530 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42474
I20260709 13:17:39.623939 12707 consensus_queue.cc:237] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:39.625629 12704 consensus_queue.cc:237] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:39.627306 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 15d86ce6de4b43719dc715ab6f4de742 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:39.628657 13383 raft_consensus.cc:2955] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.649084 13382 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:39.650873 12633 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:39.653929 13384 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:39.655180 12781 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:39.661679 12773 tablet_service.cc:1558] Processing DeleteTablet for tablet 15d86ce6de4b43719dc715ab6f4de742 with delete_type TABLET_DATA_TOMBSTONED (TS 3d539f8d95d343d8b1696803f18b30d2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36472
I20260709 13:17:39.672678 13471 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:39.676136 13489 tablet_replica.cc:333] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:17:39.679651 13473 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:39.679747 13296 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:39.681761 13470 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:39.698319 13470 raft_consensus.cc:2955] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.699573 13488 tablet_replica.cc:333] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:17:39.702090 13488 raft_consensus.cc:2243] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:39.701937 13229 raft_consensus.cc:2955] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.704957 12781 raft_consensus.cc:2955] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.710711 13385 raft_consensus.cc:2955] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.717726 13489 raft_consensus.cc:2243] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:39.718506 13489 raft_consensus.cc:2272] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:39.718223 12633 raft_consensus.cc:2955] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.703114 13488 raft_consensus.cc:2272] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:39.722246 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1fa805ee07a7429abe6b520bbfb94cd9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:39.722947 13383 raft_consensus.cc:2955] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.728505 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 059429c0c4024dfb941d1ff325fe8f98 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:39.731134 12528 catalog_manager.cc:5697] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:39.734443 13489 ts_tablet_manager.cc:1916] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:39.735276 12529 catalog_manager.cc:5697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:39.747061 13488 ts_tablet_manager.cc:1916] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:39.765046 12624 tablet_service.cc:1558] Processing DeleteTablet for tablet 059429c0c4024dfb941d1ff325fe8f98 with delete_type TABLET_DATA_TOMBSTONED (TS 19cd864c053744bebd96093a7982537e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56830
I20260709 13:17:39.773875 13489 ts_tablet_manager.cc:1929] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:39.775527 13489 log.cc:1199] T 15d86ce6de4b43719dc715ab6f4de742 P 3d539f8d95d343d8b1696803f18b30d2: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-2-root/wals/15d86ce6de4b43719dc715ab6f4de742
I20260709 13:17:39.776827 12773 tablet_service.cc:1558] Processing DeleteTablet for tablet 1fa805ee07a7429abe6b520bbfb94cd9 with delete_type TABLET_DATA_TOMBSTONED (TS 3d539f8d95d343d8b1696803f18b30d2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36472
I20260709 13:17:39.789438 13489 tablet_replica.cc:333] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:17:39.790833 13489 raft_consensus.cc:2243] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:39.791153 13490 tablet_replica.cc:333] T 059429c0c4024dfb941d1ff325fe8f98 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:17:39.791649 13489 raft_consensus.cc:2272] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:39.786998 12516 catalog_manager.cc:5028] TS 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): tablet 15d86ce6de4b43719dc715ab6f4de742 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:17:39.794039 13490 raft_consensus.cc:2243] T 059429c0c4024dfb941d1ff325fe8f98 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:39.794853 13490 raft_consensus.cc:2272] T 059429c0c4024dfb941d1ff325fe8f98 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:39.803395 13490 ts_tablet_manager.cc:1916] T 059429c0c4024dfb941d1ff325fe8f98 P 19cd864c053744bebd96093a7982537e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:39.811729 13489 ts_tablet_manager.cc:1916] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:39.843199 13488 ts_tablet_manager.cc:1929] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:39.843729 13488 log.cc:1199] T d46747dede234934bea0f0c324106c28 P 639ab2db265f4747bb7afb93e833f530: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-1-root/wals/d46747dede234934bea0f0c324106c28
I20260709 13:17:39.852865 13490 ts_tablet_manager.cc:1929] T 059429c0c4024dfb941d1ff325fe8f98 P 19cd864c053744bebd96093a7982537e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:39.853358 13490 log.cc:1199] T 059429c0c4024dfb941d1ff325fe8f98 P 19cd864c053744bebd96093a7982537e: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-0-root/wals/059429c0c4024dfb941d1ff325fe8f98
I20260709 13:17:39.859313 13489 ts_tablet_manager.cc:1929] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:39.859881 13489 log.cc:1199] T 1fa805ee07a7429abe6b520bbfb94cd9 P 3d539f8d95d343d8b1696803f18b30d2: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-2-root/wals/1fa805ee07a7429abe6b520bbfb94cd9
I20260709 13:17:39.890990 12517 catalog_manager.cc:5028] TS 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): tablet 059429c0c4024dfb941d1ff325fe8f98 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:17:39.902321 12516 catalog_manager.cc:5028] TS 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): tablet 1fa805ee07a7429abe6b520bbfb94cd9 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:17:39.919474 12517 catalog_manager.cc:5028] TS 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): tablet d46747dede234934bea0f0c324106c28 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:17:40.900121 12709 consensus_queue.cc:237] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:40.909991 12783 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:17:40.912407 12672 consensus_peers.cc:597] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 58239dd301f64758846dd758e449e4d8. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:40.913825 13229 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:40.921334 12626 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:40.923468 13471 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:40.929438 13493 raft_consensus.cc:2955] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:40.931385 12781 raft_consensus.cc:2955] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:40.959581 12532 catalog_manager.cc:5697] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:40.965499 12626 raft_consensus.cc:2955] T 58239dd301f64758846dd758e449e4d8 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:40.997435 12781 consensus_queue.cc:237] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:41.023305 12709 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:41.023648 12626 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:41.025804 13486 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:41.030555 13487 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:17:41.047024 12746 consensus_peers.cc:597] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e79ffe63999346abaccd90ab600091d7. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:41.056475 13487 raft_consensus.cc:2955] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:41.058854 12626 raft_consensus.cc:2955] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:41.068243 12709 raft_consensus.cc:2955] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:41.074641 12707 consensus_queue.cc:237] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:41.093058 12530 catalog_manager.cc:5697] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:41.128427 12626 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:41.128610 12781 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:41.130147 13470 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:41.131400 13229 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:17:41.132560 12672 consensus_peers.cc:597] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b13b87a5cea4263937041d123c5889d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:41.139240 13229 raft_consensus.cc:2955] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:41.142614 12626 raft_consensus.cc:2955] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:41.152277 12781 raft_consensus.cc:2955] T 1b13b87a5cea4263937041d123c5889d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:41.154266 12633 consensus_queue.cc:237] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:41.159588 12530 catalog_manager.cc:5697] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:41.181317 12597 consensus_peers.cc:597] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e1a0f7a1130449548d1659cd21760fc9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:41.182670 12781 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:41.184688 12707 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:41.189179 13288 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:41.191218 13511 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:41.205888 13484 raft_consensus.cc:2955] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:41.208871 12707 raft_consensus.cc:2955] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:41.208866 12781 raft_consensus.cc:2955] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:41.228796 12529 catalog_manager.cc:5697] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:41.398747 13515 ts_tablet_manager.cc:933] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:41.406868 13515 tablet_copy_client.cc:323] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.66:45173
I20260709 13:17:41.409193 12718 tablet_copy_service.cc:140] P 639ab2db265f4747bb7afb93e833f530: Received BeginTabletCopySession request for tablet 58239dd301f64758846dd758e449e4d8 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:38496)
I20260709 13:17:41.409833 12718 tablet_copy_service.cc:161] P 639ab2db265f4747bb7afb93e833f530: Beginning new tablet copy session on tablet 58239dd301f64758846dd758e449e4d8 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:38496: session id = 522fd2ba0e9f402691b5ef29af8579cd-58239dd301f64758846dd758e449e4d8
I20260709 13:17:41.418874 12718 tablet_copy_source_session.cc:215] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:41.431829 13515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58239dd301f64758846dd758e449e4d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.459748 13515 tablet_copy_client.cc:806] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:41.462996 13515 tablet_copy_client.cc:670] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:41.467940 13515 tablet_copy_client.cc:538] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:41.478860 13515 tablet_bootstrap.cc:492] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:41.503998 13515 tablet_bootstrap.cc:492] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:41.505194 13515 tablet_bootstrap.cc:492] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:17:41.506143 13515 ts_tablet_manager.cc:1403] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.028s	user 0.020s	sys 0.004s
I20260709 13:17:41.510231 13515 raft_consensus.cc:359] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:41.511402 13515 raft_consensus.cc:740] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:17:41.512136 13515 consensus_queue.cc:260] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:41.532387 13515 ts_tablet_manager.cc:1434] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.026s	user 0.010s	sys 0.011s
I20260709 13:17:41.534307 12718 tablet_copy_service.cc:342] P 639ab2db265f4747bb7afb93e833f530: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-58239dd301f64758846dd758e449e4d8 received from {username='slave'} at 127.0.0.1:38496
I20260709 13:17:41.534818 12718 tablet_copy_service.cc:434] P 639ab2db265f4747bb7afb93e833f530: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-58239dd301f64758846dd758e449e4d8 on tablet 58239dd301f64758846dd758e449e4d8 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:41.612268 13515 ts_tablet_manager.cc:933] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:41.615511 13515 tablet_copy_client.cc:323] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.67:41181
I20260709 13:17:41.664795 12793 tablet_copy_service.cc:140] P 3d539f8d95d343d8b1696803f18b30d2: Received BeginTabletCopySession request for tablet e79ffe63999346abaccd90ab600091d7 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:54346)
I20260709 13:17:41.665434 12793 tablet_copy_service.cc:161] P 3d539f8d95d343d8b1696803f18b30d2: Beginning new tablet copy session on tablet e79ffe63999346abaccd90ab600091d7 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:54346: session id = 522fd2ba0e9f402691b5ef29af8579cd-e79ffe63999346abaccd90ab600091d7
I20260709 13:17:41.674963 12793 tablet_copy_source_session.cc:215] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:41.678992 13515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e79ffe63999346abaccd90ab600091d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.703353 13515 tablet_copy_client.cc:806] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:41.704016 13515 tablet_copy_client.cc:670] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:41.713282 13515 tablet_copy_client.cc:538] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:41.725385 13522 ts_tablet_manager.cc:933] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:17:41.744148 13522 tablet_copy_client.cc:323] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.65:43603
I20260709 13:17:41.746735 12644 tablet_copy_service.cc:140] P 19cd864c053744bebd96093a7982537e: Received BeginTabletCopySession request for tablet e1a0f7a1130449548d1659cd21760fc9 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:39590)
I20260709 13:17:41.747363 12644 tablet_copy_service.cc:161] P 19cd864c053744bebd96093a7982537e: Beginning new tablet copy session on tablet e1a0f7a1130449548d1659cd21760fc9 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:39590: session id = 522fd2ba0e9f402691b5ef29af8579cd-e1a0f7a1130449548d1659cd21760fc9
I20260709 13:17:41.748387 13515 tablet_bootstrap.cc:492] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:41.780718 13515 tablet_bootstrap.cc:492] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:41.781855 13515 tablet_bootstrap.cc:492] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:17:41.783063 13515 ts_tablet_manager.cc:1403] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.018s
I20260709 13:17:41.785982 13515 raft_consensus.cc:359] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:41.786968 13515 raft_consensus.cc:740] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:17:41.787637 13515 consensus_queue.cc:260] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:41.795758 13515 ts_tablet_manager.cc:1434] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 13:17:41.800048 12793 tablet_copy_service.cc:342] P 3d539f8d95d343d8b1696803f18b30d2: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-e79ffe63999346abaccd90ab600091d7 received from {username='slave'} at 127.0.0.1:54346
I20260709 13:17:41.800606 12793 tablet_copy_service.cc:434] P 3d539f8d95d343d8b1696803f18b30d2: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-e79ffe63999346abaccd90ab600091d7 on tablet e79ffe63999346abaccd90ab600091d7 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:41.806525 13515 ts_tablet_manager.cc:933] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:41.807632 12644 tablet_copy_source_session.cc:215] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:41.811944 13522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1a0f7a1130449548d1659cd21760fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:41.866786 13522 tablet_copy_client.cc:806] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:41.867378 13522 tablet_copy_client.cc:670] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:41.890408 13522 tablet_copy_client.cc:538] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:41.903206 13522 tablet_bootstrap.cc:492] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:41.953666 13385 raft_consensus.cc:1217] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:41.957998 13515 tablet_copy_client.cc:323] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.66:45173
I20260709 13:17:41.976277 13522 tablet_bootstrap.cc:492] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:41.977469 13522 tablet_bootstrap.cc:492] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:17:41.978355 13522 ts_tablet_manager.cc:1403] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.075s	user 0.025s	sys 0.005s
I20260709 13:17:41.984323 13522 raft_consensus.cc:359] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:41.990139 13522 raft_consensus.cc:740] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:17:41.990937 13522 consensus_queue.cc:260] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:41.995700 12718 tablet_copy_service.cc:140] P 639ab2db265f4747bb7afb93e833f530: Received BeginTabletCopySession request for tablet 1b13b87a5cea4263937041d123c5889d from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:38496)
I20260709 13:17:41.996343 12718 tablet_copy_service.cc:161] P 639ab2db265f4747bb7afb93e833f530: Beginning new tablet copy session on tablet 1b13b87a5cea4263937041d123c5889d from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:38496: session id = 522fd2ba0e9f402691b5ef29af8579cd-1b13b87a5cea4263937041d123c5889d
I20260709 13:17:42.001056 13522 ts_tablet_manager.cc:1434] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.022s	user 0.003s	sys 0.004s
I20260709 13:17:42.007237 12718 tablet_copy_source_session.cc:215] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:42.009013 12644 tablet_copy_service.cc:342] P 19cd864c053744bebd96093a7982537e: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-e1a0f7a1130449548d1659cd21760fc9 received from {username='slave'} at 127.0.0.1:39590
I20260709 13:17:42.009472 12644 tablet_copy_service.cc:434] P 19cd864c053744bebd96093a7982537e: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-e1a0f7a1130449548d1659cd21760fc9 on tablet e1a0f7a1130449548d1659cd21760fc9 with peer 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:17:42.071398 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 53556.7us
I20260709 13:17:42.074453 13515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b13b87a5cea4263937041d123c5889d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.092587 13515 tablet_copy_client.cc:806] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:42.093122 13515 tablet_copy_client.cc:670] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:42.100343 13515 tablet_copy_client.cc:538] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:42.112229 13515 tablet_bootstrap.cc:492] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:42.139051 13515 tablet_bootstrap.cc:492] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:42.140026 13515 tablet_bootstrap.cc:492] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:17:42.140681 13515 ts_tablet_manager.cc:1403] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.029s	user 0.016s	sys 0.011s
I20260709 13:17:42.143185 13515 raft_consensus.cc:359] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:42.143993 13515 raft_consensus.cc:740] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:17:42.144608 13515 consensus_queue.cc:260] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:42.147014 13515 ts_tablet_manager.cc:1434] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:17:42.150591 12718 tablet_copy_service.cc:342] P 639ab2db265f4747bb7afb93e833f530: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-1b13b87a5cea4263937041d123c5889d received from {username='slave'} at 127.0.0.1:38496
I20260709 13:17:42.151043 12718 tablet_copy_service.cc:434] P 639ab2db265f4747bb7afb93e833f530: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-1b13b87a5cea4263937041d123c5889d on tablet 1b13b87a5cea4263937041d123c5889d with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:42.162901 13385 raft_consensus.cc:1217] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:42.177495 13385 raft_consensus.cc:1217] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 13:17:42.225888 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 48632.3us
I20260709 13:17:42.322793 13385 raft_consensus.cc:1217] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:42.390792 13528 raft_consensus.cc:1064] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:17:42.393153 13528 consensus_queue.cc:237] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:42.408377 13383 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:42.410228 12627 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:42.411222 13229 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:42.413542 12776 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:42.414893 13528 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:42.432117 13471 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:42.459614 13471 raft_consensus.cc:2955] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:42.471788 12627 raft_consensus.cc:2955] T 58239dd301f64758846dd758e449e4d8 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:42.484170 13383 raft_consensus.cc:2955] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:42.495394 12530 catalog_manager.cc:5697] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:42.501950 12776 raft_consensus.cc:2955] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:42.576669 13288 raft_consensus.cc:1064] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:17:42.579280 13288 consensus_queue.cc:237] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:42.588579 13532 raft_consensus.cc:1064] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:17:42.598035 13532 consensus_queue.cc:237] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:42.614092 12702 consensus_queue.cc:237] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:42.617578 13383 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:42.617710 12783 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:42.619555 13484 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:42.622176 13511 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:42.626272 12776 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:42.629271 13471 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:42.637950 13384 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:42.640607 12702 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:42.642023 12627 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:42.642179 13534 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:42.644838 13487 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:42.648216 13528 raft_consensus.cc:2955] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:42.649183 13487 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:42.652294 13383 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:42.654805 13470 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:42.660944 13382 raft_consensus.cc:2955] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:42.675143 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 58239dd301f64758846dd758e449e4d8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:42.678903 12707 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:42.680469 13511 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:42.681283 12783 raft_consensus.cc:2955] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:42.715054 12529 catalog_manager.cc:5697] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:42.730018 13493 raft_consensus.cc:1064] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:17:42.732635 13493 consensus_queue.cc:237] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:42.768893 13385 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:42.769743 12627 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:42.771874 13229 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:42.772244 12783 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:42.776171 13528 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:42.777648 13229 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:42.778545 13484 raft_consensus.cc:2955] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:42.785825 12702 raft_consensus.cc:2955] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:42.810590 13382 raft_consensus.cc:2955] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:42.825093 12709 raft_consensus.cc:2955] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:42.840133 12627 raft_consensus.cc:2955] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:42.821492 13534 raft_consensus.cc:2955] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:42.880144 12624 tablet_service.cc:1558] Processing DeleteTablet for tablet 58239dd301f64758846dd758e449e4d8 with delete_type TABLET_DATA_TOMBSTONED (TS 19cd864c053744bebd96093a7982537e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56830
I20260709 13:17:42.878091 12532 catalog_manager.cc:5697] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:42.899741 13470 raft_consensus.cc:2955] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:42.906399 13385 raft_consensus.cc:2955] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:42.909173 12778 raft_consensus.cc:2955] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:42.909788 12633 raft_consensus.cc:2955] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:42.922003 12781 raft_consensus.cc:2955] T 1b13b87a5cea4263937041d123c5889d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:42.902143 13557 tablet_replica.cc:333] T 58239dd301f64758846dd758e449e4d8 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:17:42.938370 12529 catalog_manager.cc:5697] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:42.939908 13557 raft_consensus.cc:2243] T 58239dd301f64758846dd758e449e4d8 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:42.943251 13557 raft_consensus.cc:2272] T 58239dd301f64758846dd758e449e4d8 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:42.948616 13557 ts_tablet_manager.cc:1916] T 58239dd301f64758846dd758e449e4d8 P 19cd864c053744bebd96093a7982537e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:42.953118 13385 raft_consensus.cc:2955] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:42.973438 12529 catalog_manager.cc:5697] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:42.982076 12778 consensus_queue.cc:237] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:42.996544 12709 consensus_queue.cc:237] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:43.002864 12633 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:43.003153 12626 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:43.004542 13505 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:43.007557 13382 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:43.012940 13229 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:17:43.014834 13528 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:43.024262 13534 raft_consensus.cc:2955] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:43.025324 12627 consensus_queue.cc:237] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:43.028752 12626 raft_consensus.cc:2955] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:43.030282 13382 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:43.031924 13550 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:43.035989 13385 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:43.040827 13532 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:43.040954 13550 raft_consensus.cc:2955] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:43.044375 13557 ts_tablet_manager.cc:1929] T 58239dd301f64758846dd758e449e4d8 P 19cd864c053744bebd96093a7982537e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:43.044843 13557 log.cc:1199] T 58239dd301f64758846dd758e449e4d8 P 19cd864c053744bebd96093a7982537e: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-0-root/wals/58239dd301f64758846dd758e449e4d8
I20260709 13:17:43.048817 12517 catalog_manager.cc:5028] TS 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): tablet 58239dd301f64758846dd758e449e4d8 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:17:43.050185 12781 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:43.047384 13384 raft_consensus.cc:2955] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:43.054397 13484 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:43.052147 13528 raft_consensus.cc:2955] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:43.066057 13385 raft_consensus.cc:2955] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:43.066272 12631 raft_consensus.cc:2955] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:43.076087 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e1a0f7a1130449548d1659cd21760fc9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:43.078145 13382 raft_consensus.cc:2955] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:43.074769 12781 raft_consensus.cc:2955] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:43.095372 12516 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e79ffe63999346abaccd90ab600091d7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:43.099059 12532 catalog_manager.cc:5697] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:43.119563 12528 catalog_manager.cc:5697] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:43.131346 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1b13b87a5cea4263937041d123c5889d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:43.144428 12530 catalog_manager.cc:5697] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:43.169175 12695 tablet_service.cc:1558] Processing DeleteTablet for tablet e79ffe63999346abaccd90ab600091d7 with delete_type TABLET_DATA_TOMBSTONED (TS 639ab2db265f4747bb7afb93e833f530 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42474
I20260709 13:17:43.230182 12692 tablet_service.cc:1558] Processing DeleteTablet for tablet e1a0f7a1130449548d1659cd21760fc9 with delete_type TABLET_DATA_TOMBSTONED (TS 639ab2db265f4747bb7afb93e833f530 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42474
I20260709 13:17:43.230867 13560 tablet_replica.cc:333] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:17:43.243171 12773 tablet_service.cc:1558] Processing DeleteTablet for tablet 1b13b87a5cea4263937041d123c5889d with delete_type TABLET_DATA_TOMBSTONED (TS 3d539f8d95d343d8b1696803f18b30d2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36472
I20260709 13:17:43.267787 13561 tablet_replica.cc:333] T 1b13b87a5cea4263937041d123c5889d P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:17:43.274583 13561 raft_consensus.cc:2243] T 1b13b87a5cea4263937041d123c5889d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:43.275358 13561 raft_consensus.cc:2272] T 1b13b87a5cea4263937041d123c5889d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:43.284014 13561 ts_tablet_manager.cc:1916] T 1b13b87a5cea4263937041d123c5889d P 3d539f8d95d343d8b1696803f18b30d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:43.285564 13560 raft_consensus.cc:2243] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:43.287320 13560 raft_consensus.cc:2272] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:43.290663 13560 ts_tablet_manager.cc:1916] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:43.307045 13561 ts_tablet_manager.cc:1929] T 1b13b87a5cea4263937041d123c5889d P 3d539f8d95d343d8b1696803f18b30d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:43.307660 13561 log.cc:1199] T 1b13b87a5cea4263937041d123c5889d P 3d539f8d95d343d8b1696803f18b30d2: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-2-root/wals/1b13b87a5cea4263937041d123c5889d
I20260709 13:17:43.308068 13560 ts_tablet_manager.cc:1929] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:43.308634 13560 log.cc:1199] T e79ffe63999346abaccd90ab600091d7 P 639ab2db265f4747bb7afb93e833f530: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-1-root/wals/e79ffe63999346abaccd90ab600091d7
I20260709 13:17:43.309511 12516 catalog_manager.cc:5028] TS 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): tablet 1b13b87a5cea4263937041d123c5889d (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:17:43.310846 12517 catalog_manager.cc:5028] TS 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): tablet e79ffe63999346abaccd90ab600091d7 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:17:43.310969 13560 tablet_replica.cc:333] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:17:43.313197 13560 raft_consensus.cc:2243] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:43.313952 13560 raft_consensus.cc:2272] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:43.319044 13560 ts_tablet_manager.cc:1916] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:43.335340 13560 ts_tablet_manager.cc:1929] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:43.335839 13560 log.cc:1199] T e1a0f7a1130449548d1659cd21760fc9 P 639ab2db265f4747bb7afb93e833f530: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-1-root/wals/e1a0f7a1130449548d1659cd21760fc9
I20260709 13:17:43.338167 12517 catalog_manager.cc:5028] TS 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): tablet e1a0f7a1130449548d1659cd21760fc9 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:17:44.283578 12778 consensus_queue.cc:237] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:44.290182 12702 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:17:44.290702 12746 consensus_peers.cc:597] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ab5ddded6fd34f278dade99f2086c4e6. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:44.291523 12631 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:44.291756 13534 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:44.293296 13505 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:44.304394 13565 raft_consensus.cc:2955] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:44.305016 12631 raft_consensus.cc:2955] T ab5ddded6fd34f278dade99f2086c4e6 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:44.305972 12702 raft_consensus.cc:2955] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:44.317675 12778 consensus_queue.cc:237] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:44.324702 12530 catalog_manager.cc:5697] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:44.330765 12702 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:17:44.331853 12746 consensus_peers.cc:597] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 18205d974a7c4592aec0153ef3e44c97. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:44.333057 12631 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:44.333350 13565 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:44.334539 13505 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:44.345485 13565 raft_consensus.cc:2955] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:44.352048 12631 raft_consensus.cc:2955] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:44.383365 12709 raft_consensus.cc:2955] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:44.408310 12528 catalog_manager.cc:5697] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:44.405021 12631 consensus_queue.cc:237] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
W20260709 13:17:44.421422 12597 consensus_peers.cc:597] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fb247fe4862541ccadc22bb79417d66c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:44.421773 12781 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:44.423367 13511 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:44.424278 12709 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:44.426052 13550 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:44.440671 13550 raft_consensus.cc:2955] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:44.441758 12778 raft_consensus.cc:2955] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:44.441852 12709 raft_consensus.cc:2955] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:44.457764 12627 consensus_queue.cc:237] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:44.459926 12528 catalog_manager.cc:5697] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:44.464838 12709 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:44.467769 12783 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:17:44.467990 12597 consensus_peers.cc:597] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9132272ce4c847a99e395a53c1015230. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:44.468158 13550 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:44.470757 13511 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:44.485891 13550 raft_consensus.cc:2955] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:44.489470 12781 raft_consensus.cc:2955] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:44.491291 12709 raft_consensus.cc:2955] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:44.517058 12529 catalog_manager.cc:5697] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:44.722385 13582 ts_tablet_manager.cc:933] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:44.727717 13582 tablet_copy_client.cc:323] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.67:41181
I20260709 13:17:44.729192 12793 tablet_copy_service.cc:140] P 3d539f8d95d343d8b1696803f18b30d2: Received BeginTabletCopySession request for tablet ab5ddded6fd34f278dade99f2086c4e6 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:54346)
I20260709 13:17:44.729692 12793 tablet_copy_service.cc:161] P 3d539f8d95d343d8b1696803f18b30d2: Beginning new tablet copy session on tablet ab5ddded6fd34f278dade99f2086c4e6 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:54346: session id = 522fd2ba0e9f402691b5ef29af8579cd-ab5ddded6fd34f278dade99f2086c4e6
I20260709 13:17:44.737731 12793 tablet_copy_source_session.cc:215] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:44.740746 13582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab5ddded6fd34f278dade99f2086c4e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.756735 13582 tablet_copy_client.cc:806] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:44.757855 13582 tablet_copy_client.cc:670] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:44.761219 13584 ts_tablet_manager.cc:933] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:44.762162 13582 tablet_copy_client.cc:538] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:44.763311 13584 tablet_copy_client.cc:323] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.67:41181
I20260709 13:17:44.765013 12793 tablet_copy_service.cc:140] P 3d539f8d95d343d8b1696803f18b30d2: Received BeginTabletCopySession request for tablet 18205d974a7c4592aec0153ef3e44c97 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:54346)
I20260709 13:17:44.767750 12793 tablet_copy_service.cc:161] P 3d539f8d95d343d8b1696803f18b30d2: Beginning new tablet copy session on tablet 18205d974a7c4592aec0153ef3e44c97 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:54346: session id = 522fd2ba0e9f402691b5ef29af8579cd-18205d974a7c4592aec0153ef3e44c97
I20260709 13:17:44.771626 13582 tablet_bootstrap.cc:492] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:44.775195 12793 tablet_copy_source_session.cc:215] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:44.780135 13584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18205d974a7c4592aec0153ef3e44c97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:44.796720 13582 tablet_bootstrap.cc:492] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:44.797766 13582 tablet_bootstrap.cc:492] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:17:44.797839 13584 tablet_copy_client.cc:806] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:44.798440 13584 tablet_copy_client.cc:670] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:44.798625 13582 ts_tablet_manager.cc:1403] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.027s	user 0.016s	sys 0.009s
I20260709 13:17:44.801672 13582 raft_consensus.cc:359] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:44.802506 13584 tablet_copy_client.cc:538] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:44.802479 13582 raft_consensus.cc:740] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:17:44.803632 13582 consensus_queue.cc:260] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:44.806906 13582 ts_tablet_manager.cc:1434] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:17:44.811592 12793 tablet_copy_service.cc:342] P 3d539f8d95d343d8b1696803f18b30d2: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-ab5ddded6fd34f278dade99f2086c4e6 received from {username='slave'} at 127.0.0.1:54346
I20260709 13:17:44.812042 12793 tablet_copy_service.cc:434] P 3d539f8d95d343d8b1696803f18b30d2: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-ab5ddded6fd34f278dade99f2086c4e6 on tablet ab5ddded6fd34f278dade99f2086c4e6 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:44.814375 13584 tablet_bootstrap.cc:492] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:44.832504 13584 tablet_bootstrap.cc:492] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:44.834029 13584 tablet_bootstrap.cc:492] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:17:44.834659 13584 ts_tablet_manager.cc:1403] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.020s	user 0.001s	sys 0.018s
I20260709 13:17:44.838732 13584 raft_consensus.cc:359] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:44.839368 13584 raft_consensus.cc:740] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:17:44.840529 13584 consensus_queue.cc:260] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:44.847458 13584 ts_tablet_manager.cc:1434] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:17:44.849139 12793 tablet_copy_service.cc:342] P 3d539f8d95d343d8b1696803f18b30d2: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-18205d974a7c4592aec0153ef3e44c97 received from {username='slave'} at 127.0.0.1:54346
I20260709 13:17:44.849642 12793 tablet_copy_service.cc:434] P 3d539f8d95d343d8b1696803f18b30d2: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-18205d974a7c4592aec0153ef3e44c97 on tablet 18205d974a7c4592aec0153ef3e44c97 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:45.009825 13584 ts_tablet_manager.cc:933] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:17:45.011513 13584 tablet_copy_client.cc:323] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.65:43603
I20260709 13:17:45.013149 12644 tablet_copy_service.cc:140] P 19cd864c053744bebd96093a7982537e: Received BeginTabletCopySession request for tablet 9132272ce4c847a99e395a53c1015230 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:39590)
I20260709 13:17:45.013710 12644 tablet_copy_service.cc:161] P 19cd864c053744bebd96093a7982537e: Beginning new tablet copy session on tablet 9132272ce4c847a99e395a53c1015230 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:39590: session id = 522fd2ba0e9f402691b5ef29af8579cd-9132272ce4c847a99e395a53c1015230
I20260709 13:17:45.015187 13582 ts_tablet_manager.cc:933] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:17:45.016870 13582 tablet_copy_client.cc:323] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.65:43603
I20260709 13:17:45.018644 12643 tablet_copy_service.cc:140] P 19cd864c053744bebd96093a7982537e: Received BeginTabletCopySession request for tablet fb247fe4862541ccadc22bb79417d66c from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:39590)
I20260709 13:17:45.019176 12643 tablet_copy_service.cc:161] P 19cd864c053744bebd96093a7982537e: Beginning new tablet copy session on tablet fb247fe4862541ccadc22bb79417d66c from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:39590: session id = 522fd2ba0e9f402691b5ef29af8579cd-fb247fe4862541ccadc22bb79417d66c
I20260709 13:17:45.024236 12644 tablet_copy_source_session.cc:215] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:45.026743 12643 tablet_copy_source_session.cc:215] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:45.030205 13582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb247fe4862541ccadc22bb79417d66c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.031387 13584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9132272ce4c847a99e395a53c1015230. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.047739 13584 tablet_copy_client.cc:806] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:45.048110 13582 tablet_copy_client.cc:806] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:45.048375 13584 tablet_copy_client.cc:670] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:45.048712 13582 tablet_copy_client.cc:670] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:45.052913 13582 tablet_copy_client.cc:538] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:45.054306 13584 tablet_copy_client.cc:538] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:45.061756 13582 tablet_bootstrap.cc:492] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:45.063727 13584 tablet_bootstrap.cc:492] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:45.088266 13584 tablet_bootstrap.cc:492] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:45.089354 13584 tablet_bootstrap.cc:492] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:17:45.090183 13584 ts_tablet_manager.cc:1403] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.027s	user 0.020s	sys 0.005s
I20260709 13:17:45.093465 13584 raft_consensus.cc:359] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:45.093668 13582 tablet_bootstrap.cc:492] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:45.094581 13584 raft_consensus.cc:740] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:17:45.095315 13582 tablet_bootstrap.cc:492] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:17:45.096181 13582 ts_tablet_manager.cc:1403] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.035s	user 0.024s	sys 0.004s
I20260709 13:17:45.095531 13584 consensus_queue.cc:260] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:45.098455 13584 ts_tablet_manager.cc:1434] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:17:45.099917 13582 raft_consensus.cc:359] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:45.100952 12644 tablet_copy_service.cc:342] P 19cd864c053744bebd96093a7982537e: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-9132272ce4c847a99e395a53c1015230 received from {username='slave'} at 127.0.0.1:39590
I20260709 13:17:45.101028 13582 raft_consensus.cc:740] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:17:45.101706 12644 tablet_copy_service.cc:434] P 19cd864c053744bebd96093a7982537e: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-9132272ce4c847a99e395a53c1015230 on tablet 9132272ce4c847a99e395a53c1015230 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:45.101958 13582 consensus_queue.cc:260] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:45.104691 13582 ts_tablet_manager.cc:1434] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:17:45.106930 12644 tablet_copy_service.cc:342] P 19cd864c053744bebd96093a7982537e: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-fb247fe4862541ccadc22bb79417d66c received from {username='slave'} at 127.0.0.1:39590
I20260709 13:17:45.107402 12644 tablet_copy_service.cc:434] P 19cd864c053744bebd96093a7982537e: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-fb247fe4862541ccadc22bb79417d66c on tablet fb247fe4862541ccadc22bb79417d66c with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:45.121212 13385 raft_consensus.cc:1217] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:45.135664 13385 raft_consensus.cc:1217] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:45.532663 13385 raft_consensus.cc:1217] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:45.599678 13593 raft_consensus.cc:1064] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:17:45.602259 13593 consensus_queue.cc:237] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:45.607908 13382 raft_consensus.cc:1217] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:45.614899 13382 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:45.617668 12702 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:45.618182 12631 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:45.616586 13593 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:17:45.619565 13534 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:45.620501 13572 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:45.636631 12631 raft_consensus.cc:2955] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:45.634879 13534 raft_consensus.cc:2955] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:45.649963 13382 raft_consensus.cc:2955] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:45.656371 12529 catalog_manager.cc:5697] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:45.667035 12702 raft_consensus.cc:2955] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:45.694713 13593 raft_consensus.cc:1064] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:17:45.697203 13593 consensus_queue.cc:237] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:45.706660 12783 consensus_queue.cc:237] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:45.709504 13382 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:45.710261 12631 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:45.710510 12702 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:45.711126 13599 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:45.712226 13534 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:45.713752 13599 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:17:45.734052 13385 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:45.734194 12704 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:45.735596 13599 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:45.736372 13534 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:45.747123 13572 raft_consensus.cc:2955] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:45.747200 12704 raft_consensus.cc:2955] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:45.758189 13385 raft_consensus.cc:2955] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:45.781857 12529 catalog_manager.cc:5697] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:45.791534 12516 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 18205d974a7c4592aec0153ef3e44c97 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:45.793756 12624 tablet_service.cc:1558] Processing DeleteTablet for tablet 18205d974a7c4592aec0153ef3e44c97 with delete_type TABLET_DATA_TOMBSTONED (TS 19cd864c053744bebd96093a7982537e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56830
I20260709 13:17:45.795709 13593 raft_consensus.cc:2955] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:45.805306 13382 raft_consensus.cc:2955] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:45.811357 12702 raft_consensus.cc:2955] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:45.813688 13608 tablet_replica.cc:333] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:17:45.814975 13608 raft_consensus.cc:2243] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:45.815662 13608 raft_consensus.cc:2272] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:45.817444 12631 raft_consensus.cc:2955] T ab5ddded6fd34f278dade99f2086c4e6 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:45.826778 13608 ts_tablet_manager.cc:1916] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:45.831239 12529 catalog_manager.cc:5697] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:45.845115 13608 ts_tablet_manager.cc:1929] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:45.845635 13608 log.cc:1199] T 18205d974a7c4592aec0153ef3e44c97 P 19cd864c053744bebd96093a7982537e: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-0-root/wals/18205d974a7c4592aec0153ef3e44c97
I20260709 13:17:45.850864 12517 catalog_manager.cc:5028] TS 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): tablet 18205d974a7c4592aec0153ef3e44c97 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:17:45.856840 12781 consensus_queue.cc:237] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:45.867075 12702 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:45.868966 13382 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:45.868798 13599 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:45.872326 13593 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:45.880681 13593 raft_consensus.cc:2955] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:45.883201 12704 raft_consensus.cc:2955] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:45.899365 12516 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ab5ddded6fd34f278dade99f2086c4e6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:45.910115 12529 catalog_manager.cc:5697] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:45.916469 13382 raft_consensus.cc:2955] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:45.922402 12624 tablet_service.cc:1558] Processing DeleteTablet for tablet ab5ddded6fd34f278dade99f2086c4e6 with delete_type TABLET_DATA_TOMBSTONED (TS 19cd864c053744bebd96093a7982537e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56830
I20260709 13:17:45.923471 13608 tablet_replica.cc:333] T ab5ddded6fd34f278dade99f2086c4e6 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:17:45.924638 13608 raft_consensus.cc:2243] T ab5ddded6fd34f278dade99f2086c4e6 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:45.925493 13608 raft_consensus.cc:2272] T ab5ddded6fd34f278dade99f2086c4e6 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:45.934163 13608 ts_tablet_manager.cc:1916] T ab5ddded6fd34f278dade99f2086c4e6 P 19cd864c053744bebd96093a7982537e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:45.952615 13608 ts_tablet_manager.cc:1929] T ab5ddded6fd34f278dade99f2086c4e6 P 19cd864c053744bebd96093a7982537e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:45.952971 13608 log.cc:1199] T ab5ddded6fd34f278dade99f2086c4e6 P 19cd864c053744bebd96093a7982537e: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-0-root/wals/ab5ddded6fd34f278dade99f2086c4e6
I20260709 13:17:45.954241 12517 catalog_manager.cc:5028] TS 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): tablet ab5ddded6fd34f278dade99f2086c4e6 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:17:46.002463 13575 raft_consensus.cc:1064] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:17:46.006114 13575 consensus_queue.cc:237] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:46.012367 13382 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:46.013259 12783 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:46.015084 13550 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:46.016057 13594 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:46.022265 12704 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:46.024888 13550 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:46.039054 13594 raft_consensus.cc:1064] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:17:46.041700 13594 consensus_queue.cc:237] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:46.060287 13550 raft_consensus.cc:2955] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:46.061456 12704 raft_consensus.cc:2955] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:46.075949 12709 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:46.077621 13511 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:46.096570 12781 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:46.096843 13382 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:46.096755 12529 catalog_manager.cc:5697] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:46.100907 13382 raft_consensus.cc:2955] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:46.121179 13594 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:46.121316 12631 consensus_queue.cc:237] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:46.122550 13550 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:46.141453 12702 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:46.144944 13511 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:46.147960 13550 raft_consensus.cc:2955] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:46.151613 12776 raft_consensus.cc:2955] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:46.155123 13575 raft_consensus.cc:2955] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:46.157703 12709 raft_consensus.cc:2955] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:46.160543 13385 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:46.169030 12529 catalog_manager.cc:5697] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:46.175349 13511 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:17:46.179288 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fb247fe4862541ccadc22bb79417d66c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:46.185110 13382 raft_consensus.cc:2955] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:46.192813 12702 raft_consensus.cc:2955] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:46.208211 12529 catalog_manager.cc:5697] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:46.220887 13382 raft_consensus.cc:2955] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:46.225713 12773 tablet_service.cc:1558] Processing DeleteTablet for tablet fb247fe4862541ccadc22bb79417d66c with delete_type TABLET_DATA_TOMBSTONED (TS 3d539f8d95d343d8b1696803f18b30d2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36472
I20260709 13:17:46.233330 13622 tablet_replica.cc:333] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:17:46.234377 13622 raft_consensus.cc:2243] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:46.235051 13622 pending_rounds.cc:70] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2: Trying to abort 1 pending ops.
I20260709 13:17:46.235426 13622 pending_rounds.cc:77] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7305638158352756736 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "fb247fe4862541ccadc22bb79417d66c" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } } }
I20260709 13:17:46.236824 13622 raft_consensus.cc:2917] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20260709 13:17:46.237267 13622 raft_consensus.cc:2889] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 13:17:46.237660 13622 raft_consensus.cc:2272] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:46.241112 13622 ts_tablet_manager.cc:1916] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:46.242830 12631 consensus_queue.cc:237] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:46.253718 12776 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:46.253801 13385 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:46.255720 13619 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:46.257550 13619 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:46.259447 13622 ts_tablet_manager.cc:1929] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:46.259948 13622 log.cc:1199] T fb247fe4862541ccadc22bb79417d66c P 3d539f8d95d343d8b1696803f18b30d2: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-2-root/wals/fb247fe4862541ccadc22bb79417d66c
I20260709 13:17:46.261524 12516 catalog_manager.cc:5028] TS 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): tablet fb247fe4862541ccadc22bb79417d66c (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:17:46.267354 13550 raft_consensus.cc:2955] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:46.270934 13385 raft_consensus.cc:2955] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:46.274097 12783 raft_consensus.cc:2955] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:46.281425 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9132272ce4c847a99e395a53c1015230 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:46.288625 12528 catalog_manager.cc:5697] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:46.299181 12695 tablet_service.cc:1558] Processing DeleteTablet for tablet 9132272ce4c847a99e395a53c1015230 with delete_type TABLET_DATA_TOMBSTONED (TS 639ab2db265f4747bb7afb93e833f530 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42474
I20260709 13:17:46.301056 13623 tablet_replica.cc:333] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:17:46.302333 13623 raft_consensus.cc:2243] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:46.303119 13623 raft_consensus.cc:2272] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:46.307121 13623 ts_tablet_manager.cc:1916] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:46.325057 13623 ts_tablet_manager.cc:1929] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:46.325676 13623 log.cc:1199] T 9132272ce4c847a99e395a53c1015230 P 639ab2db265f4747bb7afb93e833f530: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-1-root/wals/9132272ce4c847a99e395a53c1015230
I20260709 13:17:46.327582 12517 catalog_manager.cc:5028] TS 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): tablet 9132272ce4c847a99e395a53c1015230 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:17:47.365411 12626 consensus_queue.cc:237] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
W20260709 13:17:47.374185 12597 consensus_peers.cc:597] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3b98bb10e3e04334b7fcf7f781593682. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:47.375300 12783 raft_consensus.cc:1275] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:47.376803 12702 raft_consensus.cc:1275] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:47.379683 13550 consensus_queue.cc:1048] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:47.380461 13619 consensus_queue.cc:1048] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:47.395303 13550 raft_consensus.cc:2955] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:47.397938 12783 raft_consensus.cc:2955] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:47.397858 12702 raft_consensus.cc:2955] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:47.415341 12530 catalog_manager.cc:5697] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:47.415695 12702 consensus_queue.cc:237] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:47.434181 12781 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:17:47.438308 12672 consensus_peers.cc:597] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a4d7de736149496b86a37ba2ca92670d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:47.439433 12626 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:47.439682 13609 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:47.442492 13609 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:47.463766 13528 raft_consensus.cc:2955] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:47.475569 12626 raft_consensus.cc:2955] T a4d7de736149496b86a37ba2ca92670d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:47.478499 12783 raft_consensus.cc:2955] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:47.493177 12781 consensus_queue.cc:237] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:47.501801 12702 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:17:47.502151 12746 consensus_peers.cc:597] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b1054d44ad14423fb3d51a5243499bb9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:47.506953 13629 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:47.507694 12627 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:47.509208 13624 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:47.522192 13629 raft_consensus.cc:2955] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:47.524349 12702 raft_consensus.cc:2955] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:47.526201 12626 raft_consensus.cc:2955] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:47.532718 12528 catalog_manager.cc:5697] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:47.544552 12781 consensus_queue.cc:237] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:47.554050 12702 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:47.554245 12626 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:17:47.555104 12746 consensus_peers.cc:597] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4d53f974eee5475b9cd2701948f533ac. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:47.555770 13593 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:47.559060 13624 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:47.581548 12529 catalog_manager.cc:5697] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:47.586061 12627 raft_consensus.cc:2955] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:47.580760 13624 raft_consensus.cc:2955] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:47.616638 12702 raft_consensus.cc:2955] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:47.621049 12529 catalog_manager.cc:5697] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:47.907445 13646 ts_tablet_manager.cc:933] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:47.916319 13646 tablet_copy_client.cc:323] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.66:45173
I20260709 13:17:47.918526 12718 tablet_copy_service.cc:140] P 639ab2db265f4747bb7afb93e833f530: Received BeginTabletCopySession request for tablet a4d7de736149496b86a37ba2ca92670d from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:38496)
I20260709 13:17:47.919128 12718 tablet_copy_service.cc:161] P 639ab2db265f4747bb7afb93e833f530: Beginning new tablet copy session on tablet a4d7de736149496b86a37ba2ca92670d from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:38496: session id = 522fd2ba0e9f402691b5ef29af8579cd-a4d7de736149496b86a37ba2ca92670d
I20260709 13:17:47.927214 12718 tablet_copy_source_session.cc:215] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:47.932302 13646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4d7de736149496b86a37ba2ca92670d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:47.950062 13646 tablet_copy_client.cc:806] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:47.950681 13646 tablet_copy_client.cc:670] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:47.954660 13646 tablet_copy_client.cc:538] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:47.963670 13646 tablet_bootstrap.cc:492] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:47.980485 13648 ts_tablet_manager.cc:933] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:17:47.983819 13648 tablet_copy_client.cc:323] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.65:43603
I20260709 13:17:47.985627 12644 tablet_copy_service.cc:140] P 19cd864c053744bebd96093a7982537e: Received BeginTabletCopySession request for tablet 3b98bb10e3e04334b7fcf7f781593682 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:39590)
I20260709 13:17:47.986472 12644 tablet_copy_service.cc:161] P 19cd864c053744bebd96093a7982537e: Beginning new tablet copy session on tablet 3b98bb10e3e04334b7fcf7f781593682 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:39590: session id = 522fd2ba0e9f402691b5ef29af8579cd-3b98bb10e3e04334b7fcf7f781593682
I20260709 13:17:47.995440 13651 ts_tablet_manager.cc:933] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:47.997874 13651 tablet_copy_client.cc:323] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.67:41181
I20260709 13:17:47.999660 12793 tablet_copy_service.cc:140] P 3d539f8d95d343d8b1696803f18b30d2: Received BeginTabletCopySession request for tablet b1054d44ad14423fb3d51a5243499bb9 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:54346)
I20260709 13:17:48.000198 12793 tablet_copy_service.cc:161] P 3d539f8d95d343d8b1696803f18b30d2: Beginning new tablet copy session on tablet b1054d44ad14423fb3d51a5243499bb9 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:54346: session id = 522fd2ba0e9f402691b5ef29af8579cd-b1054d44ad14423fb3d51a5243499bb9
I20260709 13:17:48.002058 12644 tablet_copy_source_session.cc:215] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:48.005430 13648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b98bb10e3e04334b7fcf7f781593682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.007654 12793 tablet_copy_source_session.cc:215] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:48.011130 13651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1054d44ad14423fb3d51a5243499bb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.014806 13646 tablet_bootstrap.cc:492] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:48.015810 13646 tablet_bootstrap.cc:492] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:17:48.016541 13646 ts_tablet_manager.cc:1403] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.053s	user 0.015s	sys 0.009s
I20260709 13:17:48.019546 13646 raft_consensus.cc:359] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:48.020715 13646 raft_consensus.cc:740] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:17:48.021576 13648 tablet_copy_client.cc:806] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:48.021426 13646 consensus_queue.cc:260] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:48.022157 13648 tablet_copy_client.cc:670] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:48.031430 13646 ts_tablet_manager.cc:1434] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.004s
I20260709 13:17:48.034783 13648 tablet_copy_client.cc:538] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:48.036161 12718 tablet_copy_service.cc:342] P 639ab2db265f4747bb7afb93e833f530: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-a4d7de736149496b86a37ba2ca92670d received from {username='slave'} at 127.0.0.1:38496
I20260709 13:17:48.036855 12718 tablet_copy_service.cc:434] P 639ab2db265f4747bb7afb93e833f530: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-a4d7de736149496b86a37ba2ca92670d on tablet a4d7de736149496b86a37ba2ca92670d with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:48.043928 13651 tablet_copy_client.cc:806] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:48.044497 13651 tablet_copy_client.cc:670] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:48.048383 13651 tablet_copy_client.cc:538] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:48.048879 13648 tablet_bootstrap.cc:492] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:48.061688 13651 tablet_bootstrap.cc:492] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:48.089255 13648 tablet_bootstrap.cc:492] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:48.090215 13648 tablet_bootstrap.cc:492] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:17:48.090878 13648 ts_tablet_manager.cc:1403] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.042s	user 0.018s	sys 0.016s
I20260709 13:17:48.094273 13648 raft_consensus.cc:359] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:48.095093 13648 raft_consensus.cc:740] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:17:48.096923 13648 consensus_queue.cc:260] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:48.099421 13648 ts_tablet_manager.cc:1434] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:17:48.099445 13651 tablet_bootstrap.cc:492] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:48.100989 13651 tablet_bootstrap.cc:492] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:17:48.101754 13651 ts_tablet_manager.cc:1403] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.040s	user 0.019s	sys 0.004s
I20260709 13:17:48.104488 12644 tablet_copy_service.cc:342] P 19cd864c053744bebd96093a7982537e: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-3b98bb10e3e04334b7fcf7f781593682 received from {username='slave'} at 127.0.0.1:39590
I20260709 13:17:48.104977 12644 tablet_copy_service.cc:434] P 19cd864c053744bebd96093a7982537e: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-3b98bb10e3e04334b7fcf7f781593682 on tablet 3b98bb10e3e04334b7fcf7f781593682 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:48.106952 13651 raft_consensus.cc:359] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:48.107945 13651 raft_consensus.cc:740] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:17:48.110997 13651 consensus_queue.cc:260] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:48.113483 13651 ts_tablet_manager.cc:1434] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.000s
I20260709 13:17:48.115239 12793 tablet_copy_service.cc:342] P 3d539f8d95d343d8b1696803f18b30d2: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-b1054d44ad14423fb3d51a5243499bb9 received from {username='slave'} at 127.0.0.1:54346
I20260709 13:17:48.115682 12793 tablet_copy_service.cc:434] P 3d539f8d95d343d8b1696803f18b30d2: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-b1054d44ad14423fb3d51a5243499bb9 on tablet b1054d44ad14423fb3d51a5243499bb9 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:48.172199 13651 ts_tablet_manager.cc:933] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:48.174045 13651 tablet_copy_client.cc:323] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.67:41181
I20260709 13:17:48.175771 12793 tablet_copy_service.cc:140] P 3d539f8d95d343d8b1696803f18b30d2: Received BeginTabletCopySession request for tablet 4d53f974eee5475b9cd2701948f533ac from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:54346)
I20260709 13:17:48.176364 12793 tablet_copy_service.cc:161] P 3d539f8d95d343d8b1696803f18b30d2: Beginning new tablet copy session on tablet 4d53f974eee5475b9cd2701948f533ac from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:54346: session id = 522fd2ba0e9f402691b5ef29af8579cd-4d53f974eee5475b9cd2701948f533ac
I20260709 13:17:48.184473 12793 tablet_copy_source_session.cc:215] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:48.188890 13651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d53f974eee5475b9cd2701948f533ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:48.208606 13651 tablet_copy_client.cc:806] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:48.209165 13651 tablet_copy_client.cc:670] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:48.212585 13651 tablet_copy_client.cc:538] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:48.219460 13651 tablet_bootstrap.cc:492] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:48.238250 13651 tablet_bootstrap.cc:492] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:48.239087 13651 tablet_bootstrap.cc:492] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:17:48.239748 13651 ts_tablet_manager.cc:1403] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.005s
I20260709 13:17:48.242676 13651 raft_consensus.cc:359] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:48.243332 13651 raft_consensus.cc:740] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:17:48.243917 13651 consensus_queue.cc:260] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:48.245792 13651 ts_tablet_manager.cc:1434] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.004s
I20260709 13:17:48.247503 12793 tablet_copy_service.cc:342] P 3d539f8d95d343d8b1696803f18b30d2: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-4d53f974eee5475b9cd2701948f533ac received from {username='slave'} at 127.0.0.1:54346
I20260709 13:17:48.247898 12793 tablet_copy_service.cc:434] P 3d539f8d95d343d8b1696803f18b30d2: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-4d53f974eee5475b9cd2701948f533ac on tablet 4d53f974eee5475b9cd2701948f533ac with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:48.412622 13385 raft_consensus.cc:1217] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:48.473513 13385 raft_consensus.cc:1217] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:48.502583 13385 raft_consensus.cc:1217] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:48.689255 13385 raft_consensus.cc:1217] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:48.830852 13630 raft_consensus.cc:1064] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:17:48.836349 13630 consensus_queue.cc:237] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:48.843554 13385 raft_consensus.cc:1275] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:48.845872 12781 raft_consensus.cc:1275] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:48.846109 12702 raft_consensus.cc:1275] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:48.847774 13619 consensus_queue.cc:1048] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:48.849058 13630 consensus_queue.cc:1048] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:48.851032 13619 consensus_queue.cc:1048] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:48.867623 13619 raft_consensus.cc:2955] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:48.870335 12702 raft_consensus.cc:2955] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:48.874502 12781 raft_consensus.cc:2955] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:48.888031 13385 raft_consensus.cc:2955] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:48.895777 12530 catalog_manager.cc:5697] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:48.932803 12626 consensus_queue.cc:237] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:48.937451 13385 raft_consensus.cc:1275] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:48.938189 12702 raft_consensus.cc:1275] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:48.939003 13630 consensus_queue.cc:1048] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:48.939945 13619 consensus_queue.cc:1048] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:48.946738 13550 raft_consensus.cc:2955] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:48.949133 12702 raft_consensus.cc:2955] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:48.955123 13385 raft_consensus.cc:2955] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:48.956707 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3b98bb10e3e04334b7fcf7f781593682 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:48.957135 13659 raft_consensus.cc:1064] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:17:48.959695 13659 consensus_queue.cc:237] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:48.975580 12528 catalog_manager.cc:5697] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.000353 12626 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:49.002350 12781 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:49.002887 13382 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:49.003134 13493 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:49.004770 13528 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:49.004933 12773 tablet_service.cc:1558] Processing DeleteTablet for tablet 3b98bb10e3e04334b7fcf7f781593682 with delete_type TABLET_DATA_TOMBSTONED (TS 3d539f8d95d343d8b1696803f18b30d2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36472
I20260709 13:17:49.007169 13528 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:49.041682 13659 raft_consensus.cc:2955] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.044301 12781 raft_consensus.cc:2955] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.046228 13669 tablet_replica.cc:333] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:17:49.047202 13669 raft_consensus.cc:2243] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:49.047951 13669 raft_consensus.cc:2272] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:49.053102 13658 raft_consensus.cc:1064] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:17:49.055732 13658 consensus_queue.cc:237] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:49.059499 13669 ts_tablet_manager.cc:1916] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:49.066150 12627 raft_consensus.cc:2955] T a4d7de736149496b86a37ba2ca92670d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.066283 12532 catalog_manager.cc:5697] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.102988 13382 raft_consensus.cc:2955] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.114104 13669 ts_tablet_manager.cc:1929] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:49.117545 12702 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:49.126171 13645 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:49.126246 12704 consensus_queue.cc:237] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:49.127562 12627 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:49.121121 13384 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:49.123288 13658 raft_consensus.cc:1064] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:17:49.131125 13593 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:49.132733 13593 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:49.138492 13385 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:49.141688 13528 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:49.145776 12776 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:49.146250 13658 consensus_queue.cc:237] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:49.151561 13528 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:49.157359 13645 raft_consensus.cc:2955] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.161252 12709 raft_consensus.cc:2955] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.165926 13528 raft_consensus.cc:2955] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.180716 12532 catalog_manager.cc:5697] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.172524 13669 log.cc:1199] T 3b98bb10e3e04334b7fcf7f781593682 P 3d539f8d95d343d8b1696803f18b30d2: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-2-root/wals/3b98bb10e3e04334b7fcf7f781593682
I20260709 13:17:49.190412 12516 catalog_manager.cc:5028] TS 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): tablet 3b98bb10e3e04334b7fcf7f781593682 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:17:49.203601 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a4d7de736149496b86a37ba2ca92670d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:49.205919 12704 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:49.206130 13382 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:49.207425 12627 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:49.207229 12532 catalog_manager.cc:5697] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:49.199306 12776 raft_consensus.cc:2955] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.210420 13593 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:49.212038 13593 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:49.220185 13385 raft_consensus.cc:2955] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.238715 13674 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:17:49.264273 12624 tablet_service.cc:1558] Processing DeleteTablet for tablet a4d7de736149496b86a37ba2ca92670d with delete_type TABLET_DATA_TOMBSTONED (TS 19cd864c053744bebd96093a7982537e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56830
I20260709 13:17:49.311825 12626 raft_consensus.cc:2955] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.324105 13682 raft_consensus.cc:2955] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.337030 12704 raft_consensus.cc:2955] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.339773 13385 raft_consensus.cc:2955] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.341755 13384 raft_consensus.cc:2955] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.355849 12532 catalog_manager.cc:5697] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.367590 12781 consensus_queue.cc:237] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:49.375039 13385 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:49.376181 13686 tablet_replica.cc:333] T a4d7de736149496b86a37ba2ca92670d P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:17:49.376725 12626 raft_consensus.cc:2955] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.377036 13658 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:49.381271 13686 raft_consensus.cc:2243] T a4d7de736149496b86a37ba2ca92670d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:49.382043 13686 raft_consensus.cc:2272] T a4d7de736149496b86a37ba2ca92670d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:49.382743 12781 consensus_queue.cc:237] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:49.385519 13686 ts_tablet_manager.cc:1916] T a4d7de736149496b86a37ba2ca92670d P 19cd864c053744bebd96093a7982537e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:49.394085 13382 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:49.397431 13658 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:49.399309 13658 raft_consensus.cc:2955] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.404192 13384 raft_consensus.cc:2955] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.405582 13645 raft_consensus.cc:2955] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.413028 13385 raft_consensus.cc:2955] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.417827 12627 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:49.420550 12627 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:49.424399 13680 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:49.425275 13674 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:49.429944 12626 raft_consensus.cc:2955] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.432168 12516 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4d53f974eee5475b9cd2701948f533ac with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:49.431371 12627 raft_consensus.cc:2955] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.439829 12532 catalog_manager.cc:5697] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:49.442113 12528 catalog_manager.cc:5697] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:49.449090 12516 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b1054d44ad14423fb3d51a5243499bb9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:49.440461 13686 ts_tablet_manager.cc:1929] T a4d7de736149496b86a37ba2ca92670d P 19cd864c053744bebd96093a7982537e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:49.454051 13686 log.cc:1199] T a4d7de736149496b86a37ba2ca92670d P 19cd864c053744bebd96093a7982537e: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-0-root/wals/a4d7de736149496b86a37ba2ca92670d
I20260709 13:17:49.457307 12517 catalog_manager.cc:5028] TS 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): tablet a4d7de736149496b86a37ba2ca92670d (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:17:49.463840 12695 tablet_service.cc:1558] Processing DeleteTablet for tablet b1054d44ad14423fb3d51a5243499bb9 with delete_type TABLET_DATA_TOMBSTONED (TS 639ab2db265f4747bb7afb93e833f530 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42474
I20260709 13:17:49.466285 13688 tablet_replica.cc:333] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:17:49.467346 13688 raft_consensus.cc:2243] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:49.468158 13688 raft_consensus.cc:2272] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:49.469838 12695 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d53f974eee5475b9cd2701948f533ac with delete_type TABLET_DATA_TOMBSTONED (TS 639ab2db265f4747bb7afb93e833f530 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42474
I20260709 13:17:49.472383 13688 ts_tablet_manager.cc:1916] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:49.496088 13688 ts_tablet_manager.cc:1929] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:49.496371 13688 log.cc:1199] T b1054d44ad14423fb3d51a5243499bb9 P 639ab2db265f4747bb7afb93e833f530: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-1-root/wals/b1054d44ad14423fb3d51a5243499bb9
I20260709 13:17:49.498023 13688 tablet_replica.cc:333] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:17:49.497893 12517 catalog_manager.cc:5028] TS 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): tablet b1054d44ad14423fb3d51a5243499bb9 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:17:49.499006 13688 raft_consensus.cc:2243] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:49.499619 13688 raft_consensus.cc:2272] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:49.502784 13688 ts_tablet_manager.cc:1916] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:49.515396 13688 ts_tablet_manager.cc:1929] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:49.515795 13688 log.cc:1199] T 4d53f974eee5475b9cd2701948f533ac P 639ab2db265f4747bb7afb93e833f530: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-1-root/wals/4d53f974eee5475b9cd2701948f533ac
I20260709 13:17:49.517127 12517 catalog_manager.cc:5028] TS 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): tablet 4d53f974eee5475b9cd2701948f533ac (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:17:50.549866 12704 consensus_queue.cc:237] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
W20260709 13:17:50.557665 12672 consensus_peers.cc:597] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8f91059867c444d89718e83475c089cd. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:50.557750 12776 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:50.557708 12633 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:50.559655 13633 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:50.560791 13659 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:50.571688 12776 raft_consensus.cc:2955] T 8f91059867c444d89718e83475c089cd P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:50.572371 12633 raft_consensus.cc:2955] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:50.569989 13633 raft_consensus.cc:2955] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:50.599278 12528 catalog_manager.cc:5697] T 8f91059867c444d89718e83475c089cd P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:50.617920 12776 consensus_queue.cc:237] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:50.626281 12709 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:17:50.627624 12746 consensus_peers.cc:597] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 866523bf2d2f442f971d504267ed200f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:50.630193 12633 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:50.630579 13645 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:50.631775 13658 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:50.665364 13658 raft_consensus.cc:2955] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:50.679476 12709 raft_consensus.cc:2955] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:50.682312 12633 raft_consensus.cc:2955] T 866523bf2d2f442f971d504267ed200f P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:50.686647 12704 consensus_queue.cc:237] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:50.693742 12529 catalog_manager.cc:5697] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:50.704537 12776 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:50.707162 12627 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:50.708779 13633 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:17:50.709611 12672 consensus_peers.cc:597] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8e0d0566acd7465f87e20053c75e28fa. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:50.711814 13690 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:50.729765 13633 raft_consensus.cc:2955] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:50.732076 12781 raft_consensus.cc:2955] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:50.734098 12633 raft_consensus.cc:2955] T 8e0d0566acd7465f87e20053c75e28fa P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:50.751443 12633 consensus_queue.cc:237] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:50.751523 12528 catalog_manager.cc:5697] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:50.762838 12597 consensus_peers.cc:597] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9155962ca7f34a8c83a1780ce0018a59. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:50.765722 12781 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:50.768064 12704 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:50.768251 13691 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:50.770494 13691 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:50.788550 13630 raft_consensus.cc:2955] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:50.791646 12781 raft_consensus.cc:2955] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:50.794296 12704 raft_consensus.cc:2955] T 9155962ca7f34a8c83a1780ce0018a59 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:50.815811 12528 catalog_manager.cc:5697] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:51.140944 13709 ts_tablet_manager.cc:933] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:51.143051 13709 tablet_copy_client.cc:323] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.66:45173
I20260709 13:17:51.145153 12718 tablet_copy_service.cc:140] P 639ab2db265f4747bb7afb93e833f530: Received BeginTabletCopySession request for tablet 8f91059867c444d89718e83475c089cd from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:38496)
I20260709 13:17:51.145792 12718 tablet_copy_service.cc:161] P 639ab2db265f4747bb7afb93e833f530: Beginning new tablet copy session on tablet 8f91059867c444d89718e83475c089cd from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:38496: session id = 522fd2ba0e9f402691b5ef29af8579cd-8f91059867c444d89718e83475c089cd
I20260709 13:17:51.152021 12718 tablet_copy_source_session.cc:215] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:51.156487 13709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f91059867c444d89718e83475c089cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:51.175558 13709 tablet_copy_client.cc:806] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:51.176275 13709 tablet_copy_client.cc:670] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:51.180380 13709 tablet_copy_client.cc:538] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:51.190830 13709 tablet_bootstrap.cc:492] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:51.220889 13711 ts_tablet_manager.cc:933] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:51.240608 13711 tablet_copy_client.cc:323] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.67:41181
I20260709 13:17:51.243083 12793 tablet_copy_service.cc:140] P 3d539f8d95d343d8b1696803f18b30d2: Received BeginTabletCopySession request for tablet 866523bf2d2f442f971d504267ed200f from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:54346)
I20260709 13:17:51.243701 12793 tablet_copy_service.cc:161] P 3d539f8d95d343d8b1696803f18b30d2: Beginning new tablet copy session on tablet 866523bf2d2f442f971d504267ed200f from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:54346: session id = 522fd2ba0e9f402691b5ef29af8579cd-866523bf2d2f442f971d504267ed200f
I20260709 13:17:51.251868 12793 tablet_copy_source_session.cc:215] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:51.253867 13709 tablet_bootstrap.cc:492] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:51.254866 13709 tablet_bootstrap.cc:492] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:17:51.255740 13709 ts_tablet_manager.cc:1403] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.065s	user 0.027s	sys 0.032s
I20260709 13:17:51.258345 13711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 866523bf2d2f442f971d504267ed200f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:51.259090 13709 raft_consensus.cc:359] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:51.259984 13709 raft_consensus.cc:740] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:17:51.260689 13709 consensus_queue.cc:260] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:51.270825 13709 ts_tablet_manager.cc:1434] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.003s
I20260709 13:17:51.272944 12718 tablet_copy_service.cc:342] P 639ab2db265f4747bb7afb93e833f530: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-8f91059867c444d89718e83475c089cd received from {username='slave'} at 127.0.0.1:38496
I20260709 13:17:51.273415 12718 tablet_copy_service.cc:434] P 639ab2db265f4747bb7afb93e833f530: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-8f91059867c444d89718e83475c089cd on tablet 8f91059867c444d89718e83475c089cd with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:51.281807 13709 ts_tablet_manager.cc:933] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:51.298031 13711 tablet_copy_client.cc:806] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:51.298871 13711 tablet_copy_client.cc:670] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:51.299752 13709 tablet_copy_client.cc:323] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.66:45173
I20260709 13:17:51.301257 12718 tablet_copy_service.cc:140] P 639ab2db265f4747bb7afb93e833f530: Received BeginTabletCopySession request for tablet 8e0d0566acd7465f87e20053c75e28fa from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:38496)
I20260709 13:17:51.301739 12718 tablet_copy_service.cc:161] P 639ab2db265f4747bb7afb93e833f530: Beginning new tablet copy session on tablet 8e0d0566acd7465f87e20053c75e28fa from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:38496: session id = 522fd2ba0e9f402691b5ef29af8579cd-8e0d0566acd7465f87e20053c75e28fa
I20260709 13:17:51.307435 13711 tablet_copy_client.cc:538] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:51.309466 12718 tablet_copy_source_session.cc:215] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:51.315382 13716 ts_tablet_manager.cc:933] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:17:51.316736 13709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e0d0566acd7465f87e20053c75e28fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:51.319667 13711 tablet_bootstrap.cc:492] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:51.324312 13716 tablet_copy_client.cc:323] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.65:43603
I20260709 13:17:51.326593 12644 tablet_copy_service.cc:140] P 19cd864c053744bebd96093a7982537e: Received BeginTabletCopySession request for tablet 9155962ca7f34a8c83a1780ce0018a59 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:39590)
I20260709 13:17:51.327123 12644 tablet_copy_service.cc:161] P 19cd864c053744bebd96093a7982537e: Beginning new tablet copy session on tablet 9155962ca7f34a8c83a1780ce0018a59 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:39590: session id = 522fd2ba0e9f402691b5ef29af8579cd-9155962ca7f34a8c83a1780ce0018a59
I20260709 13:17:51.340766 12644 tablet_copy_source_session.cc:215] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:51.346870 13716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9155962ca7f34a8c83a1780ce0018a59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:51.387821 13716 tablet_copy_client.cc:806] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:51.388764 13716 tablet_copy_client.cc:670] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:51.388886 13711 tablet_bootstrap.cc:492] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:51.390373 13711 tablet_bootstrap.cc:492] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:17:51.391225 13711 ts_tablet_manager.cc:1403] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.072s	user 0.027s	sys 0.010s
I20260709 13:17:51.392933 13716 tablet_copy_client.cc:538] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:51.395596 13711 raft_consensus.cc:359] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:51.396549 13711 raft_consensus.cc:740] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:17:51.397176 13711 consensus_queue.cc:260] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:51.400501 13711 ts_tablet_manager.cc:1434] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:17:51.402052 12793 tablet_copy_service.cc:342] P 3d539f8d95d343d8b1696803f18b30d2: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-866523bf2d2f442f971d504267ed200f received from {username='slave'} at 127.0.0.1:54346
I20260709 13:17:51.402499 12793 tablet_copy_service.cc:434] P 3d539f8d95d343d8b1696803f18b30d2: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-866523bf2d2f442f971d504267ed200f on tablet 866523bf2d2f442f971d504267ed200f with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:51.403052 13716 tablet_bootstrap.cc:492] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:51.404413 13709 tablet_copy_client.cc:806] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:51.405243 13709 tablet_copy_client.cc:670] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:51.418222 13709 tablet_copy_client.cc:538] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:51.432739 13716 tablet_bootstrap.cc:492] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:51.433854 13716 tablet_bootstrap.cc:492] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:17:51.434618 13716 ts_tablet_manager.cc:1403] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.013s
I20260709 13:17:51.436887 13709 tablet_bootstrap.cc:492] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:51.437590 13716 raft_consensus.cc:359] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:51.438549 13716 raft_consensus.cc:740] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:17:51.439553 13716 consensus_queue.cc:260] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:51.442420 13716 ts_tablet_manager.cc:1434] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:17:51.444888 12644 tablet_copy_service.cc:342] P 19cd864c053744bebd96093a7982537e: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-9155962ca7f34a8c83a1780ce0018a59 received from {username='slave'} at 127.0.0.1:39590
I20260709 13:17:51.445355 12644 tablet_copy_service.cc:434] P 19cd864c053744bebd96093a7982537e: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-9155962ca7f34a8c83a1780ce0018a59 on tablet 9155962ca7f34a8c83a1780ce0018a59 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:51.465590 13709 tablet_bootstrap.cc:492] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:51.467468 13709 tablet_bootstrap.cc:492] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:17:51.468164 13709 ts_tablet_manager.cc:1403] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.031s	user 0.022s	sys 0.005s
I20260709 13:17:51.471284 13709 raft_consensus.cc:359] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:51.472110 13709 raft_consensus.cc:740] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:17:51.472668 13709 consensus_queue.cc:260] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:51.474888 13709 ts_tablet_manager.cc:1434] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:17:51.480711 12718 tablet_copy_service.cc:342] P 639ab2db265f4747bb7afb93e833f530: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-8e0d0566acd7465f87e20053c75e28fa received from {username='slave'} at 127.0.0.1:38496
I20260709 13:17:51.481135 12718 tablet_copy_service.cc:434] P 639ab2db265f4747bb7afb93e833f530: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-8e0d0566acd7465f87e20053c75e28fa on tablet 8e0d0566acd7465f87e20053c75e28fa with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:51.697417 13384 raft_consensus.cc:1217] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:51.788550 13384 raft_consensus.cc:1217] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:51.804668 13384 raft_consensus.cc:1217] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:51.917006 13384 raft_consensus.cc:1217] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:52.174094 13724 raft_consensus.cc:1064] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:17:52.177419 13724 consensus_queue.cc:237] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:52.185478 12633 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:52.186163 13384 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:52.187103 13493 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:52.185478 12776 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:52.188007 13724 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:52.190527 13724 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:52.205163 13633 raft_consensus.cc:2955] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.206977 12627 raft_consensus.cc:2955] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.227100 13384 raft_consensus.cc:2955] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.235121 12528 catalog_manager.cc:5697] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.235037 12778 raft_consensus.cc:2955] T 8f91059867c444d89718e83475c089cd P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.293241 12709 consensus_queue.cc:237] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:52.301610 12627 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:52.302558 13384 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:52.303089 13724 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:52.304525 13633 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:52.309934 13493 raft_consensus.cc:2955] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.311501 12633 raft_consensus.cc:2955] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.342840 13702 raft_consensus.cc:1064] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:17:52.351784 12532 catalog_manager.cc:5697] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.356804 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8f91059867c444d89718e83475c089cd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:52.345350 13702 consensus_queue.cc:237] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:52.359262 13384 raft_consensus.cc:2955] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.371423 13633 raft_consensus.cc:1064] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:17:52.381711 12773 tablet_service.cc:1558] Processing DeleteTablet for tablet 8f91059867c444d89718e83475c089cd with delete_type TABLET_DATA_TOMBSTONED (TS 3d539f8d95d343d8b1696803f18b30d2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36472
I20260709 13:17:52.399842 13736 tablet_replica.cc:333] T 8f91059867c444d89718e83475c089cd P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:17:52.400921 13736 raft_consensus.cc:2243] T 8f91059867c444d89718e83475c089cd P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:52.401919 13736 raft_consensus.cc:2272] T 8f91059867c444d89718e83475c089cd P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:52.409152 13384 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:52.409818 13736 ts_tablet_manager.cc:1916] T 8f91059867c444d89718e83475c089cd P 3d539f8d95d343d8b1696803f18b30d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:52.411288 13722 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:52.419016 13630 raft_consensus.cc:1064] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:17:52.421195 13630 consensus_queue.cc:237] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:52.420864 13633 consensus_queue.cc:237] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:52.430398 12709 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:52.433270 12626 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:52.433435 13702 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:52.434892 13722 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:52.449158 12783 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:52.449249 13382 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:52.450810 13659 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:52.452165 13734 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:52.458132 12633 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:52.459740 13493 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:52.471776 13702 raft_consensus.cc:2955] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.477815 12709 raft_consensus.cc:2955] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.482517 12626 raft_consensus.cc:2955] T 866523bf2d2f442f971d504267ed200f P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.486722 13384 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:52.488436 12702 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:52.491218 13691 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:52.493074 13736 ts_tablet_manager.cc:1929] T 8f91059867c444d89718e83475c089cd P 3d539f8d95d343d8b1696803f18b30d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:52.498585 13736 log.cc:1199] T 8f91059867c444d89718e83475c089cd P 3d539f8d95d343d8b1696803f18b30d2: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-2-root/wals/8f91059867c444d89718e83475c089cd
I20260709 13:17:52.511989 12776 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:52.515764 13725 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:52.530547 12516 catalog_manager.cc:5028] TS 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): tablet 8f91059867c444d89718e83475c089cd (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:17:52.534147 13725 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:52.517849 12532 catalog_manager.cc:5697] T 866523bf2d2f442f971d504267ed200f P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.524473 13384 raft_consensus.cc:2955] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.561753 13659 raft_consensus.cc:2955] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.564031 13630 raft_consensus.cc:2955] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.571689 12626 raft_consensus.cc:2955] T 8e0d0566acd7465f87e20053c75e28fa P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.565797 12778 raft_consensus.cc:2955] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.579882 12783 consensus_queue.cc:237] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:52.576555 13382 raft_consensus.cc:2955] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.597257 12709 raft_consensus.cc:2955] T 9155962ca7f34a8c83a1780ce0018a59 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.610126 12778 raft_consensus.cc:2955] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.612869 13384 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:52.615828 13722 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:52.621276 12702 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:52.623329 12528 catalog_manager.cc:5697] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.625839 12529 catalog_manager.cc:5697] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:52.628319 13737 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:52.644423 13723 raft_consensus.cc:2955] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.645681 13385 raft_consensus.cc:2955] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.662075 12709 raft_consensus.cc:2955] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.671770 13382 raft_consensus.cc:2955] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.677973 12516 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 866523bf2d2f442f971d504267ed200f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:52.681335 12702 consensus_queue.cc:237] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:52.702876 13385 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:52.703167 12527 catalog_manager.cc:5697] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New cstate: current_term: 1 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:52.707723 13734 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:52.709383 12777 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:52.712229 13724 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:52.721743 13734 raft_consensus.cc:2955] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.724043 12633 consensus_queue.cc:237] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:52.727444 13384 raft_consensus.cc:2955] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.731333 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8e0d0566acd7465f87e20053c75e28fa with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:52.740846 12624 tablet_service.cc:1558] Processing DeleteTablet for tablet 866523bf2d2f442f971d504267ed200f with delete_type TABLET_DATA_TOMBSTONED (TS 19cd864c053744bebd96093a7982537e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56830
I20260709 13:17:52.747432 13382 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:52.752131 13630 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:52.755059 12778 raft_consensus.cc:2955] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.751294 12531 catalog_manager.cc:5697] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:52.763698 13719 raft_consensus.cc:2955] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.772001 12620 tablet_service.cc:1558] Processing DeleteTablet for tablet 8e0d0566acd7465f87e20053c75e28fa with delete_type TABLET_DATA_TOMBSTONED (TS 19cd864c053744bebd96093a7982537e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56830
I20260709 13:17:52.775754 13382 raft_consensus.cc:2955] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.791113 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9155962ca7f34a8c83a1780ce0018a59 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:52.794031 12778 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:52.801705 13746 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:52.804366 13752 tablet_replica.cc:333] T 866523bf2d2f442f971d504267ed200f P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:17:52.805739 13752 raft_consensus.cc:2243] T 866523bf2d2f442f971d504267ed200f P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:52.805284 12531 catalog_manager.cc:5697] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:52.806742 13752 raft_consensus.cc:2272] T 866523bf2d2f442f971d504267ed200f P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:52.808353 12778 raft_consensus.cc:2955] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:52.811666 13752 ts_tablet_manager.cc:1916] T 866523bf2d2f442f971d504267ed200f P 19cd864c053744bebd96093a7982537e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:52.839672 13752 ts_tablet_manager.cc:1929] T 866523bf2d2f442f971d504267ed200f P 19cd864c053744bebd96093a7982537e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:52.840093 13752 log.cc:1199] T 866523bf2d2f442f971d504267ed200f P 19cd864c053744bebd96093a7982537e: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-0-root/wals/866523bf2d2f442f971d504267ed200f
I20260709 13:17:52.842053 13752 tablet_replica.cc:333] T 8e0d0566acd7465f87e20053c75e28fa P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:17:52.843021 13752 raft_consensus.cc:2243] T 8e0d0566acd7465f87e20053c75e28fa P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:52.843667 13752 raft_consensus.cc:2272] T 8e0d0566acd7465f87e20053c75e28fa P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:52.851987 13752 ts_tablet_manager.cc:1916] T 8e0d0566acd7465f87e20053c75e28fa P 19cd864c053744bebd96093a7982537e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:52.858703 12517 catalog_manager.cc:5028] TS 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): tablet 866523bf2d2f442f971d504267ed200f (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:17:52.865880 12695 tablet_service.cc:1558] Processing DeleteTablet for tablet 9155962ca7f34a8c83a1780ce0018a59 with delete_type TABLET_DATA_TOMBSTONED (TS 639ab2db265f4747bb7afb93e833f530 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42474
I20260709 13:17:52.870209 13752 ts_tablet_manager.cc:1929] T 8e0d0566acd7465f87e20053c75e28fa P 19cd864c053744bebd96093a7982537e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:52.870764 13752 log.cc:1199] T 8e0d0566acd7465f87e20053c75e28fa P 19cd864c053744bebd96093a7982537e: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-0-root/wals/8e0d0566acd7465f87e20053c75e28fa
I20260709 13:17:52.874804 12517 catalog_manager.cc:5028] TS 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): tablet 8e0d0566acd7465f87e20053c75e28fa (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:17:52.879923 13753 tablet_replica.cc:333] T 9155962ca7f34a8c83a1780ce0018a59 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:17:52.881274 13753 raft_consensus.cc:2243] T 9155962ca7f34a8c83a1780ce0018a59 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:52.882121 13753 raft_consensus.cc:2272] T 9155962ca7f34a8c83a1780ce0018a59 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:52.886210 13753 ts_tablet_manager.cc:1916] T 9155962ca7f34a8c83a1780ce0018a59 P 639ab2db265f4747bb7afb93e833f530: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:52.902879 13753 ts_tablet_manager.cc:1929] T 9155962ca7f34a8c83a1780ce0018a59 P 639ab2db265f4747bb7afb93e833f530: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:52.903240 13753 log.cc:1199] T 9155962ca7f34a8c83a1780ce0018a59 P 639ab2db265f4747bb7afb93e833f530: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-1-root/wals/9155962ca7f34a8c83a1780ce0018a59
I20260709 13:17:52.905622 12517 catalog_manager.cc:5028] TS 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): tablet 9155962ca7f34a8c83a1780ce0018a59 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:17:53.938381 12626 consensus_queue.cc:237] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:53.946174 12702 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:17:53.946374 12597 consensus_peers.cc:597] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5febc2df9d3c41359ceccc1bef8a54d8. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:53.946189 12783 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:53.947652 13755 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:53.948625 13719 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:53.957021 13746 raft_consensus.cc:2955] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:53.958858 12783 raft_consensus.cc:2955] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:53.959141 12702 raft_consensus.cc:2955] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:53.979461 12529 catalog_manager.cc:5697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:53.984581 12702 consensus_queue.cc:237] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
W20260709 13:17:53.995330 12672 consensus_peers.cc:597] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: eb7fca96394f4ba9b4fe3d7a6e278878. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:53.997200 12783 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:53.997576 12626 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:53.999292 13659 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:54.000211 13756 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:54.014833 13659 raft_consensus.cc:2955] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:54.018595 12783 raft_consensus.cc:2955] T eb7fca96394f4ba9b4fe3d7a6e278878 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:54.019260 12626 raft_consensus.cc:2955] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:54.030877 12633 consensus_queue.cc:237] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:54.036545 12527 catalog_manager.cc:5697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:54.053192 12709 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:17:54.056576 12597 consensus_peers.cc:597] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb2fc32fdf0a4cd981b02de0a47a7f96. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:54.057906 12783 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:54.059604 13719 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:54.060968 13746 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:54.068563 13746 raft_consensus.cc:2955] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:54.070395 12709 raft_consensus.cc:2955] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:54.072209 12783 raft_consensus.cc:2955] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:54.085291 12702 consensus_queue.cc:237] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:54.088754 12531 catalog_manager.cc:5697] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
W20260709 13:17:54.114044 12672 consensus_peers.cc:597] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 522bcf0a155f4da196cb33f1aae90a82. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:54.117591 12783 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:54.119176 13659 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:54.130374 13734 raft_consensus.cc:2955] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:54.136431 12626 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:54.138753 12783 raft_consensus.cc:2955] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:54.153838 12531 catalog_manager.cc:5697] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:54.161458 13659 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:54.185714 12626 raft_consensus.cc:2955] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:54.383460 13776 ts_tablet_manager.cc:933] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:17:54.385430 13776 tablet_copy_client.cc:323] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.65:43603
I20260709 13:17:54.386947 12644 tablet_copy_service.cc:140] P 19cd864c053744bebd96093a7982537e: Received BeginTabletCopySession request for tablet 5febc2df9d3c41359ceccc1bef8a54d8 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:39590)
I20260709 13:17:54.387473 12644 tablet_copy_service.cc:161] P 19cd864c053744bebd96093a7982537e: Beginning new tablet copy session on tablet 5febc2df9d3c41359ceccc1bef8a54d8 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:39590: session id = 522fd2ba0e9f402691b5ef29af8579cd-5febc2df9d3c41359ceccc1bef8a54d8
I20260709 13:17:54.397200 12644 tablet_copy_source_session.cc:215] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:54.400444 13776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5febc2df9d3c41359ceccc1bef8a54d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:54.415278 13776 tablet_copy_client.cc:806] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:54.415853 13776 tablet_copy_client.cc:670] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:54.419759 13776 tablet_copy_client.cc:538] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:54.428416 13776 tablet_bootstrap.cc:492] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:54.448071 13776 tablet_bootstrap.cc:492] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:54.449328 13776 tablet_bootstrap.cc:492] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:17:54.450085 13776 ts_tablet_manager.cc:1403] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.022s	user 0.022s	sys 0.001s
I20260709 13:17:54.453012 13776 raft_consensus.cc:359] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:54.453898 13776 raft_consensus.cc:740] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:17:54.454648 13776 consensus_queue.cc:260] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:54.479450 13779 ts_tablet_manager.cc:933] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:17:54.493510 13779 tablet_copy_client.cc:323] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.65:43603
I20260709 13:17:54.494062 13776 ts_tablet_manager.cc:1434] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.044s	user 0.026s	sys 0.013s
I20260709 13:17:54.495744 12644 tablet_copy_service.cc:140] P 19cd864c053744bebd96093a7982537e: Received BeginTabletCopySession request for tablet bb2fc32fdf0a4cd981b02de0a47a7f96 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:39590)
I20260709 13:17:54.496299 12644 tablet_copy_service.cc:161] P 19cd864c053744bebd96093a7982537e: Beginning new tablet copy session on tablet bb2fc32fdf0a4cd981b02de0a47a7f96 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:39590: session id = 522fd2ba0e9f402691b5ef29af8579cd-bb2fc32fdf0a4cd981b02de0a47a7f96
I20260709 13:17:54.497548 12643 tablet_copy_service.cc:342] P 19cd864c053744bebd96093a7982537e: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-5febc2df9d3c41359ceccc1bef8a54d8 received from {username='slave'} at 127.0.0.1:39590
I20260709 13:17:54.497987 12643 tablet_copy_service.cc:434] P 19cd864c053744bebd96093a7982537e: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-5febc2df9d3c41359ceccc1bef8a54d8 on tablet 5febc2df9d3c41359ceccc1bef8a54d8 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:54.503661 12644 tablet_copy_source_session.cc:215] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:54.506914 13779 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb2fc32fdf0a4cd981b02de0a47a7f96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:54.525465 13779 tablet_copy_client.cc:806] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:54.526136 13779 tablet_copy_client.cc:670] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:54.530232 13779 tablet_copy_client.cc:538] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:54.536098 13776 ts_tablet_manager.cc:933] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:54.541659 13779 tablet_bootstrap.cc:492] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:54.545080 13776 tablet_copy_client.cc:323] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.66:45173
I20260709 13:17:54.547004 12718 tablet_copy_service.cc:140] P 639ab2db265f4747bb7afb93e833f530: Received BeginTabletCopySession request for tablet eb7fca96394f4ba9b4fe3d7a6e278878 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:38496)
I20260709 13:17:54.547559 12718 tablet_copy_service.cc:161] P 639ab2db265f4747bb7afb93e833f530: Beginning new tablet copy session on tablet eb7fca96394f4ba9b4fe3d7a6e278878 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:38496: session id = 522fd2ba0e9f402691b5ef29af8579cd-eb7fca96394f4ba9b4fe3d7a6e278878
I20260709 13:17:54.555439 12718 tablet_copy_source_session.cc:215] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:54.558790 13776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb7fca96394f4ba9b4fe3d7a6e278878. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:54.561084 13779 tablet_bootstrap.cc:492] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:54.562011 13779 tablet_bootstrap.cc:492] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:17:54.562644 13779 ts_tablet_manager.cc:1403] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.011s
I20260709 13:17:54.565657 13779 raft_consensus.cc:359] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:54.566458 13779 raft_consensus.cc:740] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:17:54.567095 13779 consensus_queue.cc:260] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:54.569522 13779 ts_tablet_manager.cc:1434] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.001s
I20260709 13:17:54.573877 12644 tablet_copy_service.cc:342] P 19cd864c053744bebd96093a7982537e: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-bb2fc32fdf0a4cd981b02de0a47a7f96 received from {username='slave'} at 127.0.0.1:39590
I20260709 13:17:54.574327 12644 tablet_copy_service.cc:434] P 19cd864c053744bebd96093a7982537e: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-bb2fc32fdf0a4cd981b02de0a47a7f96 on tablet bb2fc32fdf0a4cd981b02de0a47a7f96 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:54.575863 13776 tablet_copy_client.cc:806] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:54.576493 13776 tablet_copy_client.cc:670] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:54.585067 13776 tablet_copy_client.cc:538] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:54.594689 13776 tablet_bootstrap.cc:492] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:54.616308 13776 tablet_bootstrap.cc:492] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:54.617205 13776 tablet_bootstrap.cc:492] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:17:54.617892 13776 ts_tablet_manager.cc:1403] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.023s	user 0.021s	sys 0.001s
I20260709 13:17:54.621021 13776 raft_consensus.cc:359] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:54.621892 13776 raft_consensus.cc:740] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:17:54.622637 13776 consensus_queue.cc:260] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:54.627838 13776 ts_tablet_manager.cc:1434] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:17:54.629721 12718 tablet_copy_service.cc:342] P 639ab2db265f4747bb7afb93e833f530: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-eb7fca96394f4ba9b4fe3d7a6e278878 received from {username='slave'} at 127.0.0.1:38496
I20260709 13:17:54.630160 12718 tablet_copy_service.cc:434] P 639ab2db265f4747bb7afb93e833f530: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-eb7fca96394f4ba9b4fe3d7a6e278878 on tablet eb7fca96394f4ba9b4fe3d7a6e278878 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:54.670821 13776 ts_tablet_manager.cc:933] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:54.672515 13776 tablet_copy_client.cc:323] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.66:45173
I20260709 13:17:54.673954 12718 tablet_copy_service.cc:140] P 639ab2db265f4747bb7afb93e833f530: Received BeginTabletCopySession request for tablet 522bcf0a155f4da196cb33f1aae90a82 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:38496)
I20260709 13:17:54.674486 12718 tablet_copy_service.cc:161] P 639ab2db265f4747bb7afb93e833f530: Beginning new tablet copy session on tablet 522bcf0a155f4da196cb33f1aae90a82 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:38496: session id = 522fd2ba0e9f402691b5ef29af8579cd-522bcf0a155f4da196cb33f1aae90a82
I20260709 13:17:54.680933 12718 tablet_copy_source_session.cc:215] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:54.685608 13776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 522bcf0a155f4da196cb33f1aae90a82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:54.701781 13776 tablet_copy_client.cc:806] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:54.702319 13776 tablet_copy_client.cc:670] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:54.705965 13776 tablet_copy_client.cc:538] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:54.714711 13776 tablet_bootstrap.cc:492] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:54.740908 13776 tablet_bootstrap.cc:492] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:54.741828 13776 tablet_bootstrap.cc:492] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:17:54.743775 13776 ts_tablet_manager.cc:1403] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.008s
I20260709 13:17:54.751432 13776 raft_consensus.cc:359] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:54.752348 13776 raft_consensus.cc:740] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:17:54.752943 13776 consensus_queue.cc:260] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:54.754968 13776 ts_tablet_manager.cc:1434] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:17:54.758740 12718 tablet_copy_service.cc:342] P 639ab2db265f4747bb7afb93e833f530: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-522bcf0a155f4da196cb33f1aae90a82 received from {username='slave'} at 127.0.0.1:38496
I20260709 13:17:54.759212 12718 tablet_copy_service.cc:434] P 639ab2db265f4747bb7afb93e833f530: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-522bcf0a155f4da196cb33f1aae90a82 on tablet 522bcf0a155f4da196cb33f1aae90a82 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:54.829249 13385 raft_consensus.cc:1217] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:54.870541 13385 raft_consensus.cc:1217] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:55.009981 13385 raft_consensus.cc:1217] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:55.116289 13385 raft_consensus.cc:1217] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:55.351572 13746 raft_consensus.cc:1064] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:17:55.353494 13746 consensus_queue.cc:237] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:55.360146 12709 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:55.361264 12778 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:55.362738 13770 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:55.363312 13382 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:55.363935 13772 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:55.384392 13772 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:55.405880 13746 raft_consensus.cc:2955] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:55.408825 13382 raft_consensus.cc:2955] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:55.415364 12778 raft_consensus.cc:2955] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:55.422366 12702 raft_consensus.cc:2955] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:55.431864 12531 catalog_manager.cc:5697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:55.451633 13772 raft_consensus.cc:1064] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:17:55.454351 13772 consensus_queue.cc:237] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:55.456568 12626 consensus_queue.cc:237] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:55.472546 13382 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:55.472642 12702 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:55.474179 12778 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:55.475936 13382 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:55.476364 13770 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:55.477474 12702 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:55.477798 13790 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:55.478668 13770 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:55.481719 13746 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:55.483516 13746 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:55.491107 13385 raft_consensus.cc:2955] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:55.496512 13771 raft_consensus.cc:1064] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:17:55.489995 13790 raft_consensus.cc:2955] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:55.501870 13771 consensus_queue.cc:237] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:55.510234 12709 raft_consensus.cc:2955] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:55.509793 12527 catalog_manager.cc:5697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:55.527316 13659 raft_consensus.cc:1064] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:17:55.551064 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5febc2df9d3c41359ceccc1bef8a54d8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:55.582934 12773 tablet_service.cc:1558] Processing DeleteTablet for tablet 5febc2df9d3c41359ceccc1bef8a54d8 with delete_type TABLET_DATA_TOMBSTONED (TS 3d539f8d95d343d8b1696803f18b30d2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36472
I20260709 13:17:55.584575 13659 consensus_queue.cc:237] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:17:55.688477 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 91071.4us
I20260709 13:17:55.736660 12626 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:55.736691 12783 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:55.739292 13382 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:55.740594 13734 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:55.767760 13810 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:55.773344 13734 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:55.770735 13807 raft_consensus.cc:2955] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:55.784309 12776 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:17:55.799134 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 44094.3us
W20260709 13:17:55.811591 13803 log.cc:927] Time spent T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.061s	user 0.003s	sys 0.000s
I20260709 13:17:55.847698 13787 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:55.854871 12776 raft_consensus.cc:2955] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:55.862486 12633 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:55.865599 13384 raft_consensus.cc:2955] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:55.871806 13382 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:55.884763 12707 raft_consensus.cc:2955] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:55.901670 12529 catalog_manager.cc:5697] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:55.903592 13804 tablet_replica.cc:333] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:17:55.994061 13804 raft_consensus.cc:2243] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:55.994863 13804 raft_consensus.cc:2272] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:55.995916 13818 log.cc:927] Time spent T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:17:55.956756 13771 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:17:56.005827 13804 ts_tablet_manager.cc:1916] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:56.015714 13659 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:56.061231 12626 consensus_queue.cc:237] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:56.070721 12776 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:56.073843 13802 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:56.080879 13385 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 13:17:56.082020 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 166009us
I20260709 13:17:56.081388 13808 raft_consensus.cc:2955] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:56.086488 13821 raft_consensus.cc:2955] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:56.087917 13659 raft_consensus.cc:2955] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:56.101153 12776 raft_consensus.cc:2955] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:56.102488 12777 raft_consensus.cc:2955] T eb7fca96394f4ba9b4fe3d7a6e278878 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:56.110267 13385 raft_consensus.cc:2955] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:56.113464 13807 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:56.113287 12529 catalog_manager.cc:5697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:56.119752 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bb2fc32fdf0a4cd981b02de0a47a7f96 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:56.122368 12781 raft_consensus.cc:2955] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:56.123057 13379 raft_consensus.cc:2955] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:56.135975 13384 raft_consensus.cc:2955] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:56.150593 12527 catalog_manager.cc:5697] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index 3 to 4, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:56.172123 12528 catalog_manager.cc:5697] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:56.180119 13804 ts_tablet_manager.cc:1929] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:56.188853 13804 log.cc:1199] T 5febc2df9d3c41359ceccc1bef8a54d8 P 3d539f8d95d343d8b1696803f18b30d2: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-2-root/wals/5febc2df9d3c41359ceccc1bef8a54d8
I20260709 13:17:56.186805 12695 tablet_service.cc:1558] Processing DeleteTablet for tablet bb2fc32fdf0a4cd981b02de0a47a7f96 with delete_type TABLET_DATA_TOMBSTONED (TS 639ab2db265f4747bb7afb93e833f530 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:42474
I20260709 13:17:56.190822 12516 catalog_manager.cc:5028] TS 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): tablet 5febc2df9d3c41359ceccc1bef8a54d8 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:17:56.207316 12704 consensus_queue.cc:237] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:56.210790 12702 consensus_queue.cc:237] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:56.215842 13379 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:56.216429 12631 raft_consensus.cc:2955] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:56.217473 12626 raft_consensus.cc:2955] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:56.215847 13828 tablet_replica.cc:333] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:17:56.220060 13828 raft_consensus.cc:2243] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:56.224188 13828 raft_consensus.cc:2272] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:56.224318 13659 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:56.230916 13828 ts_tablet_manager.cc:1916] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:56.238106 12625 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:56.242928 13659 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:56.247838 13828 ts_tablet_manager.cc:1929] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:56.248412 13828 log.cc:1199] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 639ab2db265f4747bb7afb93e833f530: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-1-root/wals/bb2fc32fdf0a4cd981b02de0a47a7f96
I20260709 13:17:56.251950 13821 raft_consensus.cc:2955] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:56.255204 12781 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:56.258225 13379 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:56.258031 12625 raft_consensus.cc:2955] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:56.261446 12517 catalog_manager.cc:5028] TS 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): tablet bb2fc32fdf0a4cd981b02de0a47a7f96 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:17:56.265504 13821 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:56.267154 13659 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:56.267551 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet eb7fca96394f4ba9b4fe3d7a6e278878 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:56.267709 13384 raft_consensus.cc:2955] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:56.277827 13823 raft_consensus.cc:2955] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:56.281531 12531 catalog_manager.cc:5697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 3 to 4, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:56.296754 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 522bcf0a155f4da196cb33f1aae90a82 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:56.298291 12773 tablet_service.cc:1558] Processing DeleteTablet for tablet eb7fca96394f4ba9b4fe3d7a6e278878 with delete_type TABLET_DATA_TOMBSTONED (TS 3d539f8d95d343d8b1696803f18b30d2 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36472
I20260709 13:17:56.298004 12777 raft_consensus.cc:2955] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:56.299242 13804 tablet_replica.cc:333] T eb7fca96394f4ba9b4fe3d7a6e278878 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:17:56.300604 13804 raft_consensus.cc:2243] T eb7fca96394f4ba9b4fe3d7a6e278878 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:56.301378 13804 raft_consensus.cc:2272] T eb7fca96394f4ba9b4fe3d7a6e278878 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:56.307096 13804 ts_tablet_manager.cc:1916] T eb7fca96394f4ba9b4fe3d7a6e278878 P 3d539f8d95d343d8b1696803f18b30d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:56.322127 13379 raft_consensus.cc:2955] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:17:56.334542 12527 catalog_manager.cc:5697] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 3 to 4, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:56.352942 13804 ts_tablet_manager.cc:1929] T eb7fca96394f4ba9b4fe3d7a6e278878 P 3d539f8d95d343d8b1696803f18b30d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:56.353495 13804 log.cc:1199] T eb7fca96394f4ba9b4fe3d7a6e278878 P 3d539f8d95d343d8b1696803f18b30d2: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-2-root/wals/eb7fca96394f4ba9b4fe3d7a6e278878
I20260709 13:17:56.355237 12516 catalog_manager.cc:5028] TS 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): tablet eb7fca96394f4ba9b4fe3d7a6e278878 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:17:56.356256 12624 tablet_service.cc:1558] Processing DeleteTablet for tablet 522bcf0a155f4da196cb33f1aae90a82 with delete_type TABLET_DATA_TOMBSTONED (TS 19cd864c053744bebd96093a7982537e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56830
I20260709 13:17:56.358072 13829 tablet_replica.cc:333] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:17:56.359009 13829 raft_consensus.cc:2243] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:56.359748 13829 raft_consensus.cc:2272] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:56.364166 13829 ts_tablet_manager.cc:1916] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:56.376811 13829 ts_tablet_manager.cc:1929] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:56.377173 13829 log.cc:1199] T 522bcf0a155f4da196cb33f1aae90a82 P 19cd864c053744bebd96093a7982537e: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-0-root/wals/522bcf0a155f4da196cb33f1aae90a82
I20260709 13:17:56.378712 12517 catalog_manager.cc:5028] TS 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): tablet 522bcf0a155f4da196cb33f1aae90a82 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:17:56.639526 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:56.700351 12704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:17:56.701000 12704 raft_consensus.cc:606] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Received request to transfer leadership to TS 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:56.716171 12631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:17:56.716734 12631 raft_consensus.cc:606] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:56.719318 12631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9132272ce4c847a99e395a53c1015230"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:17:56.719957 12631 raft_consensus.cc:606] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Received request to transfer leadership to TS 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:56.723803 12631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:17:56.724365 12631 raft_consensus.cc:606] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:56.727443 12626 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:17:56.727978 12626 raft_consensus.cc:606] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Received request to transfer leadership to TS 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:56.730832 12626 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:17:56.731317 12626 raft_consensus.cc:606] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:56.732713 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:17:56.733963 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:56.894443 13807 raft_consensus.cc:993] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e: : Instructing follower 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:17:56.896379 13816 raft_consensus.cc:1081] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Signalling peer 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:17:56.898284 13384 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9"
dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:51754
I20260709 13:17:56.898947 13384 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:56.899312 13384 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:56.909811 13384 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:56.912110 13384 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:56.912781 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:56.913544 12631 raft_consensus.cc:3055] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:56.913540 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:56.914113 12631 raft_consensus.cc:740] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:56.914290 12781 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:56.914932 12631 consensus_queue.cc:260] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:56.916431 12631 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:56.921639 12781 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:17:56.923136 13348 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:17:56.924320 12631 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:17:56.925990 13834 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:56.928205 13834 raft_consensus.cc:697] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:17:56.929076 13834 consensus_queue.cc:237] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:56.929903 13809 raft_consensus.cc:993] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e: : Instructing follower 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:17:56.930467 13807 raft_consensus.cc:1081] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Signalling peer 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:17:56.932070 12781 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:36488
I20260709 13:17:56.932627 12781 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:56.932984 12781 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:56.938110 12781 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:56.938716 12527 catalog_manager.cc:5697] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 1 to 2, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 2 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:56.940542 12781 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:56.940780 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:56.941476 12631 raft_consensus.cc:3055] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:56.941596 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:56.941856 12631 raft_consensus.cc:740] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:56.942157 12704 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:56.942565 12631 consensus_queue.cc:260] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:56.943722 12631 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:56.948417 12704 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:17:56.949857 12631 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:17:56.949941 12747 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:56.950778 13723 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:56.951212 13723 raft_consensus.cc:697] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:17:56.951969 13723 consensus_queue.cc:237] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:56.963876 12527 catalog_manager.cc:5697] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 1 to 2, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 2 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:57.015197 13816 raft_consensus.cc:993] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e: : Instructing follower 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:17:57.015717 13807 raft_consensus.cc:1081] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Signalling peer 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:17:57.017443 12777 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:36488
I20260709 13:17:57.017946 12777 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:57.018281 12777 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.018375 13787 raft_consensus.cc:993] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530: : Instructing follower 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:17:57.018997 13659 raft_consensus.cc:1081] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Signalling peer 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:17:57.020586 13384 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742"
dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:51762
I20260709 13:17:57.021178 13384 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:57.022401 13384 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.023887 12777 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:57.025362 12777 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:57.026190 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:57.026444 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:57.026773 12626 raft_consensus.cc:3055] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:57.027091 12704 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.027138 12626 raft_consensus.cc:740] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:57.027979 12626 consensus_queue.cc:260] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:57.028851 12626 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.032150 12704 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:17:57.033206 12626 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:17:57.033247 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:57.034288 13723 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:57.034858 13384 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:57.035601 13723 raft_consensus.cc:697] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:17:57.037333 13723 consensus_queue.cc:237] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:57.038460 13384 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:57.038910 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:57.039363 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:57.039534 12631 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.039909 12704 raft_consensus.cc:3055] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:57.040309 12704 raft_consensus.cc:740] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:57.041289 12704 consensus_queue.cc:260] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:57.042594 12704 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.048995 12631 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:17:57.052053 12704 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:17:57.053932 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:17:57.054994 13834 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:57.055526 13834 raft_consensus.cc:697] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:17:57.056514 13834 consensus_queue.cc:237] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:57.067057 12527 catalog_manager.cc:5697] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 1 to 2, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 2 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:57.067885 12529 catalog_manager.cc:5697] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 1 to 2, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 2 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:57.084422 13816 raft_consensus.cc:993] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e: : Instructing follower 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:17:57.084862 13807 raft_consensus.cc:1081] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Signalling peer 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:17:57.086602 12777 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:36488
I20260709 13:17:57.087108 12777 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:57.087409 12777 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.093740 12777 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:57.095196 12777 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:57.096084 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:57.096225 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:57.096740 12626 raft_consensus.cc:3055] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:57.096974 12704 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.097043 12626 raft_consensus.cc:740] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:57.097986 12626 consensus_queue.cc:260] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:57.098875 12626 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.102138 12704 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:17:57.103204 12747 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:17:57.103967 13723 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:57.104497 13723 raft_consensus.cc:697] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:17:57.105489 13723 consensus_queue.cc:237] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:57.105871 12626 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:17:57.120060 12529 catalog_manager.cc:5697] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 1 to 2, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 2 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:57.321368 13816 raft_consensus.cc:993] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e: : Instructing follower 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:17:57.321918 13807 raft_consensus.cc:1081] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Signalling peer 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:17:57.323431 13384 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9132272ce4c847a99e395a53c1015230"
dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:51754
I20260709 13:17:57.323863 13384 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:57.324193 13384 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.329116 13384 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:57.330789 13384 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:57.331544 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:57.331833 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:57.332198 12626 raft_consensus.cc:3055] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:57.332435 12781 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.332617 12626 raft_consensus.cc:740] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:57.335268 12626 consensus_queue.cc:260] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:57.336304 12626 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.337431 12781 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:17:57.338496 13348 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:17:57.339560 13834 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:57.339956 13834 raft_consensus.cc:697] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:17:57.340809 13834 consensus_queue.cc:237] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:57.341859 12626 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:17:57.348928 12529 catalog_manager.cc:5697] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 1 to 2, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 2 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:57.416575 12702 consensus_queue.cc:237] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:57.423509 12781 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:57.426862 12625 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:17:57.426932 12672 consensus_peers.cc:597] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4c13fa2aff324f65ad0bf6191ea68fab. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:57.428467 13834 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:57.429736 12625 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:57.431610 13847 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:57.435361 13659 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:57.459344 12776 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:57.461141 13834 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:57.462587 13847 raft_consensus.cc:2955] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:57.465131 12776 raft_consensus.cc:2955] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:57.482851 12626 raft_consensus.cc:2955] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:57.486074 12631 raft_consensus.cc:1275] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:57.487550 13723 consensus_queue.cc:1048] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:57.499526 12631 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:57.501510 13835 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:57.507532 12702 consensus_queue.cc:237] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:57.511464 12527 catalog_manager.cc:5697] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.524878 12702 raft_consensus.cc:1275] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:17:57.563963 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 77915.3us
I20260709 13:17:57.580559 12630 raft_consensus.cc:1275] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:57.587745 13859 consensus_queue.cc:1048] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:57.591765 13811 consensus_queue.cc:1048] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:17:57.595428 12672 consensus_peers.cc:597] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6609d5f93ab048558ea46e511ef3b2ec. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:57.601392 12776 raft_consensus.cc:1275] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:57.607438 13787 consensus_queue.cc:1048] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:57.609138 12704 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:57.618144 12625 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:57.621114 12704 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:57.622792 13862 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:57.624274 13862 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:57.626246 13834 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:57.655869 13659 raft_consensus.cc:2955] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:57.660066 12781 raft_consensus.cc:2955] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:57.662407 12630 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:57.674088 13811 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:57.712468 12632 raft_consensus.cc:2955] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:57.714319 12527 catalog_manager.cc:5697] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:57.753335 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:57.700362 12625 consensus_queue.cc:237] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
W20260709 13:17:57.797192 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 71982us
I20260709 13:17:57.817595 12704 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:57.821879 13875 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:57.859292 12634 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:57.860196 12776 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:57.885010 13834 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:17:57.907914 13835 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:57.962082 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:17:57.981112 12597 consensus_peers.cc:597] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 981fb2f435e24d06a81fc02cf4591cc5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:57.984084 13882 ts_tablet_manager.cc:933] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:57.987401 12777 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:57.989074 13807 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:58.001665 12701 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:58.005838 13809 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:58.052467 13877 raft_consensus.cc:2955] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:58.062309 12781 raft_consensus.cc:2955] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:58.205682 13882 tablet_copy_client.cc:323] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.66:45173
I20260709 13:17:58.287408 13887 ts_tablet_manager.cc:933] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:58.294052 13887 tablet_copy_client.cc:323] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.66:45173
I20260709 13:17:58.426800 12704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:17:58.427670 12704 raft_consensus.cc:606] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Received request to transfer leadership to TS 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:58.459030 12704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:17:58.459668 12704 raft_consensus.cc:606] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:17:58.464763 12704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:17:58.465436 12704 raft_consensus.cc:606] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Received request to transfer leadership to TS 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:17:58.564670 12704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:17:58.565542 12704 raft_consensus.cc:606] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:17:58.593194 12781 consensus_queue.cc:237] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:58.598294 12704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:17:58.598953 12704 raft_consensus.cc:606] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:17:58.606643 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:17:58.608201 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:17:58.623051 12746 consensus_peers.cc:597] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: af96e52b004d4c8c942cc0ec33c95c99. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:58.686489 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 264111us
I20260709 13:17:58.740175 12529 catalog_manager.cc:5697] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:58.747893 13896 raft_consensus.cc:993] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530: : Instructing follower 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:17:58.748441 13896 raft_consensus.cc:1081] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Signalling peer 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:17:58.766553 12700 raft_consensus.cc:2955] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:58.778522 12704 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:17:58.778939 12633 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:17:58.780432 12718 tablet_copy_service.cc:140] P 639ab2db265f4747bb7afb93e833f530: Received BeginTabletCopySession request for tablet 4c13fa2aff324f65ad0bf6191ea68fab from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:38496)
I20260709 13:17:58.780920 12719 tablet_copy_service.cc:140] P 639ab2db265f4747bb7afb93e833f530: Received BeginTabletCopySession request for tablet 6609d5f93ab048558ea46e511ef3b2ec from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:38496)
I20260709 13:17:58.781004 12718 tablet_copy_service.cc:161] P 639ab2db265f4747bb7afb93e833f530: Beginning new tablet copy session on tablet 4c13fa2aff324f65ad0bf6191ea68fab from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:38496: session id = 522fd2ba0e9f402691b5ef29af8579cd-4c13fa2aff324f65ad0bf6191ea68fab
I20260709 13:17:58.781742 12719 tablet_copy_service.cc:161] P 639ab2db265f4747bb7afb93e833f530: Beginning new tablet copy session on tablet 6609d5f93ab048558ea46e511ef3b2ec from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:38496: session id = 522fd2ba0e9f402691b5ef29af8579cd-6609d5f93ab048558ea46e511ef3b2ec
I20260709 13:17:58.793211 12718 tablet_copy_source_session.cc:215] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:58.797427 13895 raft_consensus.cc:993] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530: : Instructing follower 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:17:58.807387 13895 raft_consensus.cc:1081] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Signalling peer 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:17:58.804790 12778 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:36478
I20260709 13:17:58.807988 12778 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:58.808344 12778 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.816486 12634 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab"
dest_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:56832
I20260709 13:17:58.817085 13787 raft_consensus.cc:993] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530: : Instructing follower 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:17:58.817721 13787 raft_consensus.cc:1081] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Signalling peer 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:17:58.817087 12634 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:58.820533 12634 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.823268 13878 raft_consensus.cc:993] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530: : Instructing follower 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:17:58.823833 13878 raft_consensus.cc:1081] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Signalling peer 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:17:58.824107 13895 raft_consensus.cc:993] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530: : Instructing follower 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:17:58.824600 13895 raft_consensus.cc:1081] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Signalling peer 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:17:58.834163 12633 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa"
dest_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:56832
I20260709 13:17:58.833403 12634 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:58.836298 12719 tablet_copy_source_session.cc:215] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:58.834859 12633 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:58.839874 12633 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.838284 12634 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:58.846242 13862 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:58.848735 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:58.849328 12781 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.805162 13902 ts_tablet_manager.cc:933] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:17:58.856317 12633 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:58.859179 12633 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:58.863987 13882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c13fa2aff324f65ad0bf6191ea68fab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.867183 13862 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:58.881683 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:58.882313 12700 raft_consensus.cc:3055] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:58.882609 12700 raft_consensus.cc:740] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:58.882638 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:58.883481 12700 consensus_queue.cc:260] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:58.883211 12783 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.884943 12700 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.878669 12778 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:58.894541 13385 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98"
dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:51762
I20260709 13:17:58.922273 13385 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:58.922680 13385 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.923452 12777 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:58.895431 13380 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9"
dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:51762
I20260709 13:17:58.926292 13380 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:58.926755 13380 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.902860 13811 raft_consensus.cc:2955] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:58.900171 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:58.929867 12705 raft_consensus.cc:3055] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:58.930204 12705 raft_consensus.cc:740] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:58.931034 12705 consensus_queue.cc:260] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:58.932467 12705 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.940019 13380 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:58.957106 13787 consensus_queue.cc:1059] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:58.958539 13857 raft_consensus.cc:3055] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:58.958946 13857 raft_consensus.cc:740] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:58.959751 13857 consensus_queue.cc:260] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:58.961326 13857 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.968559 12705 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:17:58.971619 13385 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:58.893513 12783 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:17:58.910553 12700 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:17:58.980408 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:17:58.981238 12777 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.914928 12634 raft_consensus.cc:2955] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:58.915230 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:58.986253 12633 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.916433 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:58.912391 12778 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:58.994454 12633 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
W20260709 13:17:58.995009 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:58.919559 13887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6609d5f93ab048558ea46e511ef3b2ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.980502 12597 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:17:58.999388 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:59.001093 12707 raft_consensus.cc:3055] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:59.001462 12707 raft_consensus.cc:740] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:17:59.001633 13385 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:17:59.005322 12781 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:17:58.951437 13380 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:17:59.007432 12777 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:17:59.006749 12707 consensus_queue.cc:260] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:58.917866 12701 raft_consensus.cc:2955] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:59.013244 13807 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:59.013813 13902 tablet_copy_client.cc:323] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.65:43603
I20260709 13:17:59.014822 12597 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:17:59.017180 13348 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:17:59.037159 13889 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:59.032881 13807 raft_consensus.cc:697] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:59.046334 13807 consensus_queue.cc:237] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:59.050711 12704 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:17:59.064443 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:17:59.065088 12700 raft_consensus.cc:3055] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:59.040974 13835 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:59.067696 12707 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:59.014919 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:17:59.068243 12625 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:59.071715 13835 raft_consensus.cc:697] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:17:59.072731 13835 consensus_queue.cc:237] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:59.081362 12707 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:17:59.037966 13889 raft_consensus.cc:697] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:17:59.065457 12700 raft_consensus.cc:740] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:17:59.128341 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 66323.7us
I20260709 13:17:59.135389 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:17:59.132500 12625 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:17:59.149536 13863 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:59.150527 13889 consensus_queue.cc:237] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:17:59.143661 12700 consensus_queue.cc:260] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:17:59.210361 12700 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:59.207360 13863 raft_consensus.cc:697] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:17:59.215413 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:17:59.213289 13863 consensus_queue.cc:237] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:17:59.232662 12644 tablet_copy_service.cc:140] P 19cd864c053744bebd96093a7982537e: Received BeginTabletCopySession request for tablet 981fb2f435e24d06a81fc02cf4591cc5 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:39590)
I20260709 13:17:59.234956 12644 tablet_copy_service.cc:161] P 19cd864c053744bebd96093a7982537e: Beginning new tablet copy session on tablet 981fb2f435e24d06a81fc02cf4591cc5 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:39590: session id = 522fd2ba0e9f402691b5ef29af8579cd-981fb2f435e24d06a81fc02cf4591cc5
I20260709 13:17:59.222402 13835 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:59.345546 12700 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:17:59.349225 12644 tablet_copy_source_session.cc:215] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:59.352944 13882 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:51762 (request call id 312) took 1378 ms. Trace:
W20260709 13:17:59.368455 13407 debug-util.cc:398] Leaking SignalData structure 0x7b08002f5060 after lost signal to thread 11520
W20260709 13:17:59.369679 13407 debug-util.cc:398] Leaking SignalData structure 0x7b08002f5740 after lost signal to thread 12566
I20260709 13:17:59.368599 13882 rpcz_store.cc:276] 0709 13:17:57.974261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.974360 (+    99us) service_pool.cc:224] Handling call
0709 13:17:57.984444 (+ 10084us) ts_tablet_manager.cc:934] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
0709 13:17:58.862763 (+878319us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:17:59.128259 (+265496us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:59.128403 (+   144us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:17:59.352218 (+223815us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:17:59.352504 (+   286us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:59.352800 (+   296us) ts_tablet_manager.cc:989] Replica registered.
0709 13:17:59.352855 (+    55us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":337,"tablet-copy.queue_time_us":9226,"thread_start_us":229318,"threads_started":2}
W20260709 13:17:59.370806 13407 debug-util.cc:398] Leaking SignalData structure 0x7b0800357840 after lost signal to thread 12658
I20260709 13:17:59.359730 12531 catalog_manager.cc:5697] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 1 to 2, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 2 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:17:59.371887 13407 debug-util.cc:398] Leaking SignalData structure 0x7b0800337da0 after lost signal to thread 12733
W20260709 13:17:59.372884 13407 debug-util.cc:398] Leaking SignalData structure 0x7b0800330d00 after lost signal to thread 12807
W20260709 13:17:59.373903 13407 debug-util.cc:398] Leaking SignalData structure 0x7b08003465c0 after lost signal to thread 13410
W20260709 13:17:59.407644 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 100704us
W20260709 13:17:59.408838 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 91957.5us
I20260709 13:17:59.377754 13882 tablet_copy_client.cc:806] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
W20260709 13:17:59.424134 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 155450us
I20260709 13:17:59.413264 13835 raft_consensus.cc:697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:17:59.445163 13887 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:51762 (request call id 315) took 1363 ms. Trace:
I20260709 13:17:59.451164 13916 ts_tablet_manager.cc:933] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:17:59.452771 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:59.448109 13882 tablet_copy_client.cc:670] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:59.463450 13887 rpcz_store.cc:276] 0709 13:17:58.077228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.077387 (+   159us) service_pool.cc:224] Handling call
0709 13:17:58.287922 (+210535us) ts_tablet_manager.cc:934] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
0709 13:17:58.917821 (+629899us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:17:59.348981 (+431160us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:59.349116 (+   135us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:17:59.373049 (+ 23933us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:17:59.373385 (+   336us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:59.422460 (+ 49075us) ts_tablet_manager.cc:989] Replica registered.
0709 13:17:59.440536 (+ 18076us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":669,"tablet-copy.queue_time_us":209376,"thread_start_us":292847,"threads_started":2}
I20260709 13:17:59.474413 13887 tablet_copy_client.cc:806] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:59.475024 12529 catalog_manager.cc:5697] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index -1 to 3, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 2 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:17:59.478012 13887 tablet_copy_client.cc:670] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:59.496546 13902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 981fb2f435e24d06a81fc02cf4591cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.500598 13916 tablet_copy_client.cc:323] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.67:41181
I20260709 13:17:59.510293 12793 tablet_copy_service.cc:140] P 3d539f8d95d343d8b1696803f18b30d2: Received BeginTabletCopySession request for tablet af96e52b004d4c8c942cc0ec33c95c99 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:54346)
I20260709 13:17:59.517483 12793 tablet_copy_service.cc:161] P 3d539f8d95d343d8b1696803f18b30d2: Beginning new tablet copy session on tablet af96e52b004d4c8c942cc0ec33c95c99 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:54346: session id = 522fd2ba0e9f402691b5ef29af8579cd-af96e52b004d4c8c942cc0ec33c95c99
W20260709 13:17:59.562279 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 45785.3us
I20260709 13:17:59.507184 13835 consensus_queue.cc:237] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:17:59.567377 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 48791.9us
W20260709 13:17:59.567853 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 50773.8us
I20260709 13:17:59.572536 12783 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:59.583170 12702 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:17:59.582330 13904 log.cc:927] Time spent T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:17:59.585134 13901 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
W20260709 13:17:59.642066 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 49058.4us
I20260709 13:17:59.642346 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:59.644002 13917 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
W20260709 13:17:59.650235 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 91195.1us
I20260709 13:17:59.660396 13887 tablet_copy_client.cc:538] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:59.669996 13887 tablet_bootstrap.cc:492] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:59.682163 12793 tablet_copy_source_session.cc:215] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:59.690836 13916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af96e52b004d4c8c942cc0ec33c95c99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.693230 12776 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:17:59.730329 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 78396us
I20260709 13:17:59.701819 13807 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:17:59.783078 12776 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:17:59.782845 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 64293.8us
I20260709 13:17:59.787528 12527 catalog_manager.cc:5697] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 1 to 2, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 2 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:59.782582 13882 tablet_copy_client.cc:538] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:59.795609 13877 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:59.799486 12531 catalog_manager.cc:5697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 1 to 2, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 2 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:59.820607 12702 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:59.864411 13902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:51754 (request call id 231) took 1225 ms. Trace:
I20260709 13:17:59.879269 12528 catalog_manager.cc:5697] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 1 to 2, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 2 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:59.882371 12528 catalog_manager.cc:5697] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 1 to 2, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 2 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:59.887694 12704 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:59.890759 13933 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:17:59.892710 12701 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:17:59.900409 13909 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
W20260709 13:17:59.905853 12597 consensus_peers.cc:597] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:59.920142 13918 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:17:59.866523 13902 rpcz_store.cc:276] 0709 13:17:58.639488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:58.640849 (+  1361us) service_pool.cc:224] Handling call
0709 13:17:58.852428 (+211579us) ts_tablet_manager.cc:934] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
0709 13:17:59.469417 (+616989us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:17:59.668082 (+198665us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:59.668211 (+   129us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:17:59.820061 (+151850us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:17:59.863928 (+ 43867us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:59.864278 (+   350us) ts_tablet_manager.cc:989] Replica registered.
0709 13:17:59.864340 (+    62us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":19236,"tablet-copy.queue_time_us":152750,"thread_start_us":278890,"threads_started":2}
I20260709 13:17:59.945681 12626 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:59.947978 13902 tablet_copy_client.cc:806] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:59.958072 13899 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:17:59.977200 13882 tablet_bootstrap.cc:492] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:17:59.967479 13902 tablet_copy_client.cc:670] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:00.097779 13916 tablet_copy_client.cc:806] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:00.098666 13916 tablet_copy_client.cc:670] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:00.177527 13902 tablet_copy_client.cc:538] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:00.210577 12631 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:00.221745 13921 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:18:00.238539 13887 tablet_bootstrap.cc:492] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:00.249707 12632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:56830
I20260709 13:18:00.250983 12632 raft_consensus.cc:606] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Received request to transfer leadership
I20260709 13:18:00.265162 13887 tablet_bootstrap.cc:492] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:18:00.282521 13918 raft_consensus.cc:993] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e: : Instructing follower 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:18:00.283456 13918 raft_consensus.cc:1081] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Signalling peer 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:18:00.287657 13916 tablet_copy_client.cc:538] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:00.292945 13902 tablet_bootstrap.cc:492] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:18:00.295096 12631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:56830
I20260709 13:18:00.303297 12701 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:00.312767 12631 raft_consensus.cc:606] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Received request to transfer leadership
W20260709 13:18:00.345536 12517 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 4c13fa2aff324f65ad0bf6191ea68fab already in progress [suppressed 3 similar messages]
I20260709 13:18:00.387411 13887 ts_tablet_manager.cc:1403] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.718s	user 0.022s	sys 0.016s
I20260709 13:18:00.416092 13835 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:00.446789 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:18:00.449738 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.StartTabletCopy -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 50428.2us
I20260709 13:18:00.481807 13887 raft_consensus.cc:359] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:00.482941 13887 raft_consensus.cc:740] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:18:00.483788 13887 consensus_queue.cc:260] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:00.491561 12774 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:36488
I20260709 13:18:00.492410 12774 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:00.492894 12774 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:00.513496 12626 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:56830
I20260709 13:18:00.514575 12626 raft_consensus.cc:606] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Received request to transfer leadership
W20260709 13:18:00.525400 12579 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:18:00.525939 12579 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 4c13fa2aff324f65ad0bf6191ea68fab already in progress
W20260709 13:18:00.526258 12579 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 4c13fa2aff324f65ad0bf6191ea68fab already in progress
I20260709 13:18:00.528247 13887 ts_tablet_manager.cc:1434] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.119s	user 0.009s	sys 0.001s
I20260709 13:18:00.532017 13916 tablet_bootstrap.cc:492] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:18:00.542338 12774 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:00.543744 12778 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 2. Current term is 3. Ops: []
W20260709 13:18:00.553666 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 102793us
I20260709 13:18:00.555140 12774 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:00.563563 13882 tablet_bootstrap.cc:492] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:01.761876 13962 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:01.762555 13962 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:01.776018 13962 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:01.776702 13962 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:01.777303 13962 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:01.779654 13962 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:01.797523 13807 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:01.802901 13807 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:01.805298 13807 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:01.844015 13963 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:01.844712 13963 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:01.845892 13962 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:01.846553 13962 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:01.847043 13963 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:01.847682 13348 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.857s	user 0.008s	sys 0.037s
W20260709 13:18:01.848006 13348 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.008s	sys 0.037s
I20260709 13:18:01.850103 13961 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:01.850229 13960 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:01.850708 13961 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:01.850986 13962 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:01.851047 13960 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:01.854218 13961 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:01.854434 13960 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:01.855264 13959 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:01.862210 13959 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:01.865221 13959 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:01.870146 13964 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:01.870952 13964 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:01.871096 13807 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:01.872157 13807 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:01.873955 13964 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:01.874393 13807 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:01.876716 13964 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:01.877538 13964 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:01.880023 13964 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:01.932142 13923 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:01.932603 13967 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:01.932713 13923 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:01.933398 13924 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:01.933473 13967 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:01.935393 13923 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:01.936755 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:01.937225 13967 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:01.937418 12776 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 1.
I20260709 13:18:01.934708 13924 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:01.942726 13924 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:01.956028 13351 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.938s	user 0.042s	sys 0.028s
W20260709 13:18:01.956342 13351 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.938s	user 0.042s	sys 0.028s
I20260709 13:18:01.957466 13961 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:01.958132 13961 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:01.958609 13962 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:01.959142 13962 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:01.960413 13961 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:01.960966 13962 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:01.962786 13960 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:01.962580 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:01.963713 13385 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:18:01.963428 13960 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:01.961820 13921 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:01.964682 13921 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:01.964962 12597 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:01.966001 13961 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:01.966627 13961 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:01.967008 13962 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:01.967113 13960 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:01.967545 13962 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:01.967970 13962 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:01.968963 13959 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:01.970712 13961 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:18:01.971558 13961 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:01.970495 13960 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:01.972139 13961 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:01.971514 13959 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:01.974990 13959 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:01.972234 13960 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:01.975338 13961 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:01.976480 13962 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:01.978169 13960 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:01.979764 13962 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:01.986213 13967 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:01.987136 13967 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:01.988709 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:01.990522 13969 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:01.990976 13385 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:01.991250 13348 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:01.991096 13969 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:01.992343 13966 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:01.989635 13967 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:01.993400 13969 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:01.993896 13968 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:01.994436 13968 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:01.996536 13968 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:01.997138 13964 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:01.997778 13969 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:01.998114 13964 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:01.998347 13969 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.000358 13964 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:02.000849 13964 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:02.002449 13963 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:01.989883 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:02.002959 13963 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.000396 13969 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:02.003533 13379 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:18:02.004436 13385 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:02.004487 13960 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.005003 13963 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:02.005236 13960 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.001353 13964 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:02.008009 13872 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:02.008548 13872 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.010582 13872 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:02.010792 13902 tablet_bootstrap.cc:492] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:02.010860 13963 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:02.011538 13963 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.011960 13902 tablet_bootstrap.cc:492] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:18:02.012698 13902 ts_tablet_manager.cc:1403] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 1.720s	user 0.030s	sys 0.005s
I20260709 13:18:02.014317 13963 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:02.025528 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:02.028146 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:01.992841 13966 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.028811 12632 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:18:02.028906 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:02.029656 12634 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:18:02.030666 13966 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:02.034209 13924 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:02.034390 13807 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:02.034719 13924 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:02.035000 13807 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.036820 13924 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:02.037106 13807 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:02.038357 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:02.039023 12776 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 1.
I20260709 13:18:02.042337 13348 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:02.042466 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:02.043121 12777 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 1.
I20260709 13:18:02.018991 13882 tablet_bootstrap.cc:492] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:18:02.020900 13971 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:02.022679 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:02.044613 12626 raft_consensus.cc:3055] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:18:02.030601 13349 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.052s	user 0.003s	sys 0.024s
I20260709 13:18:02.044936 12626 raft_consensus.cc:740] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:18:02.045086 13349 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.067s	user 0.004s	sys 0.025s
I20260709 13:18:02.023564 13964 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:02.024230 13962 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:02.049228 13962 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.051174 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:02.043148 13961 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.030740 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:02.016671 13902 raft_consensus.cc:359] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:02.044289 13882 ts_tablet_manager.cc:1403] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 2.067s	user 0.037s	sys 0.031s
I20260709 13:18:02.043892 13971 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:02.048292 12626 consensus_queue.cc:260] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:02.056013 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:02.056305 12626 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:02.051363 13962 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:02.041625 13963 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:02.053867 13961 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.057811 13961 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.054580 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:02.005702 13960 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.058225 13959 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:02.053763 13964 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:02.065821 13964 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:02.068341 13964 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:02.071691 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:02.072587 13379 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:18:02.072786 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:02.073660 13382 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:18:02.073698 13963 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.076210 13963 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:02.055123 13902 raft_consensus.cc:740] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:18:02.056663 13385 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:18:02.060822 13807 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:02.078512 13971 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:02.088714 12626 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:02.090072 13807 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:02.037271 13924 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:02.063548 13872 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:02.077278 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:02.078835 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:02.079788 13835 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.087481 13964 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:02.077401 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:02.083793 13966 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:02.080658 13959 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:02.109337 13723 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:02.116564 12747 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:02.169389 13807 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:02.164053 13723 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
W20260709 13:18:02.171298 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 109601us
I20260709 13:18:02.172017 12747 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:02.106591 13971 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:02.118912 13959 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:02.079937 13960 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:02.102527 13924 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.116600 13899 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.103788 13872 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.194346 13872 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:02.195250 13872 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.195879 13872 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.199151 13872 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:02.108553 12625 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:18:02.162673 13963 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.200006 13872 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.200977 13872 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.179548 13807 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:02.202026 13807 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:02.203052 13960 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:02.204510 13807 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:02.166069 13923 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.205602 13923 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.208371 13923 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:02.209334 13890 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:02.209729 13723 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:02.108898 13966 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:02.107400 13964 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.104947 13835 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:02.200579 13963 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.111073 13961 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:02.219511 13872 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.224946 13960 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.235550 13960 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:02.236006 13924 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:02.225684 13976 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.209971 13890 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.105824 13902 consensus_queue.cc:260] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:02.188974 13968 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.233189 12747 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:02.210810 13969 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.305433 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:02.333494 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 100387us
I20260709 13:18:02.335773 13807 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.318979 12531 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54792 (request call id 221) took 1784 ms. Trace:
I20260709 13:18:02.333811 12628 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:18:02.210669 13967 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.218036 13899 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.237547 13960 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:02.236538 13964 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:02.332300 13961 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:02.303689 13963 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:02.349404 13807 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.352031 12747 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:02.356418 13723 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.334530 13962 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:02.356748 13723 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.358887 13723 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.249298 13966 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:02.302264 13978 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:02.223721 13971 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:02.334322 13969 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.247717 13835 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:02.242120 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:02.358603 13348 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:02.373979 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:02.358641 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:02.318830 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:02.371362 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:02.371804 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:02.352720 13968 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.335726 13976 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:02.355284 13882 raft_consensus.cc:359] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:02.374811 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:02.380786 13961 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.482527 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:02.482678 12632 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
W20260709 13:18:02.482970 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:02.451750 13921 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:02.336580 12531 rpcz_store.cc:276] 0709 13:18:00.534548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:00.535650 (+  1102us) service_pool.cc:224] Handling call
0709 13:18:02.318711 (+1783061us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":5}
I20260709 13:18:02.381057 13807 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:02.489177 13723 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.364919 13964 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.481392 13902 ts_tablet_manager.cc:1434] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.468s	user 0.024s	sys 0.007s
I20260709 13:18:02.498327 13723 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:02.499863 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:02.500536 12628 raft_consensus.cc:2393] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 2: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:02.461936 13981 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:02.474704 13971 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:02.359716 13967 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.367206 13962 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:02.461172 13966 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:02.462690 13972 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.463480 13890 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:02.484169 13882 raft_consensus.cc:740] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:18:02.468961 13899 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:02.479422 13872 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.489346 13961 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:02.501591 13923 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:02.370419 13969 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.512207 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:02.520117 12628 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 1.
I20260709 13:18:02.503516 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:02.372325 13983 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.423166 13978 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:02.449954 13982 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.497128 12625 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:18:02.518354 13976 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:02.525104 13982 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.520488 13961 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.544212 13872 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:02.548058 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:02.549161 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:02.549777 12628 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 1.
I20260709 13:18:02.521761 13986 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.548758 12627 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 1.
I20260709 13:18:02.531742 13972 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.524133 13981 raft_consensus.cc:697] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:02.554373 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:02.517733 13964 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.536645 13924 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.522049 13966 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.554983 12628 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 1.
I20260709 13:18:02.556540 13972 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:02.526405 13899 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.523396 13923 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.591655 13899 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:02.554029 13973 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:02.598057 13988 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.598163 13973 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:02.547375 13983 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:02.618757 13961 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:02.559590 13924 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.624559 13978 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:02.625099 13964 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:02.629103 13984 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 13:18:02.630743 13961 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:02.631284 13923 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:02.536314 13882 consensus_queue.cc:260] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:02.635317 13882 ts_tablet_manager.cc:1434] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.580s	user 0.007s	sys 0.011s
W20260709 13:18:02.635700 13882 ts_tablet_manager.cc:1467] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Tablet startup took 580ms
I20260709 13:18:02.633242 13916 tablet_bootstrap.cc:492] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:02.598850 13988 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.592139 13986 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.591764 13981 consensus_queue.cc:237] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:02.625967 13924 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.723423 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:02.642201 13983 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:02.727605 13916 tablet_bootstrap.cc:492] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:18:02.729154 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:02.730782 12644 tablet_copy_service.cc:342] P 19cd864c053744bebd96093a7982537e: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-981fb2f435e24d06a81fc02cf4591cc5 received from {username='slave'} at 127.0.0.1:39590
I20260709 13:18:02.733951 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:02.736600 12627 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.734086 12628 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:18:02.737634 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:02.740170 13349 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:02.742254 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:02.722846 13982 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:02.745997 13349 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:02.748078 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:02.747293 13349 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:02.751401 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:02.751968 13968 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:02.750027 13835 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.752501 13835 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:02.752898 13835 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.751539 13989 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:02.753587 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:02.741313 13916 ts_tablet_manager.cc:1403] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 2.210s	user 0.034s	sys 0.008s
I20260709 13:18:02.755396 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:02.737663 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:02.749884 13971 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.761561 13989 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.740414 13978 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.761246 13977 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:18:02.765900 12563 debug-util.cc:398] Leaking SignalData structure 0x7b08000916c0 after lost signal to thread 11520
I20260709 13:18:02.766333 13989 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:02.747922 12644 tablet_copy_service.cc:434] P 19cd864c053744bebd96093a7982537e: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-981fb2f435e24d06a81fc02cf4591cc5 on tablet 981fb2f435e24d06a81fc02cf4591cc5 with peer 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:02.767082 12563 debug-util.cc:398] Leaking SignalData structure 0x7b0800091da0 after lost signal to thread 12566
W20260709 13:18:02.768576 12563 debug-util.cc:398] Leaking SignalData structure 0x7b080032fda0 after lost signal to thread 12658
I20260709 13:18:02.768399 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:02.765182 13977 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:02.769695 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:02.770238 13977 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.769248 12597 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:02.774500 12597 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:02.770864 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:02.777662 13964 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:02.777675 12629 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 1.
I20260709 13:18:02.778339 13964 raft_consensus.cc:697] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:02.779107 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:02.779259 13964 consensus_queue.cc:237] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:02.776585 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:02.761250 13971 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:02.780727 12597 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:02.787285 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:02.785094 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:02.765237 13978 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.786870 12777 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 1.
W20260709 13:18:02.775938 12563 debug-util.cc:398] Leaking SignalData structure 0x7b0800253720 after lost signal to thread 12733
I20260709 13:18:02.788770 13978 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.777256 13983 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.789252 13968 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.782260 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:02.789486 12597 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:02.794777 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:02.795559 12777 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.787405 13971 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.796775 12597 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:02.798399 13972 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.806624 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:02.803745 13994 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.810482 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:02.811395 12783 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:18:02.793895 13983 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.810563 13994 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:02.818152 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:02.818955 12781 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:18:02.798398 13976 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.828202 13976 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.828975 13976 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.820039 13993 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.820082 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:02.794709 13991 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.796705 13989 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:02.803423 13990 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.832796 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:02.806169 13972 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.838124 13972 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.833880 12782 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:18:02.835489 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:02.845983 12783 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.828029 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:02.831589 13973 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:02.834329 13993 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.820851 13983 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.848116 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:02.801709 13982 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.844509 13991 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:02.813090 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:02.907183 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 48209.8us
I20260709 13:18:02.821070 13968 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.911034 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:02.892333 13982 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:02.922210 12528 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40528 (request call id 132) took 2403 ms. Trace:
I20260709 13:18:02.922667 12528 rpcz_store.cc:276] 0709 13:18:00.510534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:00.510698 (+   164us) service_pool.cc:224] Handling call
0709 13:18:02.913940 (+2403242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":459086,"reported_tablets":1}
I20260709 13:18:02.924253 12527 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54814 (request call id 216) took 2389 ms. Trace:
I20260709 13:18:02.924664 12527 rpcz_store.cc:276] 0709 13:18:00.534910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:00.535354 (+   444us) service_pool.cc:224] Handling call
0709 13:18:02.924089 (+2388735us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":899815,"reported_tablets":3}
I20260709 13:18:02.926426 13899 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:02.636119 13882 ts_tablet_manager.cc:1469] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Trace:
0709 13:17:57.974261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:57.974360 (+    99us) service_pool.cc:224] Handling call
0709 13:17:57.984444 (+ 10084us) ts_tablet_manager.cc:934] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
0709 13:17:58.862763 (+878319us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:17:59.128259 (+265496us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:59.128403 (+   144us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:17:59.352218 (+223815us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:17:59.352504 (+   286us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:59.352800 (+   296us) ts_tablet_manager.cc:989] Replica registered.
0709 13:17:59.352855 (+    55us) inbound_call.cc:177] Queueing success response
0709 13:17:59.976812 (+623957us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:18:02.054842 (+2078030us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:18:02.058068 (+  3226us) tablet_replica.cc:261] Starting instrumentation
0709 13:18:02.060055 (+  1987us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":337,"raft.queue_time_us":7896,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":28816,"tablet-copy.queue_time_us":9226,"tablet-download-pool.queue_time_us":6411,"tablet-download-pool.run_cpu_time_us":1684,"tablet-download-pool.run_wall_time_us":295281,"thread_start_us":229318,"threads_started":2}
I20260709 13:18:02.910343 12776 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 1.
I20260709 13:18:02.845814 13989 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:02.908773 13991 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.937775 12627 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:02.940619 12781 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 1.
W20260709 13:18:02.927871 12563 debug-util.cc:398] Leaking SignalData structure 0x7b0800349e40 after lost signal to thread 12807
I20260709 13:18:02.946995 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:02.915472 13967 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:02.950829 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:02.930627 13966 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:02.952625 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:02.954983 13990 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:02.941082 13969 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:02.952203 13994 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:02.955485 13990 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.960240 12778 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 2: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:02.962333 12777 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:02.963253 13994 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.963701 13994 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:02.964095 13994 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.963521 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:02.963253 13986 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:02.974409 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:02.993477 13958 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:02.959805 13967 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.952812 13988 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:03.002701 13920 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:02.994549 13895 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:03.009413 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:02.970260 13922 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:02.998811 13967 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:03.004976 13929 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:03.008890 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:03.006814 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:03.017828 13962 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:03.057477 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 50784.4us
I20260709 13:18:02.952435 13916 raft_consensus.cc:359] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:03.062582 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:03.067174 12747 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:03.074026 13936 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:03.074311 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:03.074662 13936 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:03.075109 12627 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:18:03.057971 13967 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.020666 13381 raft_consensus.cc:1217] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 13:18:03.062569 12563 debug-util.cc:398] Leaking SignalData structure 0x7b08003495e0 after lost signal to thread 13410
I20260709 13:18:03.061482 13981 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.994156 13958 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:03.061230 13916 raft_consensus.cc:740] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:18:03.085098 13348 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:03.058782 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:03.056265 13895 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:03.087738 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:03.080192 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:03.085531 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:03.086215 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:03.086789 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.087128 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:03.089289 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:03.089987 12630 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:18:03.060498 13993 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:03.091181 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:03.075861 13923 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:03.093593 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:03.067313 13920 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:03.097401 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.098181 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:03.083870 12563 debug-util.cc:398] Leaking SignalData structure 0x7b0800349440 after lost signal to thread 13877
I20260709 13:18:03.058342 13922 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:03.080827 13981 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:03.101620 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.103348 13787 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:03.138849 12747 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:03.148144 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:03.098078 13961 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:03.154819 12783 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
W20260709 13:18:03.147028 12563 debug-util.cc:398] Leaking SignalData structure 0x7b0800319160 after lost signal to thread 13887
I20260709 13:18:03.160260 12628 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.110807 13923 raft_consensus.cc:697] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:03.104969 13982 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:03.147477 13981 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.100601 13989 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:03.168627 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 68803.7us
I20260709 13:18:03.152859 13899 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:03.171469 13348 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:18:03.164835 12563 debug-util.cc:398] Leaking SignalData structure 0x7b08002a31e0 after lost signal to thread 13888
I20260709 13:18:03.173794 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.163254 13966 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:03.157696 13961 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:03.178040 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:03.180179 13996 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:03.190321 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:03.157445 13997 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:03.078346 13929 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:03.171239 13892 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:03.195132 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:03.195273 14000 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:18:03.189682 12563 debug-util.cc:398] Leaking SignalData structure 0x7b080021b460 after lost signal to thread 13904
I20260709 13:18:03.197532 13878 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.175374 13899 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:03.197685 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:03.201155 13999 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:03.191377 13961 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.179594 13966 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:03.201766 13936 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:03.203215 13899 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.190061 13968 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:03.151031 13916 consensus_queue.cc:260] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:03.172710 13983 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:03.205179 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 45910.3us
I20260709 13:18:03.152693 13787 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:03.199752 12563 debug-util.cc:398] Leaking SignalData structure 0x7b0800347680 after lost signal to thread 13912
W20260709 13:18:03.218454 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 70389.7us
I20260709 13:18:03.208400 13966 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.221122 13998 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:03.216814 13936 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:03.217698 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:03.220481 12563 debug-util.cc:398] Leaking SignalData structure 0x7b0800361040 after lost signal to thread 13928
I20260709 13:18:03.203007 14000 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:03.208648 13996 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:03.234112 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:03.231627 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.210284 13999 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:18:03.230265 12563 debug-util.cc:398] Leaking SignalData structure 0x7b0800347740 after lost signal to thread 13934
I20260709 13:18:03.229646 12625 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.207733 14001 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:03.207525 13892 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:03.247555 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:03.241122 12563 debug-util.cc:398] Leaking SignalData structure 0x7b08003476e0 after lost signal to thread 13937
I20260709 13:18:03.230875 13976 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:03.245615 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 74194.4us
I20260709 13:18:03.208096 13878 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:03.230243 13936 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:03.245311 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.251050 13922 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:03.222016 13923 consensus_queue.cc:237] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:03.239599 13999 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.210014 13997 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:03.253386 12563 debug-util.cc:398] Leaking SignalData structure 0x7b0800347600 after lost signal to thread 13945
I20260709 13:18:03.248951 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:03.259289 13835 consensus_meta.cc:287] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.506s	user 0.012s	sys 0.000s
I20260709 13:18:03.260696 13835 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:03.261003 13929 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:03.262698 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.263339 12702 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:18:03.269767 13971 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:03.271190 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:03.274112 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:18:03.280385 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 65075.4us
I20260709 13:18:03.282202 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:03.261334 13878 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:03.288357 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:03.254429 14001 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:03.285275 13380 raft_consensus.cc:2393] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 2: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:03.266269 13787 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:03.286726 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.259621 13895 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:03.245870 13998 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:03.290566 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:03.298269 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.296617 12707 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:18:03.272377 13994 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:03.301579 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:03.303259 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:03.303711 12778 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:18:03.290357 13835 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:18:03.306632 14000 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:03.283368 13972 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:03.294700 13983 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:03.295998 13892 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:03.300280 13971 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:03.318998 12627 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.307066 13998 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:03.305188 13997 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:03.307190 13920 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:03.315302 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.320189 12775 raft_consensus.cc:2393] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 2: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:03.315379 13976 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:03.320426 13977 consensus_meta.cc:287] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.550s	user 0.012s	sys 0.009s
I20260709 13:18:03.322299 13916 ts_tablet_manager.cc:1434] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.557s	user 0.003s	sys 0.010s
I20260709 13:18:03.320596 14000 consensus_queue.cc:1059] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:18:03.331377 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.331889 12704 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:03.336513 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:03.330775 13916 ts_tablet_manager.cc:1467] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd: Tablet startup took 572ms
I20260709 13:18:03.339099 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:03.339331 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:03.340283 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.340956 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:03.342875 12704 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:18:03.341499 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:03.326710 12700 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:03.334142 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:03.346004 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:03.346446 13961 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:03.349223 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:03.335579 13936 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:03.354048 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:03.356282 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:03.346129 13924 consensus_meta.cc:287] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.620s	user 0.007s	sys 0.017s
I20260709 13:18:03.357910 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:03.357834 13924 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:03.358587 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:03.358982 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:03.352588 12782 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:18:03.359791 13969 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:03.359890 13924 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:03.360724 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:03.360724 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:03.361948 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:03.362089 12597 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:03.362223 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:03.362875 13973 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:03.362942 12597 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:03.363072 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:03.363332 13973 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:03.363759 13973 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.363931 13983 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:03.363783 12597 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:03.336840 13996 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:03.364316 13983 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:03.333412 13977 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:03.374168 13972 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:03.374577 13962 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:03.378226 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:03.378475 13962 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:03.378908 13962 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.375490 13983 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.375548 12527 catalog_manager.cc:5697] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 1 to 2, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 2 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:03.383213 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:03.383817 12778 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:18:03.389268 13966 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:03.390293 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.390941 12700 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:18:03.391327 13966 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:03.392740 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:03.393008 12746 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:03.392037 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:03.339864 13916 ts_tablet_manager.cc:1469] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd: Trace:
0709 13:17:59.218965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.219098 (+   133us) service_pool.cc:224] Handling call
0709 13:17:59.469695 (+250597us) ts_tablet_manager.cc:934] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
0709 13:17:59.689076 (+219381us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:18:00.089034 (+399958us) tablet_metadata.cc:665] Metadata flushed
0709 13:18:00.089158 (+   124us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:18:00.096379 (+  7221us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:18:00.096682 (+   303us) tablet_replica.cc:206] Creating consensus instance
0709 13:18:00.097096 (+   414us) ts_tablet_manager.cc:989] Replica registered.
0709 13:18:00.097279 (+   183us) inbound_call.cc:177] Queueing success response
0709 13:18:00.531507 (+434228us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:18:02.763882 (+2232375us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:18:02.794468 (+ 30586us) tablet_replica.cc:261] Starting instrumentation
0709 13:18:02.795289 (+   821us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":316,"mutex_wait_us":64162,"raft.queue_time_us":41573,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":3568,"tablet-copy.queue_time_us":180830,"tablet-download-pool.queue_time_us":534,"tablet-download-pool.run_cpu_time_us":2903,"tablet-download-pool.run_wall_time_us":23972,"thread_start_us":115886,"threads_started":2}
I20260709 13:18:03.395416 12701 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:18:03.398068 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.398365 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.399606 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.400243 13967 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:03.364604 13936 consensus_queue.cc:1059] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:18:03.356972 12531 catalog_manager.cc:5697] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 2 to 3, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 3 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:03.404278 12597 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:03.348073 13991 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:03.375373 13981 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:03.406215 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.411859 13968 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:03.412817 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:03.407218 13979 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:03.413681 13961 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:03.400267 12700 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:18:03.414399 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.414984 12704 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:03.397310 13958 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:03.354584 13997 raft_consensus.cc:3055] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:03.401278 13962 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:03.419272 13997 raft_consensus.cc:740] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:03.420149 13997 consensus_queue.cc:260] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:03.422509 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.418831 13936 raft_consensus.cc:3055] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:03.416838 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:03.421461 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.422695 13997 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.424115 12781 raft_consensus.cc:2393] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 2: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:03.424970 12597 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:03.420609 13962 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:03.422842 13854 consensus_queue.cc:1059] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:18:03.422729 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.427313 12629 raft_consensus.cc:1240] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.431143 12597 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:03.428156 12774 raft_consensus.cc:2393] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 2: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:03.429883 13979 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:03.428521 13981 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:03.429203 13972 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:03.434726 13972 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:03.434809 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:03.435189 13972 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.436146 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:03.436542 12718 tablet_copy_service.cc:342] P 639ab2db265f4747bb7afb93e833f530: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-6609d5f93ab048558ea46e511ef3b2ec received from {username='slave'} at 127.0.0.1:38496
I20260709 13:18:03.436970 12718 tablet_copy_service.cc:434] P 639ab2db265f4747bb7afb93e833f530: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-6609d5f93ab048558ea46e511ef3b2ec on tablet 6609d5f93ab048558ea46e511ef3b2ec with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:03.437234 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.437704 13994 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:03.437881 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.438301 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:03.438633 14001 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:03.439184 12700 raft_consensus.cc:3055] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:03.438921 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:03.439558 12700 raft_consensus.cc:740] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:03.439819 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.430439 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.430487 12701 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:03.442176 13979 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:03.442569 13979 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:03.442958 13979 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.427696 13991 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:03.440796 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:03.424121 13936 raft_consensus.cc:740] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:03.400943 13983 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:03.443571 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 44255.2us
I20260709 13:18:03.446978 12597 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:03.449312 13936 consensus_queue.cc:260] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:03.440419 13978 consensus_meta.cc:287] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.645s	user 0.018s	sys 0.000s
I20260709 13:18:03.457108 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:03.458755 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.459369 13978 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:03.459937 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.460696 12701 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 1.
I20260709 13:18:03.463047 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.463620 13978 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:03.464272 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:03.464969 13999 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:03.465762 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.466415 12707 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 1.
I20260709 13:18:03.466480 13994 consensus_queue.cc:1059] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:03.467473 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:03.467900 13981 raft_consensus.cc:3055] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:03.468127 12781 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.468313 13981 raft_consensus.cc:740] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:03.469273 13981 consensus_queue.cc:260] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:03.470225 13854 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:03.470443 13878 raft_consensus.cc:3055] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:03.470781 13981 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.464993 12632 raft_consensus.cc:2393] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 2: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:18:03.473788 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 221446us
I20260709 13:18:03.474483 13348 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:03.470908 13878 raft_consensus.cc:740] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:03.476720 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:03.472162 13964 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:03.477420 13878 consensus_queue.cc:260] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:03.478185 13964 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:03.478632 13964 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.478788 13878 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.477892 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:03.480409 13348 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:03.482924 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:03.470242 13990 consensus_meta.cc:287] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.501s	user 0.010s	sys 0.000s
I20260709 13:18:03.487807 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:03.488590 12709 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:03.487205 13348 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:03.490291 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.490746 13348 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:03.490417 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.493705 13971 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:03.494128 13971 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:03.494515 13971 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.496590 13961 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:03.497171 13961 raft_consensus.cc:697] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:03.499452 13385 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.501046 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:03.486346 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.503239 12702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:18:03.470878 13854 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:03.505224 13385 raft_consensus.cc:2393] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 2: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:03.480777 13978 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:03.510949 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:03.493472 13976 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:03.521274 12632 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:18:03.521369 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:03.522902 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:03.529465 12793 tablet_copy_service.cc:342] P 3d539f8d95d343d8b1696803f18b30d2: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-af96e52b004d4c8c942cc0ec33c95c99 received from {username='slave'} at 127.0.0.1:54346
I20260709 13:18:03.529770 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:03.510039 13936 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.530035 12793 tablet_copy_service.cc:434] P 3d539f8d95d343d8b1696803f18b30d2: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-af96e52b004d4c8c942cc0ec33c95c99 on tablet af96e52b004d4c8c942cc0ec33c95c99 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:03.540392 13961 consensus_queue.cc:237] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:03.541915 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:03.496963 13990 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:03.532428 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:03.530359 13966 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:18:03.533306 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:03.534590 13381 raft_consensus.cc:1240] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.545243 13977 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:03.527163 12718 tablet_copy_service.cc:342] P 639ab2db265f4747bb7afb93e833f530: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-4c13fa2aff324f65ad0bf6191ea68fab received from {username='slave'} at 127.0.0.1:38496
I20260709 13:18:03.538652 13976 raft_consensus.cc:697] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:03.546212 12718 tablet_copy_service.cc:434] P 639ab2db265f4747bb7afb93e833f530: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-4c13fa2aff324f65ad0bf6191ea68fab on tablet 4c13fa2aff324f65ad0bf6191ea68fab with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:03.487046 13972 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:03.547233 13976 consensus_queue.cc:237] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:03.548919 13972 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:03.550349 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.544238 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.504858 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.537137 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.539844 13379 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Advancing to term 3
W20260709 13:18:03.554759 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 56654.9us
I20260709 13:18:03.555691 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:03.556427 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.451925 12700 consensus_queue.cc:260] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:03.567137 12625 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 1. Current term is 2. Ops: []
W20260709 13:18:03.633016 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:03.632997 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.634400 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.635540 13385 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.635587 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.642146 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.878218 13379 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 3 LEARNER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:18:03.892604 13979 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:03.886565 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:03.956054 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 302816us
I20260709 13:18:03.888823 13967 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:03.773640 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:03.872020 13854 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:03.977134 12700 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:03.908444 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 247696us
I20260709 13:18:03.857368 13983 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:03.787978 12709 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
W20260709 13:18:03.745111 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:03.645365 13973 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:03.657063 13385 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.683403 12780 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.980149 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.770857 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:03.980787 12781 consensus_meta.cc:287] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.512s	user 0.001s	sys 0.010s
I20260709 13:18:03.672495 12778 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.696903 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.829186 12775 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:03.893994 13378 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.839572 13384 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.990986 13348 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:03.770155 12774 raft_consensus.cc:1240] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.994211 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.590860 13972 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:03.741657 13899 consensus_meta.cc:287] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.001s
I20260709 13:18:03.655426 13962 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.710071 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:03.732270 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:04.012610 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:03.704629 13999 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:04.010455 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.633330 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:03.561676 12701 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 1.
I20260709 13:18:03.704278 13990 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:04.131404 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 52402.4us
I20260709 13:18:04.020955 12780 raft_consensus.cc:1240] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.022368 12629 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.023015 13979 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:04.023312 13379 raft_consensus.cc:1240] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.023453 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:04.022436 13380 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.024133 13381 raft_consensus.cc:2393] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 2: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
W20260709 13:18:04.000447 13964 consensus_meta.cc:287] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.521s	user 0.008s	sys 0.004s
I20260709 13:18:04.027204 12781 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
W20260709 13:18:04.038136 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 47310.1us
I20260709 13:18:04.147691 13348 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:04.108800 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:03.770810 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:03.783532 13971 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:04.079067 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:04.025125 14015 consensus_queue.cc:1059] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 13:18:04.039847 13967 consensus_queue.cc:1059] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
W20260709 13:18:04.089206 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.089728 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:04.089937 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:04.091754 12628 raft_consensus.cc:2393] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 2: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:04.089164 13384 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:04.125413 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:03.871095 12703 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:04.133853 12783 raft_consensus.cc:1240] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.131280 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:18:04.080785 13961 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.006s	sys 0.050s
I20260709 13:18:04.158268 12746 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:04.138304 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 118919us
I20260709 13:18:04.107261 13377 raft_consensus.cc:2393] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 2: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:04.142308 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:04.089758 12782 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.152659 13999 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.106168 13973 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:04.170743 13936 consensus_queue.cc:1059] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:04.171871 13936 raft_consensus.cc:3055] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:04.172243 13936 raft_consensus.cc:740] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:04.172339 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:04.106665 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:18:04.173795 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.173933 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:04.174491 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:04.174890 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:04.175323 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:04.175683 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:04.175925 14001 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:04.176616 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:04.099516 13962 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:04.161005 12778 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.180588 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:04.177482 12780 raft_consensus.cc:2393] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 2: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:04.146765 13921 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.181905 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:04.182853 13962 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:04.183064 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:04.163962 12630 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.119558 13899 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:04.165494 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:04.146029 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:04.164611 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:04.168380 13382 raft_consensus.cc:2393] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 2: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:04.168404 13385 raft_consensus.cc:1240] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.169243 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:04.175462 12775 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.176446 14001 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:04.093454 13969 consensus_queue.cc:1059] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:18:04.185055 13981 consensus_queue.cc:1059] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:04.186117 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:04.188673 13968 raft_consensus.cc:3055] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:18:04.154161 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 116269us
I20260709 13:18:04.217342 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:04.189836 14018 log.cc:927] Time spent T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:18:04.230152 12778 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:04.192278 12746 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:04.232116 14017 log.cc:927] Time spent T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:18:04.190582 12780 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.234498 13966 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.234663 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:04.236032 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:04.236027 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:04.195966 13972 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.237715 12777 raft_consensus.cc:2393] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 2: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:04.196900 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:04.157323 12708 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:04.238739 13899 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:04.238766 12625 raft_consensus.cc:1240] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 1. Current term is 2. Ops: []
W20260709 13:18:04.202342 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.239892 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:04.241010 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:04.198766 13979 raft_consensus.cc:3055] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:04.241899 13979 raft_consensus.cc:740] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:04.242092 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:04.243242 12776 raft_consensus.cc:1240] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.242816 13979 consensus_queue.cc:260] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:04.244401 13979 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:04.202806 12599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.006s	sys 0.009s
W20260709 13:18:04.245081 12599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.006s	sys 0.009s
I20260709 13:18:04.246177 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:04.247812 14001 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:04.173178 13936 consensus_queue.cc:260] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:04.250615 13936 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:04.253639 14000 consensus_queue.cc:1175] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3d539f8d95d343d8b1696803f18b30d2" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 13:18:04.254518 13996 consensus_queue.cc:1059] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:04.255153 14000 consensus_queue.cc:1059] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:04.198704 14022 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.168723 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:04.212471 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:04.208730 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:04.285689 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:04.286406 14000 raft_consensus.cc:3055] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:04.214632 13997 consensus_queue.cc:1059] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:04.286854 14000 raft_consensus.cc:740] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:04.287600 13997 raft_consensus.cc:3055] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:04.288003 13997 raft_consensus.cc:740] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:04.288079 14000 consensus_queue.cc:260] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:04.215638 13969 raft_consensus.cc:3055] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:04.289820 13969 raft_consensus.cc:740] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:04.290732 13969 consensus_queue.cc:260] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:04.291464 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:04.292119 13969 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.292858 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:04.294032 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:04.294689 13962 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:04.295204 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:04.295285 13962 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:04.296783 12706 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:18:04.217295 13384 raft_consensus.cc:2393] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 2: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:04.221257 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:04.222102 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:04.298832 12708 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:04.298974 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:04.299695 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:04.300197 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:04.300665 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:04.225554 13378 raft_consensus.cc:1217] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:18:04.301787 12700 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:04.302361 12702 raft_consensus.cc:606] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Received request to transfer leadership to TS 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:04.302688 12702 raft_consensus.cc:614] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:04.304239 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:04.228988 13971 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:04.191236 13967 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:04.288887 13997 consensus_queue.cc:260] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:04.206506 13376 raft_consensus.cc:2393] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 2: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:04.151216 13964 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:04.307250 13997 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.308862 13964 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:04.154626 12597 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:04.255605 13996 consensus_queue.cc:1059] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:04.253654 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:04.309736 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 155188us
I20260709 13:18:04.256260 13854 raft_consensus.cc:3055] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:04.310415 13854 raft_consensus.cc:740] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:04.311425 13854 consensus_queue.cc:260] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:04.312747 13854 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.256914 12672 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:04.269686 13968 raft_consensus.cc:740] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:18:04.314148 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 57318.5us
I20260709 13:18:04.314487 13968 consensus_queue.cc:260] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:04.314988 12672 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:04.315212 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:04.315943 13968 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.316367 12672 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:04.203348 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 47322.3us
I20260709 13:18:04.317020 12597 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:04.261391 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:04.317726 12783 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.317889 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:04.318409 13878 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:04.316382 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:04.265281 12778 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:04.319207 13878 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.319847 13878 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.316402 13965 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:04.320658 13965 raft_consensus.cc:697] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:04.320734 14016 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:04.321652 12701 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:04.322114 13996 raft_consensus.cc:3055] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:04.322424 12598 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:04.322636 13996 raft_consensus.cc:740] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:04.323480 13973 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.323562 13996 consensus_queue.cc:260] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:04.324998 13996 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.325067 13991 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:04.320047 13986 consensus_queue.cc:1059] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:18:04.297405 13962 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:04.281275 12630 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:04.214169 12629 raft_consensus.cc:1240] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.216706 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:04.297863 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:04.316797 14000 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.318727 12782 raft_consensus.cc:2393] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 2: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:04.321738 13965 consensus_queue.cc:237] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:04.261934 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:04.265511 13348 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:04.346132 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 80749.8us
I20260709 13:18:04.346813 13348 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:04.348646 12782 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:04.270411 12704 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:04.326226 12672 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:04.325539 13991 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:04.344234 12783 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:04.351974 12672 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:04.353098 12672 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:04.356161 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.326587 13986 raft_consensus.cc:3055] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:18:04.357035 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.358074 12672 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:04.355103 12597 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:04.345643 12775 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:04.393610 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:04.358960 13976 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:04.360110 13986 raft_consensus.cc:740] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:18:04.401106 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:04.395401 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 73048.5us
I20260709 13:18:04.352432 14001 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.403014 13986 consensus_queue.cc:260] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:04.359580 13892 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:04.403920 12598 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:04.352730 13961 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.404294 13986 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.352617 13991 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.360261 13920 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.407500 14001 consensus_queue.cc:1059] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:18:04.412706 13920 raft_consensus.cc:3055] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:04.413167 13920 raft_consensus.cc:740] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:04.417827 13976 raft_consensus.cc:697] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:04.419204 13976 consensus_queue.cc:237] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:04.422055 13920 consensus_queue.cc:260] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:04.424422 12598 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:04.427026 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:04.364398 13958 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.364056 13964 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:04.356010 13878 consensus_queue.cc:1059] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:18:04.433809 12747 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:04.435518 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:04.364205 13977 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 13:18:04.446996 12598 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:04.450969 12597 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:04.448120 13967 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.449154 13964 raft_consensus.cc:697] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:04.445807 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:04.452567 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:04.467474 12598 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:04.436287 13972 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:18:04.470564 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 85332.9us
I20260709 13:18:04.443261 13899 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.472530 12672 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:04.475059 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:04.475426 12747 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:04.484767 12747 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:04.502774 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:04.493932 12597 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:04.485354 13990 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:04.471540 13972 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:04.482775 13966 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.517643 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:04.529635 14027 consensus_queue.cc:1059] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:04.502880 13994 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:18:04.537743 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 65277.4us
W20260709 13:18:04.538506 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 44691.9us
I20260709 13:18:04.538525 12672 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:04.507679 14016 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:04.541103 12747 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:04.509161 13920 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.509600 13990 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:04.546074 12747 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:04.511584 13967 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.471076 13986 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:04.523041 13972 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.522650 14026 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.520985 14031 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.553651 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:04.554898 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:04.555716 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:04.540879 14015 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:18:04.560168 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.547427 13899 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.562009 12625 raft_consensus.cc:1240] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.551024 13990 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.549216 13966 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.554203 13923 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.551765 13986 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:04.559559 13994 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.556721 13981 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.563464 13967 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.562574 14030 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.589738 12708 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:04.564073 13936 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:04.558450 14028 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.587886 13997 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.565073 14016 raft_consensus.cc:697] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:04.562633 13899 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:04.559055 13964 consensus_queue.cc:237] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:04.626950 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 70007.9us
I20260709 13:18:04.574220 13986 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.629149 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:04.635949 12747 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:04.638567 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:04.641993 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 138250us
I20260709 13:18:04.644773 12598 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:04.646735 12598 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:04.654913 12598 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:04.624516 13936 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:04.657379 12598 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:04.647889 13994 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.657449 14034 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:04.660085 14040 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:04.660665 14040 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:04.661000 14040 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:04.658859 12598 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:04.660156 14034 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:04.666172 14013 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.635627 14000 consensus_queue.cc:1059] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:18:04.656471 13966 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.624667 14030 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:04.666365 12598 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:04.657742 14022 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.666755 12630 raft_consensus.cc:1240] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
W20260709 13:18:04.680267 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:04.666728 14036 log.cc:927] Time spent T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:18:04.689920 12747 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:04.691164 14031 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:04.675930 14033 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:18:04.724185 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 44890.6us
I20260709 13:18:04.680779 14034 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.725754 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:04.666741 14015 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.779145 13348 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:04.681715 13996 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:04.676424 14035 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:04.783605 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 57916.8us
I20260709 13:18:04.788204 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:04.788496 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 60157.4us
I20260709 13:18:04.649554 14016 consensus_queue.cc:237] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:04.784101 13376 raft_consensus.cc:1240] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.728127 13969 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:18:04.811271 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.809468 14042 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.818361 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:04.818974 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:04.822544 14035 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:04.826383 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:04.827536 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:04.828194 12706 raft_consensus.cc:2376] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3. Re-sending same reply.
I20260709 13:18:04.828576 14035 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:04.826611 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:04.829288 12708 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:04.830336 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:04.831225 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.830953 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:04.811789 13958 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.790375 14029 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:04.832724 13958 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:04.835772 14029 raft_consensus.cc:697] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:04.832235 12708 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:04.780740 13971 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:18:04.811821 14015 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.811246 14027 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:04.787344 13996 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:04.876780 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:04.813436 14028 consensus_queue.cc:1059] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 13:18:04.836843 14029 consensus_queue.cc:237] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:04.840775 13923 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.891992 14026 raft_consensus.cc:1064] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:18:04.918030 14007 raft_consensus.cc:3055] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:04.918586 14007 raft_consensus.cc:740] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:04.895022 13991 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:04.899822 14032 log.cc:927] Time spent T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.302s	user 0.000s	sys 0.002s
I20260709 13:18:04.901492 13936 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:04.923772 13969 raft_consensus.cc:1064] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:18:04.929714 14001 raft_consensus.cc:3055] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:04.885385 14031 raft_consensus.cc:2764] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:18:04.876834 13899 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:04.878844 14038 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.212s
I20260709 13:18:04.931815 13991 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:04.880336 14027 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:04.956138 14045 log.cc:927] Time spent T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 13:18:04.956382 13967 raft_consensus.cc:1064] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
W20260709 13:18:04.959018 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 420205us
W20260709 13:18:04.974948 13967 raft_consensus.cc:1068] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2: Unable to promote non-voter 522fd2ba0e9f402691b5ef29af8579cd: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:18:04.977530 13349 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:04.960665 13854 consensus_meta.cc:287] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.647s	user 0.011s	sys 0.002s
I20260709 13:18:04.943645 13958 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:04.943168 14001 raft_consensus.cc:740] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:04.941040 12527 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54792 (request call id 223) took 1648 ms. Trace:
I20260709 13:18:04.990381 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:05.032855 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:18:04.992581 13976 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.044s	sys 0.089s
W20260709 13:18:05.034309 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.980022 13892 raft_consensus.cc:3055] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:18:05.038883 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:05.041637 13976 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.044s	sys 0.089s
I20260709 13:18:05.050933 12630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:18:05.051602 12630 raft_consensus.cc:606] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Received request to transfer leadership to TS 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:05.052230 12630 raft_consensus.cc:614] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:05.051489 14027 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:05.051504 13996 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:05.002091 13977 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:04.998471 12527 rpcz_store.cc:276] 0709 13:18:03.292218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.295521 (+  3303us) service_pool.cc:224] Handling call
0709 13:18:04.940954 (+1645433us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:03.729608 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:03.730277 (+   669us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:03.730341 (+    64us) write_op.cc:435] Acquired schema lock
0709 13:18:03.730383 (+    42us) tablet.cc:662] Decoding operations
0709 13:18:03.730856 (+   473us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:03.730945 (+    89us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:03.730974 (+    29us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:18:03.731325 (+   351us) tablet.cc:701] Row locks acquired
0709 13:18:03.731364 (+    39us) write_op.cc:260] PREPARE: finished
0709 13:18:03.731573 (+   209us) write_op.cc:270] Start()
0709 13:18:03.731752 (+   179us) write_op.cc:276] Timestamp: P: 1783603083731505 usec, L: 0
0709 13:18:03.731799 (+    47us) op_driver.cc:348] REPLICATION: starting
0709 13:18:03.732728 (+   929us) log.cc:844] Serialized 533 byte log entry
0709 13:18:04.553196 (+820468us) op_driver.cc:464] REPLICATION: finished
0709 13:18:04.730671 (+177475us) write_op.cc:301] APPLY: starting
0709 13:18:04.783287 (+ 52616us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:04.810361 (+ 27074us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:04.879377 (+ 69016us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:04.890512 (+ 11135us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:04.890807 (+   295us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:18:04.890868 (+    61us) write_op.cc:312] APPLY: finished
0709 13:18:04.891232 (+   364us) log.cc:844] Serialized 23 byte log entry
0709 13:18:04.891483 (+   251us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:04.891567 (+    84us) write_op.cc:454] Released schema lock
0709 13:18:04.891620 (+    53us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:04.892066 (+   446us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:04.892112 (+    46us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":124888,"apply.run_cpu_time_us":3068,"apply.run_wall_time_us":168063,"num_ops":1,"prepare.queue_time_us":113884,"prepare.run_cpu_time_us":20988,"prepare.run_wall_time_us":404898,"raft.queue_time_us":525164,"raft.run_cpu_time_us":16013,"raft.run_wall_time_us":133728,"replication_time_us":808680,"thread_start_us":511142,"threads_started":4,"wal-append.queue_time_us":304359}]]}
W20260709 13:18:05.068972 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.067590 14027 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:05.104102 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 285225us
I20260709 13:18:04.941905 14040 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:05.105055 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:05.045993 13892 raft_consensus.cc:740] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:05.065587 13977 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:05.078673 13878 raft_consensus.cc:3055] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:05.118186 12747 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:05.118463 12748 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.007s	sys 0.002s
W20260709 13:18:05.119746 12672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.005s	sys 0.018s
I20260709 13:18:05.123301 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:05.125092 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 458777us
I20260709 13:18:05.116760 14038 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:18:05.130923 12672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.005s	sys 0.018s
W20260709 13:18:05.136588 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 598090us
I20260709 13:18:05.131140 12598 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:05.132099 12747 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:05.137825 12672 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:05.141045 12748 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.007s	sys 0.003s
W20260709 13:18:05.136952 13972 consensus_meta.cc:287] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.578s	user 0.011s	sys 0.000s
W20260709 13:18:05.140659 13968 consensus_meta.cc:287] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.824s	user 0.012s	sys 0.003s
W20260709 13:18:05.190464 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 58460.2us
W20260709 13:18:05.190912 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 53166.1us
I20260709 13:18:05.189772 12629 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:05.127686 13977 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.129352 13971 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:05.103494 14027 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:05.194607 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:05.136162 13967 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:05.046527 14007 consensus_queue.cc:260] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:05.189163 13994 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:05.195863 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:05.208976 12709 raft_consensus.cc:2393] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 2: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:05.104219 14001 consensus_queue.cc:260] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:05.196723 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:05.197045 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:05.197835 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:05.201170 12747 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:05.204519 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:05.142618 14051 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:05.205363 14052 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:05.128868 13878 raft_consensus.cc:740] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:05.229542 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:05.230240 12707 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:05.206144 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:05.211695 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:05.216704 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:05.193224 14025 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.000s	sys 0.032s
I20260709 13:18:05.203596 14013 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:05.206048 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:05.195276 13972 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:05.216526 12708 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:05.235040 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:05.241025 12776 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:05.243736 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:05.220343 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:05.234650 14025 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.000s	sys 0.032s
I20260709 13:18:05.247928 14037 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:05.248040 12779 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:05.248214 12780 raft_consensus.cc:1240] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:05.248760 12777 raft_consensus.cc:1240] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:05.249298 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:05.249298 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:05.222980 14051 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:05.254562 12702 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
W20260709 13:18:05.245465 13986 consensus_meta.cc:287] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.613s	user 0.004s	sys 0.006s
W20260709 13:18:05.263074 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.256286 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:05.269203 14049 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:05.269691 12777 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
W20260709 13:18:05.270419 14048 log.cc:927] Time spent T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:18:05.271910 14026 consensus_queue.cc:237] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:05.274209 13990 consensus_meta.cc:287] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.694s	user 0.010s	sys 0.000s
I20260709 13:18:05.275203 13972 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:05.276690 14030 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:05.258589 12700 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:05.278764 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:05.279914 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:05.265290 12774 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:05.281306 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:05.282040 12703 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:05.284571 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:05.285281 12704 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:05.285386 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:05.285967 12702 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:05.289459 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:05.289865 13878 consensus_queue.cc:260] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:05.291241 13878 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.274609 13990 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:05.273444 13966 consensus_meta.cc:287] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.588s	user 0.001s	sys 0.007s
I20260709 13:18:05.267333 14051 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:05.266656 14040 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:05.284853 13986 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:05.303040 13986 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:05.297319 14027 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:05.307621 12630 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:05.236606 13892 consensus_queue.cc:260] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:05.308763 13892 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.308872 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:05.309023 13977 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:05.311002 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:05.311123 13378 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 1. Current term is 2. Ops: []
W20260709 13:18:05.313200 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:05.314697 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.316023 12703 raft_consensus.cc:1275] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:18:05.325848 14034 consensus_meta.cc:287] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.540s	user 0.011s	sys 0.000s
I20260709 13:18:05.326306 14034 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:05.327203 14052 consensus_queue.cc:1059] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:05.301158 13966 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:05.328483 14034 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:05.328580 12778 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:05.330770 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:05.339003 12672 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:05.342235 14031 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.342965 14031 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:05.346846 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:05.347548 12625 raft_consensus.cc:1240] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:05.348657 12628 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:05.349514 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:05.349761 12625 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:05.352154 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:05.346016 13854 raft_consensus.cc:3055] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:05.358465 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:05.360515 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:05.359328 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:05.365526 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.366581 12780 raft_consensus.cc:1275] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:05.368506 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:05.369499 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:05.370348 12709 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 1.
W20260709 13:18:05.370862 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.371491 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:05.372752 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:05.388135 12628 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:05.408968 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:05.408968 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 45238.6us
W20260709 13:18:05.413921 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 423573us
I20260709 13:18:05.417580 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:05.341460 14027 consensus_queue.cc:1059] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.063s
I20260709 13:18:05.358723 13920 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:05.358162 13854 raft_consensus.cc:740] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:05.357822 14000 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.129s
I20260709 13:18:05.404001 14031 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:05.460763 12598 leader_election.cc:336] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.569s (SENT)
I20260709 13:18:05.460331 14028 consensus_queue.cc:1059] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
I20260709 13:18:05.519392 12598 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:05.390018 14052 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:05.384068 13997 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:18:05.465023 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 49586.7us
W20260709 13:18:05.472488 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 60776.4us
I20260709 13:18:05.526026 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:05.357450 13969 consensus_queue.cc:237] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:05.530973 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 113040us
I20260709 13:18:05.531494 13973 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.531817 14001 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.497597 13977 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:05.533352 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 115856us
I20260709 13:18:05.535173 12672 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:05.488153 13971 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:05.537379 13976 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:05.547906 13971 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:05.417795 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:05.546613 13349 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:05.546705 12598 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:05.543967 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:05.517632 14041 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:05.551084 12700 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:05.553807 13976 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:05.554240 13976 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:05.554574 13976 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.554775 13971 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:05.554411 13854 consensus_queue.cc:260] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:05.555649 12702 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:05.498158 13990 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:05.552349 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:05.559154 12598 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:05.563735 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:05.557562 12634 raft_consensus.cc:1240] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:05.566196 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:05.566907 13854 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.567428 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:05.548537 14034 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:05.572517 12672 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:05.570708 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.414234 13936 consensus_queue.cc:1059] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.136s
I20260709 13:18:05.574880 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:05.575013 12598 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:05.577054 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:05.578153 13384 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:05.417244 14035 consensus_queue.cc:1059] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.139s
I20260709 13:18:05.582417 12598 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:05.583462 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:05.584149 13382 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:18:05.584538 12598 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:05.501480 14051 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:05.580907 14058 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:05.586612 13990 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:05.587028 14040 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:05.595597 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:05.581346 14031 consensus_queue.cc:1059] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:18:05.599164 14058 raft_consensus.cc:697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:05.598834 13976 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:05.599931 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:05.601734 14062 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:05.602974 12782 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:18:05.608593 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.587966 13964 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:18:05.602604 14015 consensus_meta.cc:287] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.727s	user 0.007s	sys 0.005s
I20260709 13:18:05.581815 14026 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:05.580942 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:05.577989 13878 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:05.610942 12598 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:05.622915 12672 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:05.626252 13385 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:05.605793 14040 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:05.613806 12701 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:05.612066 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:05.573009 14061 consensus_queue.cc:1048] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 13:18:05.635149 12702 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:05.615845 13994 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:05.625447 14015 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:05.647013 12531 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54814 (request call id 217) took 2465 ms. Trace:
W20260709 13:18:05.647828 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 57143.2us
I20260709 13:18:05.641834 14035 raft_consensus.cc:3055] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:05.624656 14062 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:05.655650 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:05.621888 12634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:18:05.628012 13990 raft_consensus.cc:697] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:05.619413 14027 consensus_queue.cc:1243] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 13:18:05.654655 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.638170 14040 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:05.651408 13349 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:05.656422 12598 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:05.652575 14035 raft_consensus.cc:740] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:05.659515 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:05.616099 14041 consensus_queue.cc:1059] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:18:05.645908 13966 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.686877 13999 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:05.689962 14068 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:05.689884 13990 consensus_queue.cc:237] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:05.664597 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:05.664721 12634 raft_consensus.cc:606] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:05.654482 13878 consensus_queue.cc:1059] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:18:05.734290 14070 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:18:05.722803 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 71478.8us
W20260709 13:18:05.722587 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.722467 14030 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:05.654623 14034 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:05.655022 13972 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:18:05.660094 14031 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:05.658129 14062 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:05.653782 14000 raft_consensus.cc:3055] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:05.660063 14028 consensus_queue.cc:1243] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:18:05.662715 14026 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:18:05.729214 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 123817us
I20260709 13:18:05.730159 12708 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:05.654058 13994 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:05.664891 13936 consensus_queue.cc:1059] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
W20260709 13:18:05.721364 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 65011.3us
I20260709 13:18:05.650846 12531 rpcz_store.cc:276] 0709 13:18:03.181171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.181329 (+   158us) service_pool.cc:224] Handling call
0709 13:18:05.646920 (+2465591us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:04.142905 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:04.186170 (+ 43265us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:04.186213 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:18:04.186234 (+    21us) tablet.cc:662] Decoding operations
0709 13:18:04.186642 (+   408us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:04.186702 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:04.186738 (+    36us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:18:04.186996 (+   258us) tablet.cc:701] Row locks acquired
0709 13:18:04.187012 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:18:04.187199 (+   187us) write_op.cc:270] Start()
0709 13:18:04.187330 (+   131us) write_op.cc:276] Timestamp: P: 1783603084187142 usec, L: 0
0709 13:18:04.187362 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 13:18:04.188122 (+   760us) log.cc:844] Serialized 595 byte log entry
0709 13:18:04.665362 (+477240us) op_driver.cc:464] REPLICATION: finished
0709 13:18:04.892592 (+227230us) write_op.cc:301] APPLY: starting
0709 13:18:04.892694 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:04.892820 (+   126us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:04.892941 (+   121us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:04.893267 (+   326us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:04.893507 (+   240us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:18:04.920321 (+ 26814us) write_op.cc:312] APPLY: finished
0709 13:18:04.944754 (+ 24433us) log.cc:844] Serialized 23 byte log entry
0709 13:18:04.958659 (+ 13905us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:04.974347 (+ 15688us) write_op.cc:454] Released schema lock
0709 13:18:04.982256 (+  7909us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":5,"spinlock_wait_cycles":41984,"child_traces":[["op",{"apply.queue_time_us":225148,"apply.run_cpu_time_us":2506,"apply.run_wall_time_us":144079,"num_ops":1,"prepare.queue_time_us":666452,"prepare.run_cpu_time_us":3167,"prepare.run_wall_time_us":52121,"replication_time_us":469095,"thread_start_us":579914,"threads_started":1}]]}
I20260709 13:18:05.720293 14051 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:05.737032 14047 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:05.737849 13899 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:05.732167 13999 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:05.790479 13999 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.739802 13924 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:05.757911 14000 raft_consensus.cc:740] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:05.793826 13924 raft_consensus.cc:697] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:05.794523 14000 consensus_queue.cc:260] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:05.795094 13924 consensus_queue.cc:237] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:05.796098 14000 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:05.741659 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 82218.6us
I20260709 13:18:05.743563 12702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:18:05.725898 13979 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:05.731498 14073 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.731843 13966 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:05.752704 12707 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:05.736004 13349 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:05.732107 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:05.745713 14007 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.770673 14062 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:05.747197 14034 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:05.749377 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:05.720690 14058 consensus_queue.cc:237] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:05.784126 14067 log.cc:927] Time spent T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:18:05.775748 14071 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:05.784588 12673 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:05.781478 14013 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:18:05.788056 14047 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:05.804431 12702 raft_consensus.cc:606] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:18:05.834110 12702 raft_consensus.cc:614] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:18:05.807340 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 71382.5us
I20260709 13:18:05.808790 12672 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:05.776288 14026 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:05.812357 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:05.825801 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:05.805210 13979 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:05.783286 12598 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:05.818742 14034 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:05.774749 14030 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:05.797519 13983 consensus_queue.cc:1048] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:18:05.864355 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:05.827201 13994 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:05.807727 13976 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:05.839592 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 55080.8us
I20260709 13:18:05.838513 13981 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:05.842577 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:05.832849 13923 consensus_queue.cc:1059] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:18:05.841251 13979 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.804613 13899 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:05.965852 13899 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:05.852591 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:05.967577 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:05.968432 12702 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:05.853698 12672 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:05.854177 12634 raft_consensus.cc:1240] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 2. Current term is 3. Ops: []
W20260709 13:18:05.970819 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 117203us
I20260709 13:18:05.854069 14077 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:05.971172 13966 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:05.809782 14078 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:05.855800 14001 raft_consensus.cc:3055] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:05.977843 14001 raft_consensus.cc:740] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:05.978030 14034 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:05.978767 14001 consensus_queue.cc:260] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:05.979929 14001 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.856810 14049 raft_consensus.cc:3055] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:05.857553 13973 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:05.810956 13968 consensus_queue.cc:1059] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:18:05.832123 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:05.812635 14073 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:05.872308 12629 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
W20260709 13:18:05.842041 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 58860.2us
I20260709 13:18:05.983669 12598 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:05.837554 14071 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:05.762456 13958 consensus_queue.cc:1059] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 13:18:05.905215 14062 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:05.990049 14062 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:05.906087 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:05.995090 13920 raft_consensus.cc:3055] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:05.995493 13920 raft_consensus.cc:740] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:05.996332 13920 consensus_queue.cc:260] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:05.911458 12708 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 2. Current term is 3. Ops: []
W20260709 13:18:05.997812 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.846215 13854 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:05.914536 13996 raft_consensus.cc:3055] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:05.846982 14046 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:05.916210 13376 raft_consensus.cc:1217] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:05.872098 14047 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:05.971511 12672 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:06.001967 12747 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:06.010048 12672 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:05.976513 13977 consensus_queue.cc:1059] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:05.973964 14068 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:05.980317 12700 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:05.904337 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:05.986768 13348 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:05.981410 14049 raft_consensus.cc:740] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:05.987783 14077 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:05.982079 12628 raft_consensus.cc:1240] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:05.981310 12777 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:05.992470 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:05.911557 13981 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:05.846580 14041 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.942401 14030 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:05.860576 14035 consensus_queue.cc:260] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:05.982839 12704 raft_consensus.cc:3055] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:06.012989 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:06.013954 12708 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:18:06.023465 12704 raft_consensus.cc:740] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:05.987797 14071 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:05.994302 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:05.998714 13920 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.980082 14034 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:06.018185 12629 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:06.004956 13996 raft_consensus.cc:740] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:05.982359 12701 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:06.032255 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:06.016217 14070 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:06.043651 12777 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:06.034855 14035 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:06.047845 12781 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:06.036805 14071 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.042968 13854 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:06.050815 12700 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:06.047703 14077 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:06.054232 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:06.033959 14068 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:06.020608 13979 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:06.037806 13977 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:06.039446 14073 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:06.101266 13979 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:06.042769 14070 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:06.051682 13976 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:18:06.025588 14041 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:06.090835 14071 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:06.110224 14077 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:06.094460 13971 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:18:06.090373 14040 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:06.111443 14070 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:06.117928 13923 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:06.120752 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:06.138659 12783 raft_consensus.cc:1240] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:06.106400 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:06.112192 13996 consensus_queue.cc:260] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:06.140589 13996 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:06.143586 12707 raft_consensus.cc:2376] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2. Re-sending same reply.
I20260709 13:18:06.113726 14049 consensus_queue.cc:260] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:06.149187 12597 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:06.150825 12781 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:06.154618 12597 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:06.127417 13973 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.159616 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:06.158154 13973 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:06.160900 13981 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:06.132793 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:06.169512 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:06.169662 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:06.157173 13979 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:06.161228 13983 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:06.172093 13973 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:06.177742 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:06.137833 13999 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:06.181982 13966 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.186138 13966 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:06.178493 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:06.191581 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 164151us
I20260709 13:18:06.203343 13349 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:06.202514 14001 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.199530 14071 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:06.210887 12780 raft_consensus.cc:2393] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 2: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:06.206305 13924 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.192054 14033 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
W20260709 13:18:06.227589 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 240880us
W20260709 13:18:06.247092 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.235752 14040 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:06.237617 14076 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:06.224326 12705 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:06.225692 12634 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:06.298408 12706 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
W20260709 13:18:06.300155 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 59818.5us
W20260709 13:18:06.332113 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 315064us
I20260709 13:18:06.345237 13385 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:06.345433 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:06.226382 14001 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:06.352603 12747 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:06.353120 13383 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:06.353664 13376 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEARNER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:18:06.346107 13976 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.354357 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:06.354964 13976 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:06.355212 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:06.337483 14076 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.341756 13977 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.301949 13924 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:06.351816 14077 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:06.355897 13377 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:06.360096 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:06.358965 14076 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.350426 14058 consensus_queue.cc:1059] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:18:06.368039 13981 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:06.381099 13981 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:06.385795 13981 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.386437 14001 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:06.388443 14070 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:06.368443 13977 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:06.394368 12705 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:18:06.144516 13958 consensus_queue.cc:1175] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3d539f8d95d343d8b1696803f18b30d2" responder_term: 2 status { last_received { term: 1 index: 1 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:18:06.398111 14091 log.cc:927] Time spent T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:18:06.401110 13981 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:06.403568 13981 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:06.389504 13924 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:06.408991 13382 raft_consensus.cc:2393] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 2: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:06.406399 14013 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:06.419724 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.456836 14088 raft_consensus.cc:1064] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:18:06.457706 13923 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:06.458204 13923 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.459441 14088 consensus_queue.cc:237] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:06.463743 14049 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:06.438611 14013 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:06.484426 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 99379.1us
W20260709 13:18:06.485044 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.487468 13348 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:06.480546 13976 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:06.502381 14070 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:06.495064 12704 consensus_queue.cc:260] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:06.491648 14022 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 13:18:06.511247 12704 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:06.512743 12747 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.65:43603 timed out after 1.733s (SENT) [suppressed 26 similar messages]
I20260709 13:18:06.515301 14037 raft_consensus.cc:3055] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:06.515817 14037 raft_consensus.cc:740] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:06.503180 14085 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:06.522794 14076 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:06.525914 13378 raft_consensus.cc:2368] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3.
I20260709 13:18:06.525296 14076 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:06.515462 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:06.509048 13976 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
W20260709 13:18:06.542697 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.546070 12704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1243) took 1302 ms. Trace:
I20260709 13:18:06.546530 12704 rpcz_store.cc:276] 0709 13:18:05.231991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.285884 (+ 53893us) service_pool.cc:224] Handling call
0709 13:18:05.592416 (+306532us) spinlock_profiling.cc:243] Waited 8.07 ms on lock 0x7b5800002798. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c7238bfc0 00007f4c72376929 00007f4c723763d2 00007f4c6b8edbd4
0709 13:18:05.592620 (+   204us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:05.982751 (+390131us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b5400047ca0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d329dc0 00007f4c6d3b41c0 00007f4c6d3b5231 00007f4c6d3b1d8f 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c
0709 13:18:06.494554 (+511803us) spinlock_profiling.cc:243] Waited 9.97 ms on lock 0x7b4c0027b080. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6b67ea20 00007f4c6b68fa29 00007f4c6b655644 00007f4c6b67cf2d 00007f4c6b67cb93 00007f4c6d3ad18b 00007f4c6d3adcb5 00007f4c6d3a8f30 00007f4c6d3aa7de 00007f4c6d3b4b49 00007f4c6d3b5416
0709 13:18:06.533936 (+ 39382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:06.533969 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:06.534007 (+    38us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:06.534033 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:06.534101 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:06.534224 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6202,"spinlock_wait_cycles":488462208}
I20260709 13:18:06.551227 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:06.551911 12704 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.516806 14037 consensus_queue.cc:260] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:06.562108 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:06.546504 14052 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:18:06.549934 12701 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:06.550347 13977 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:06.562813 12625 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:06.565068 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:06.573058 12673 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:06.573272 14013 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:06.574096 12673 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:06.587237 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 74518.8us
W20260709 13:18:06.587793 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:06.591003 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.592149 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:06.592790 12782 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 1.
I20260709 13:18:06.578895 13997 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:06.577843 13854 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:06.578065 14078 raft_consensus.cc:3055] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:06.597210 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:06.588871 14037 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:06.601186 12776 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:06.604878 12776 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:18:06.605417 12776 raft_consensus.cc:606] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Received request to transfer leadership to TS 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:06.593357 12634 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:06.606725 12776 raft_consensus.cc:614] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:06.598868 13997 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.608426 12704 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:06.612520 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:18:06.613906 14095 log.cc:927] Time spent T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:18:06.614300 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:06.613507 12529 catalog_manager.cc:5697] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 1 to 2, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 2 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:06.611500 12708 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:18:06.617755 14082 log.cc:927] Time spent T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.364s	user 0.003s	sys 0.001s
I20260709 13:18:06.618860 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:06.619513 12700 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:06.620639 13999 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:06.620568 13923 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:06.622068 12782 raft_consensus.cc:1240] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:06.622869 13923 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:06.623066 14041 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:06.623939 13997 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.626439 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:06.627054 12628 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:06.628057 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:06.629225 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:06.630390 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:06.632644 12673 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:06.600950 14078 raft_consensus.cc:740] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:06.635377 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:06.635954 12782 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.638787 12709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 894) took 1377 ms. Trace:
I20260709 13:18:06.639151 12709 rpcz_store.cc:276] 0709 13:18:05.261398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.384294 (+122896us) service_pool.cc:224] Handling call
0709 13:18:05.384631 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:06.006486 (+621855us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:06.006513 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:06.006548 (+    35us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:06.006574 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:06.638568 (+631994us) spinlock_profiling.cc:243] Waited 622 ms on lock 0x7b540021f2d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:06.638611 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:06.638734 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1429707136}
I20260709 13:18:06.641433 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:06.641819 14026 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:06.642155 12779 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 1.
I20260709 13:18:06.643452 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:06.643277 14078 consensus_queue.cc:260] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:06.644326 14078 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.636159 14041 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:06.650172 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:06.635555 13923 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:06.651274 13385 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
W20260709 13:18:06.651995 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.651050 14041 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.656977 14041 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.660681 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:06.661489 14022 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.661408 13383 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:06.662027 14022 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:06.662760 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:06.663632 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:06.664047 14022 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:06.661007 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.667145 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:06.669539 12782 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:06.669811 13965 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:06.670081 12781 raft_consensus.cc:1275] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:06.670202 13965 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.670805 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:06.670928 13965 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.706090 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:06.672755 14049 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.747007 14049 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:06.747448 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:06.750691 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:06.751413 12783 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 2: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:06.751464 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:06.752616 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:06.753194 12778 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
W20260709 13:18:06.755853 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 82547.3us
I20260709 13:18:06.756275 12779 raft_consensus.cc:1275] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:06.762207 14026 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.653703 13923 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:06.762799 14026 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:06.764199 13923 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:06.764829 14026 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:06.783470 12746 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:06.790827 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:06.791829 13382 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:06.800390 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:06.800681 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:06.801602 12630 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.802299 12632 raft_consensus.cc:1240] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 2. Current term is 3. Ops: []
W20260709 13:18:06.802947 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 130621us
W20260709 13:18:06.804121 12706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1220) took 1601 ms (client timeout 1825 ms). Trace:
I20260709 13:18:06.805590 12629 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:06.807861 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:06.808686 12702 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:18:06.749276 14049 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:06.810141 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:06.801414 12625 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:06.797425 12597 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:06.811614 12707 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 2. Current term is 3. Ops: []
W20260709 13:18:06.811817 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.820577 12625 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:18:06.822057 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:06.804647 12706 rpcz_store.cc:269] 0709 13:18:05.194797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.195031 (+   234us) service_pool.cc:224] Handling call
0709 13:18:06.795615 (+1600584us) spinlock_profiling.cc:243] Waited 910 ms on lock 0x7b540006de54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:06.795722 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2093204352}
I20260709 13:18:06.822175 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:06.823051 13382 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:06.825111 12628 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:06.822059 12629 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:06.822781 12746 leader_election.cc:400] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3.
I20260709 13:18:06.827261 12746 leader_election.cc:403] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:06.828724 13348 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:06.830478 12631 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:06.830312 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:06.831115 12626 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:06.833201 12746 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:06.833695 13348 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:06.804322 13966 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:06.813738 14087 raft_consensus.cc:3055] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:06.835376 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:06.836405 13348 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:06.838032 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:06.847018 12626 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:06.838843 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:06.835323 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:06.845082 12630 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:06.818284 13958 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:06.822170 14040 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:06.818547 14096 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:06.855602 13349 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:06.857210 13349 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:06.796548 13968 consensus_queue.cc:1059] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:18:06.826783 13981 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.839229 14065 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:06.824208 14031 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:18:06.819109 13923 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:18:06.815176 14022 consensus_queue.cc:1059] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:18:06.848958 14076 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:06.877203 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:06.878463 12630 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:06.846447 14087 raft_consensus.cc:740] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:06.861366 14058 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:06.872064 13899 raft_consensus.cc:3055] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:06.850129 13966 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3.
I20260709 13:18:06.854120 14040 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.852680 13971 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:06.858155 14073 raft_consensus.cc:2764] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:18:06.858240 14096 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.838984 13977 consensus_queue.cc:1059] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.036s
W20260709 13:18:07.008463 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 151325us
I20260709 13:18:06.989399 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:06.943663 12747 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:06.982653 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:07.018146 12777 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:06.999112 12628 raft_consensus.cc:1240] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 1. Current term is 2. Ops: []
W20260709 13:18:06.860809 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.032249 12631 raft_consensus.cc:1275] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:07.032634 12672 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
W20260709 13:18:07.033861 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 90277.7us
I20260709 13:18:07.034184 13349 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:06.875118 14065 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.041545 12747 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:06.865722 13958 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:06.870458 13981 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:06.866791 13997 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:06.941936 14088 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:07.025801 14085 raft_consensus.cc:3055] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:06.944434 13999 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:06.940740 14076 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:07.090154 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 45790.5us
I20260709 13:18:07.071604 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:06.973799 13968 raft_consensus.cc:3055] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:07.071733 14073 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:07.018949 13996 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.091488 14088 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:07.045282 14086 log.cc:927] Time spent T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.804s	user 0.002s	sys 0.010s
I20260709 13:18:06.989548 13854 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:07.102437 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 69900.3us
I20260709 13:18:07.036131 14040 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.018146 14096 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.965296 13994 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.040055 13971 raft_consensus.cc:697] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:07.034480 13976 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
W20260709 13:18:07.107658 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 274502us
I20260709 13:18:07.127815 12782 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:07.097549 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:07.129074 12629 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.103427 12631 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:18:07.044165 14112 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:07.092861 14085 raft_consensus.cc:740] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:06.994745 14022 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:07.074663 13899 raft_consensus.cc:740] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:07.107268 14088 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.140988 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:07.095422 13997 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:07.044646 14065 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.044215 14031 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:18:07.105535 14076 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.104332 13996 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:07.106659 13968 raft_consensus.cc:740] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:07.105778 14110 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:07.135169 12628 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:07.163309 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:07.164001 12631 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:06.988085 14087 consensus_queue.cc:260] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:07.169616 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 119675us
I20260709 13:18:07.148141 13969 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:07.138994 13923 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:07.137851 14112 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.178010 13383 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 3 LEARNER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:18:07.178277 14109 log.cc:927] Time spent T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
W20260709 13:18:07.178874 14106 log.cc:927] Time spent T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:18:07.180676 13923 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.137843 14115 consensus_queue.cc:1059] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:18:07.175762 14078 consensus_queue.cc:1059] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:18:07.181823 13971 consensus_queue.cc:237] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:07.182626 13965 consensus_meta.cc:287] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.511s	user 0.008s	sys 0.003s
I20260709 13:18:07.184154 14049 raft_consensus.cc:3055] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:18:07.171661 14108 log.cc:927] Time spent T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 13:18:07.184685 12529 catalog_manager.cc:5697] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 1 to 2, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 2 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:18:07.186529 14037 consensus_meta.cc:287] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.578s	user 0.005s	sys 0.005s
I20260709 13:18:07.182643 13958 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:07.182047 13969 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.190074 14085 consensus_queue.cc:260] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:07.191105 14119 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:07.182716 12528 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54800 (request call id 227) took 3691 ms. Trace:
I20260709 13:18:07.147459 14022 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:07.196870 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.137907 13994 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.198259 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:07.199612 13924 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:07.199306 13899 consensus_queue.cc:260] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.187913 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:07.200068 14120 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.175709 14110 raft_consensus.cc:697] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:07.117229 13854 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:07.266460 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:07.206604 12630 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
W20260709 13:18:07.215595 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 111830us
I20260709 13:18:07.267850 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:07.163893 13997 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:07.220206 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.244307 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:07.265789 12627 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:18:07.214731 14112 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.248651 12528 rpcz_store.cc:276] 0709 13:18:03.491386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.493442 (+  2056us) service_pool.cc:224] Handling call
0709 13:18:07.182646 (+3689204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":370776,"reported_tablets":5}
I20260709 13:18:07.253432 12628 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:07.254539 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:07.256646 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.257723 12632 raft_consensus.cc:2368] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3.
I20260709 13:18:07.250463 12633 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:07.238920 14049 raft_consensus.cc:740] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:07.264786 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:07.276677 12702 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:07.277244 14049 consensus_queue.cc:260] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:07.264945 12707 raft_consensus.cc:1240] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:07.278503 14049 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:07.268448 12630 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:07.268999 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:07.279693 12708 raft_consensus.cc:2376] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2. Re-sending same reply.
I20260709 13:18:07.231791 14040 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:07.288985 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:07.289535 12672 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:07.249800 14119 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:07.484266 13996 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:07.504501 14041 consensus_meta.cc:287] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.847s	user 0.010s	sys 0.002s
I20260709 13:18:07.504840 13969 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:07.505463 13969 consensus_queue.cc:1059] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:07.503353 14119 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.282289 13967 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:07.262427 13924 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:07.256511 14058 consensus_queue.cc:1175] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "639ab2db265f4747bb7afb93e833f530" responder_term: 3 status { last_received { term: 2 index: 5 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:18:07.512637 14022 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:07.513931 13924 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:07.257014 14118 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:07.514882 14134 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.514719 14118 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:07.515409 14134 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.372152 14031 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.243680 13968 consensus_queue.cc:260] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:07.283012 13997 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.422724 14033 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.324239 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:07.386515 14117 log.cc:927] Time spent T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:18:07.295370 13966 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.092s
I20260709 13:18:07.517324 14118 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:07.383219 13981 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:07.356880 14115 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:18:07.256174 13958 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:07.265727 13983 consensus_queue.cc:1048] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.098s
I20260709 13:18:07.241384 13965 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:07.420691 14124 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.286103 14037 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:07.282384 14085 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.538062 14040 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:07.539067 14130 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.370668 13899 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:07.266934 14120 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:07.487946 13994 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:07.383917 14127 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:18:07.471381 14098 log.cc:927] Time spent T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:18:07.326928 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 54507.2us
I20260709 13:18:07.427349 14126 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:18:07.474267 14129 log.cc:927] Time spent T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:18:07.498184 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:07.514735 14121 log.cc:927] Time spent T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 13:18:07.521405 14029 raft_consensus.cc:3055] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:18:07.325265 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 49195.3us
I20260709 13:18:07.502532 14131 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:07.282748 13973 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.097s
W20260709 13:18:07.583247 14128 log.cc:927] Time spent T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:18:07.396555 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:07.517554 14134 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:07.586131 14134 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.586607 14134 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:07.588905 14134 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:07.506953 13967 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.519068 14033 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.590837 13967 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:07.527235 12629 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:07.513303 14022 raft_consensus.cc:3055] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:07.591523 14022 raft_consensus.cc:740] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:07.486044 14110 consensus_queue.cc:237] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:07.523509 14133 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:07.528272 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 238808us
I20260709 13:18:07.596885 12672 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:07.504993 14041 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:07.599311 14041 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:07.599994 14041 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:07.600039 14120 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:07.600665 14041 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.544144 14127 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.603374 14127 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:07.502274 14052 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:07.604173 14052 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.525362 14124 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.606642 14052 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:07.607425 12701 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:07.570691 12625 raft_consensus.cc:2393] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:07.332847 12706 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:07.610631 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:07.611380 12706 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:18:07.520186 13997 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:07.302145 13382 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.534657 13996 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.576509 12598 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:07.379196 12626 raft_consensus.cc:2393] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:07.524235 13965 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:07.528474 13923 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:07.582217 14140 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:07.585322 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:07.511585 13854 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:07.521812 13958 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:07.520526 14118 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:07.594810 14033 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:07.616364 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:07.607856 14124 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:07.623078 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:07.623939 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:07.627375 14087 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.517974 14031 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:07.635000 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:07.554456 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.637367 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:18:07.682219 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.630370 14088 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:07.686019 12672 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:07.640918 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:07.641762 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:07.595518 14131 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.574306 14126 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.577211 14029 raft_consensus.cc:740] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:07.647135 14052 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:07.648909 14041 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:07.652154 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 296916us
I20260709 13:18:07.655690 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:18:07.656746 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 81344.5us
I20260709 13:18:07.769058 13384 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.649487 14127 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:07.667490 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:07.610038 14134 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:07.543696 14037 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.663921 14013 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:07.659101 13997 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:18:07.762265 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 76364.2us
I20260709 13:18:07.742180 13923 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.763036 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:07.654642 13996 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.764369 13383 raft_consensus.cc:2393] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 2: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:07.678364 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:07.650357 14133 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.766208 14029 consensus_queue.cc:260] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:07.672245 13854 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.771421 12708 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:07.675733 14118 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.774582 12747 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:07.778920 12778 raft_consensus.cc:2393] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 3: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:07.779297 12598 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:07.770759 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:07.681408 14124 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:07.594856 14022 consensus_queue.cc:260] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:07.767567 14096 consensus_meta.cc:287] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.630s	user 0.009s	sys 0.002s
I20260709 13:18:07.788100 12777 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:07.788640 12673 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:07.671554 14076 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:07.742463 12774 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
W20260709 13:18:07.833700 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 45161.8us
I20260709 13:18:07.665975 14065 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:07.771961 13379 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:07.809448 12598 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:07.778559 14041 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.788866 14052 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:07.784420 14013 raft_consensus.cc:697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:07.820287 12706 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
W20260709 13:18:07.828275 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.778117 14127 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.814517 14142 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.778699 14134 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.832928 12672 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:07.834748 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 64635.5us
I20260709 13:18:07.826069 14029 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.787566 13997 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:07.843487 12747 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:07.846932 12598 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:07.848569 12598 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:07.851290 12598 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:07.796753 13923 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.870443 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:07.838969 14033 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:07.818298 13854 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.840739 14139 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:07.880237 14139 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:07.880604 14137 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:07.880748 14139 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.881269 13958 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:07.881379 14137 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:07.881902 13958 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:07.882058 14137 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:07.882426 13958 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:07.885133 13384 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:07.843955 14052 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:07.841305 14144 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:07.830525 14124 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:07.857981 14049 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:07.874866 14040 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:18:07.905730 12707 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.019s	sys 0.039s
I20260709 13:18:07.843130 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:07.908495 13380 raft_consensus.cc:2393] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 3: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:07.879242 14033 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:18:07.911157 12707 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.019s	sys 0.039s
I20260709 13:18:07.836372 14096 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:07.849882 14041 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.892798 13899 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:07.898591 14052 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.896605 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:07.875303 13972 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:18:07.905035 14049 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:07.885305 13967 consensus_queue.cc:1048] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:18:07.888666 13994 consensus_queue.cc:1059] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:18:07.918546 14033 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:07.933924 12782 raft_consensus.cc:2393] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 3: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:07.928525 12781 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:07.877648 14122 consensus_queue.cc:1059] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:18:07.947790 13382 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:07.949522 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:07.947726 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:07.921468 14112 consensus_meta.cc:287] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.628s	user 0.009s	sys 0.003s
I20260709 13:18:07.908581 14144 raft_consensus.cc:697] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:07.929643 13899 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.952821 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:07.956348 12629 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:07.951586 12625 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:07.916142 14040 raft_consensus.cc:697] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:07.949606 14031 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:07.961877 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 110697us
I20260709 13:18:07.905505 14119 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.942592 14049 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:07.868008 14142 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:07.985738 12709 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:07.962909 12598 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:07.952170 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 119316us
I20260709 13:18:07.934922 12774 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:07.958370 13899 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.958273 14131 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:07.972853 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:07.986900 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:07.965466 14126 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:07.956610 14148 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:07.988466 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:07.991336 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:07.993765 12629 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.972664 13996 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:07.994938 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:08.009127 12634 raft_consensus.cc:2393] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 2: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:08.010375 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:07.984557 14110 raft_consensus.cc:3055] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:07.973920 14037 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:08.019634 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:07.957548 14112 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:08.003978 12598 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:07.933120 14013 consensus_queue.cc:237] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:08.011006 12633 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:08.000221 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:07.987272 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:07.990239 14118 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:07.990243 13972 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:08.020197 13996 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:08.017367 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:08.025189 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:08.057982 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 54090.7us
I20260709 13:18:08.026382 12629 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:08.027052 12672 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:08.036486 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:08.063752 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:08.015897 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:08.015416 14065 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:08.065364 12776 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:08.020306 14155 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:08.067963 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:08.030845 14145 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:08.069453 12633 raft_consensus.cc:2393] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 2: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:08.062021 12598 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:08.014144 14110 raft_consensus.cc:740] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:08.002220 14131 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:08.066022 12632 raft_consensus.cc:2393] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 2: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:08.066114 12625 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:08.030411 13923 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:08.076946 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:08.019587 12774 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:08.021518 14133 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:08.088387 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:08.055428 13968 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:08.074775 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:08.073107 13962 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:18:08.075268 14146 log.cc:927] Time spent T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:18:08.031284 14126 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:08.010185 14148 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:08.021162 14040 consensus_queue.cc:237] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:08.014122 14144 consensus_queue.cc:237] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:08.083779 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:08.075964 14155 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.065272 14118 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:08.021108 14050 consensus_queue.cc:1059] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.178s
I20260709 13:18:08.082922 14157 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:08.090579 13382 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:08.095302 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:08.108062 14118 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.111387 14050 raft_consensus.cc:3055] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:08.111656 14118 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:08.074115 14124 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:08.056241 14127 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:08.113708 14154 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
W20260709 13:18:08.114326 14150 log.cc:927] Time spent T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:18:08.114573 14138 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:08.115911 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:08.117295 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:08.118446 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:08.118659 14149 log.cc:927] Time spent T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
W20260709 13:18:08.119935 14147 log.cc:927] Time spent T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 13:18:08.121399 14142 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:08.111826 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:08.123253 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:08.064729 13996 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:08.098026 14130 consensus_meta.cc:287] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.558s	user 0.013s	sys 0.000s
W20260709 13:18:08.106436 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.105744 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:08.062772 13972 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:08.082984 14153 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.068032 14156 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:08.111826 14050 raft_consensus.cc:740] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:08.082168 14131 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:08.084359 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:08.173657 14119 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:08.174201 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 89983.5us
I20260709 13:18:08.174331 14119 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:08.174734 14119 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:08.174862 13976 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:08.175132 14119 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:08.175352 14131 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:08.175436 13976 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:08.176491 12672 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:08.078186 14145 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:08.151655 14127 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:08.177999 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:08.178221 14127 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:08.179188 14133 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.179685 14133 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:08.180318 14127 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:08.180910 14127 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:08.181239 14127 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.181583 14133 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:08.181654 14031 raft_consensus.cc:3055] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:08.182196 14031 raft_consensus.cc:740] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:08.181591 14127 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:08.183058 14031 consensus_queue.cc:260] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:08.184243 14031 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:08.187181 12633 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:18:08.188583 13976 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:08.188855 12708 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:18:08.195066 14119 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:08.195502 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:08.196146 12633 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:08.197156 14127 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:08.110332 14155 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:08.198169 14037 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:08.198819 14037 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:08.199563 14127 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:08.200928 14037 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:08.163259 12780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 92) took 1510 ms. Trace:
I20260709 13:18:08.163681 14049 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:08.151798 14130 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:08.125463 14139 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:08.140060 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:08.202381 12702 raft_consensus.cc:1240] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:08.185010 14118 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:08.201608 12780 rpcz_store.cc:276] 0709 13:18:06.652306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.652822 (+   516us) service_pool.cc:224] Handling call
0709 13:18:06.655481 (+  2659us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:06.655837 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:06.655868 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:06.656364 (+   496us) log.cc:844] Serialized 26 byte log entry
0709 13:18:06.877727 (+221363us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:06.937792 (+ 60065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:06.950466 (+ 12674us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:08.162321 (+1211855us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b24001f9e68. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a14b9c4 00007f4c6a149acb 00007f4c6a304dc9 00007f4c6a304a55 00007f4c6a303fb0 00007f4c6cf7a488 00007f4c6cf7abe8 00007f4c6cf52398 00007f4c6cf4d76a 00007f4c6cf4cbb6
0709 13:18:08.162930 (+   609us) raft_consensus.cc:1685] finished
0709 13:18:08.162955 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:08.163168 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1316,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":91,"spinlock_wait_cycles":24193792,"thread_start_us":203946,"threads_started":1,"wal-append.queue_time_us":333998}
I20260709 13:18:08.203619 14139 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:08.204555 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:08.123584 12703 raft_consensus.cc:1275] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:08.165819 14050 consensus_queue.cc:260] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:08.179207 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:08.210825 12672 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:08.212419 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:08.213424 12706 raft_consensus.cc:2393] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 2: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:08.164830 13899 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:08.214120 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.215600 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:08.151288 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:08.216760 14118 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:08.216895 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:08.217252 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:08.217712 13382 raft_consensus.cc:2393] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:08.217777 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:08.214951 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:08.218437 12779 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:18:08.218689 12781 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:18:08.197130 13854 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:08.220937 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:08.227648 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:08.221936 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:08.100554 14126 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:08.170109 14052 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:08.228736 13382 raft_consensus.cc:2368] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3.
I20260709 13:18:08.221457 12625 raft_consensus.cc:2393] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:08.231274 13349 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:08.218689 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:08.218626 14050 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:08.126051 13996 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:08.231922 12746 leader_election.cc:400] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3.
I20260709 13:18:08.237831 12746 leader_election.cc:403] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:18:08.233894 12672 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:08.231936 12632 raft_consensus.cc:2393] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 2: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:08.188395 13983 raft_consensus.cc:2955] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:08.108785 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:08.205973 14049 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:08.098361 14041 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:08.219262 14087 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:08.240942 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:08.244081 12778 raft_consensus.cc:2393] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:08.109072 14096 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:08.244050 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 172282us
I20260709 13:18:08.246031 13967 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:08.219126 14131 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:08.218878 14058 consensus_queue.cc:1059] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:18:08.136045 14142 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:08.249429 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.219631 14037 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:08.240610 12672 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:08.256659 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:08.197201 14119 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:08.260354 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.247262 13854 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:08.249883 14029 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:08.246601 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:08.272435 12598 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:08.289961 14040 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:08.290565 14040 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:08.291078 14040 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:08.272241 12633 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:08.299651 14096 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:08.300494 14096 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:08.270941 13854 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:08.306458 13854 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:08.260686 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:08.259371 14131 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:08.136788 14110 consensus_queue.cc:260] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:08.271613 14142 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:08.232434 14153 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:08.386384 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:08.387496 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:08.388460 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 148676us
W20260709 13:18:08.389101 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 157874us
I20260709 13:18:08.255820 12774 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
W20260709 13:18:08.265553 14132 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.012s	sys 0.015s
W20260709 13:18:08.389664 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.391068 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:08.392395 13349 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:08.392427 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:08.259629 12781 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:08.400977 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:08.401876 12781 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:18:08.402922 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:08.404949 14041 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:08.312311 13377 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:08.407953 12598 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:08.270813 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:08.410983 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:08.422080 12780 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:08.271529 14037 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:08.232774 14156 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:08.394310 14132 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.019s	sys 0.021s
I20260709 13:18:08.365921 14148 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:08.336802 14158 log.cc:927] Time spent T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:18:08.363317 14126 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:08.411653 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:08.417546 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:08.416085 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:08.395058 14144 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:08.407097 14058 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:08.430838 13379 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:08.431162 14058 raft_consensus.cc:697] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:08.263252 14087 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:08.419921 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:08.407284 14131 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:08.271312 14112 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:08.411108 12779 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:08.427481 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:08.433807 12632 raft_consensus.cc:2393] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 2: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:08.357964 14040 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:08.423254 13972 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:08.430467 12672 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:08.428995 13899 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:08.362108 14155 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:08.436620 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:08.440819 14138 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:08.427356 14052 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:08.433318 14126 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:08.423091 14041 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:08.445086 12598 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:08.432230 14144 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.450218 12633 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:08.430887 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:08.462525 12672 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:08.467778 14137 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.449571 14148 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:08.444454 14096 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:08.479902 14148 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.474009 14126 raft_consensus.cc:697] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 2 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:08.480971 12774 raft_consensus.cc:2410] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:18:08.481258 12672 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:08.480348 14148 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:08.482200 13996 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:08.482618 13996 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:08.483018 13996 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:08.456446 14076 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:08.480336 13979 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:08.468592 14144 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:08.457859 14138 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.484845 14119 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:08.459337 13972 consensus_queue.cc:1059] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:18:08.473358 14137 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:08.481348 14126 consensus_queue.cc:237] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:08.503614 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:18:08.771183 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.884032 12634 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:08.665999 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 912) took 1104 ms. Trace:
I20260709 13:18:08.567240 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:18:08.886116 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 441076us
I20260709 13:18:08.890486 12704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 911) took 1329 ms. Trace:
I20260709 13:18:08.657487 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:08.890901 12704 rpcz_store.cc:276] 0709 13:18:07.561336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.561503 (+   167us) service_pool.cc:224] Handling call
0709 13:18:07.561926 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.602144 (+ 40218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:07.602175 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:07.602217 (+    42us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:07.602250 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:08.890260 (+1288010us) spinlock_profiling.cc:243] Waited 40 ms on lock 0x7b540006d454. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:08.890305 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:08.890430 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92038400}
W20260709 13:18:08.477165 14033 consensus_meta.cc:287] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.521s	user 0.004s	sys 0.006s
I20260709 13:18:08.882259 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:08.881836 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.885048 12702 rpcz_store.cc:276] 0709 13:18:07.561667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.563265 (+  1598us) service_pool.cc:224] Handling call
0709 13:18:07.563534 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:08.202868 (+639334us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:08.665654 (+462786us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b54001cd4d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:08.665915 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1468769664}
I20260709 13:18:08.656288 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:08.602681 12626 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56838 (request call id 895) took 1724 ms (client timeout 2079 ms). Trace:
W20260709 13:18:08.899608 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:08.900115 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:08.900607 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.901131 12628 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:08.902185 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:08.729509 13384 raft_consensus.cc:2955] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:08.903493 12782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36488 (request call id 1356) took 1280 ms. Trace:
W20260709 13:18:08.490679 14160 log.cc:927] Time spent T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:18:08.912490 12746 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:18:08.899878 12626 rpcz_store.cc:269] 0709 13:18:06.876617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.876815 (+   198us) service_pool.cc:224] Handling call
0709 13:18:08.600908 (+1724093us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b54002db9d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:08.601090 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1014039296}
W20260709 13:18:08.543929 14169 log.cc:927] Time spent T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
W20260709 13:18:08.920653 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.922099 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:08.450664 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:08.903923 12782 rpcz_store.cc:276] 0709 13:18:07.614665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.615916 (+  1251us) service_pool.cc:224] Handling call
0709 13:18:08.715804 (+1099888us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b5400127854. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:08.895467 (+179663us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":565249792}
I20260709 13:18:08.502650 14138 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:08.929311 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:08.566557 14145 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:08.662686 14088 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:08.929700 14140 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:08.946691 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:08.954360 12707 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:18:08.955292 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:08.955997 12634 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:08.956252 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:08.864364 14076 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.930065 14112 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:08.930152 14159 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.504565 14058 consensus_queue.cc:237] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:08.548748 14029 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:08.723937 12701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 87) took 1916 ms. Trace:
I20260709 13:18:08.913183 14033 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:08.933746 14096 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:08.934170 13899 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.934590 14050 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.936152 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:08.936744 13968 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:08.531380 13979 raft_consensus.cc:697] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:08.925278 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:18:09.010286 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 85081.1us
I20260709 13:18:09.010973 12597 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:08.502298 14119 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:08.504698 14134 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:09.026142 14134 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:09.026722 14134 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:09.029165 14134 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:08.630648 13377 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:08.502311 14151 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:08.941733 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:09.031361 12704 raft_consensus.cc:2393] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 3: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:09.032994 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:08.911597 13983 consensus_meta.cc:287] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.528s	user 0.009s	sys 0.002s
I20260709 13:18:08.530520 13967 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3.
I20260709 13:18:08.708389 12700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1271) took 1441 ms. Trace:
I20260709 13:18:08.942873 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:08.945322 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:09.045393 12706 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:18:08.631561 14037 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:08.705415 14166 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:08.895188 14040 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:08.944604 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:09.001984 14087 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:09.051131 12708 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:18:09.051892 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:09.052373 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:09.006922 14052 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:08.916720 14118 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:08.980093 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:09.035383 14119 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:08.929749 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:18:08.991050 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:09.073261 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:09.074229 12747 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:08.995036 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:08.951287 12780 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.044668 13976 raft_consensus.cc:3055] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:09.000874 14159 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:09.016938 13379 consensus_meta.cc:287] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.585s	user 0.008s	sys 0.003s
I20260709 13:18:08.978294 14139 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:08.985896 14155 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:09.001521 14096 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.959897 12707 raft_consensus.cc:2393] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 3: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
W20260709 13:18:09.099236 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 173228us
I20260709 13:18:09.100894 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:09.000473 13969 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:18:09.041272 14031 consensus_meta.cc:287] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.857s	user 0.009s	sys 0.005s
I20260709 13:18:09.053241 12708 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:18:08.599560 13854 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:09.067878 14087 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.069612 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:09.075815 12782 raft_consensus.cc:2393] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 2: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:08.952127 14145 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.046219 14154 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:08.991539 14076 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:08.990542 14112 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:09.040014 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:09.044974 14026 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:09.043557 12700 rpcz_store.cc:276] 0709 13:18:07.267175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.272049 (+  4874us) service_pool.cc:224] Handling call
0709 13:18:07.522207 (+250158us) spinlock_profiling.cc:243] Waited 32.7 ms on lock 0x7b5800002798. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c7238bfc0 00007f4c72376929 00007f4c723763d2 00007f4c6b8edbd4
0709 13:18:07.522450 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.772083 (+249633us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:07.773701 (+  1618us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:07.778737 (+  5036us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:07.786504 (+  7767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:08.708050 (+921546us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b540011c1d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:08.708126 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:08.708309 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":456441344}
I20260709 13:18:09.007572 13899 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:09.012919 14050 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:08.990300 14140 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:09.013192 13968 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:09.098606 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 172761us
I20260709 13:18:09.030799 14151 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:09.101109 12598 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:09.072129 14134 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:09.081899 13997 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:09.079371 14144 consensus_meta.cc:287] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.548s	user 0.009s	sys 0.000s
I20260709 13:18:08.993721 12701 rpcz_store.cc:276] 0709 13:18:06.807725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.815526 (+  7801us) service_pool.cc:224] Handling call
0709 13:18:06.839399 (+ 23873us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.607951 (+768552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:08.705892 (+1097941us) spinlock_profiling.cc:243] Waited 760 ms on lock 0x7b5400249ad4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:08.723825 (+ 17933us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1748651264}
I20260709 13:18:09.083375 13976 raft_consensus.cc:740] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:09.091686 13379 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:09.121438 12783 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:09.240278 14144 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:09.056478 14166 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:09.077896 14052 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:09.245652 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:09.246412 12706 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:18:09.246564 13381 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:09.247794 14052 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:09.105391 14096 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.080356 14118 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:09.079921 14088 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:09.120814 13971 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.128664 13979 consensus_queue.cc:237] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:09.102121 14139 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:09.152073 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 136469us
I20260709 13:18:09.266639 12597 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:09.267791 14157 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:09.131139 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:09.268186 14157 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.268471 12703 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.268641 14157 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.117301 13969 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:09.199658 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:09.271611 12708 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:09.208097 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 107101us
I20260709 13:18:09.204316 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:09.234862 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 160722us
I20260709 13:18:09.147737 14145 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:09.149889 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:09.280694 13382 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 2: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:09.262631 12707 raft_consensus.cc:2368] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3.
I20260709 13:18:09.144798 14087 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:09.252420 12778 raft_consensus.cc:2955] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:09.269886 13971 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:09.273099 12747 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:09.157186 14154 raft_consensus.cc:697] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:09.275354 14112 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:09.277268 12781 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:09.215476 14137 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:09.234982 14026 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:09.206369 12705 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:18:09.285203 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:09.295406 14174 log.cc:927] Time spent T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:18:09.295553 13968 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:09.295701 14173 log.cc:927] Time spent T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:18:09.237991 14134 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:09.691989 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 418921us
I20260709 13:18:09.377215 12783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36478 (request call id 1609) took 1383 ms. Trace:
W20260709 13:18:09.725526 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 440422us
W20260709 13:18:09.700480 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 480424us
I20260709 13:18:09.539844 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:09.619987 12708 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:09.312867 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:09.401139 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 242121us
I20260709 13:18:09.752657 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:09.366441 12709 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:09.757769 12782 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:09.758421 12598 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:09.765336 12707 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:09.725406 12783 rpcz_store.cc:276] 0709 13:18:07.993739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.178971 (+185232us) service_pool.cc:224] Handling call
0709 13:18:09.376962 (+1197991us) spinlock_profiling.cc:243] Waited 913 ms on lock 0x7b54001eb754. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:09.377133 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2100989184}
I20260709 13:18:09.292326 14144 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:09.770547 12708 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:09.420560 13958 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:09.422122 14142 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:09.415442 14085 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.263960 14088 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:09.322570 14172 log.cc:927] Time spent T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 13:18:09.284145 14182 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:09.375289 14022 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:09.528741 13923 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.249s
I20260709 13:18:09.559109 13967 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:09.386129 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:09.701963 14181 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:09.800001 14181 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:09.774917 13348 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.000s	sys 0.016s
I20260709 13:18:09.293016 14177 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:09.423795 14065 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:09.803828 13348 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.000s	sys 0.016s
W20260709 13:18:09.719192 14013 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.009s	sys 0.011s
W20260709 13:18:09.814587 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:09.827365 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:09.488934 14186 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:09.426101 14029 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:09.733955 14159 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:09.322109 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:09.836229 14029 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:09.836947 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:09.837586 13996 consensus_meta.cc:287] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 1.354s	user 0.004s	sys 0.017s
I20260709 13:18:09.777415 14178 consensus_queue.cc:1059] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:18:09.753275 14180 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:09.842386 14193 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.318835 13854 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:09.742954 12775 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.027s	sys 0.031s
W20260709 13:18:09.843196 12775 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.028s	sys 0.031s
I20260709 13:18:09.842912 14193 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:09.843665 14110 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:09.844084 12775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1355) took 2229 ms. Trace:
I20260709 13:18:09.767633 14052 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:09.818501 12703 consensus_meta.cc:287] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.549s	user 0.013s	sys 0.019s
W20260709 13:18:09.830770 14013 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.894s	user 0.010s	sys 0.012s
I20260709 13:18:09.767544 14155 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:09.770448 14118 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:09.792868 13385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 252) took 2045 ms. Trace:
I20260709 13:18:09.776918 14026 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:09.375887 13976 consensus_queue.cc:260] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:09.786345 14148 consensus_meta.cc:287] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.882s	user 0.009s	sys 0.008s
W20260709 13:18:09.795919 14179 log.cc:927] Time spent T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.319s	user 0.000s	sys 0.001s
I20260709 13:18:09.787664 13971 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:09.792512 14131 consensus_meta.cc:287] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 1.342s	user 0.008s	sys 0.011s
I20260709 13:18:09.785234 14085 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:09.313598 14154 consensus_queue.cc:237] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:09.830001 12673 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:09.837993 13996 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:09.900503 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 70619.4us
I20260709 13:18:09.902180 12673 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:09.822060 14065 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:09.821823 14177 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:09.841984 14029 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:09.811650 14096 consensus_meta.cc:287] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.563s	user 0.011s	sys 0.006s
I20260709 13:18:09.845031 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:09.796723 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:09.845854 13382 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:09.804199 14076 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:09.320423 14033 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:09.849402 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:09.844462 12775 rpcz_store.cc:276] 0709 13:18:07.614315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.632968 (+ 18653us) service_pool.cc:224] Handling call
0709 13:18:08.656789 (+1023821us) spinlock_profiling.cc:243] Waited 6.69 ms on lock 0x7b5800193e98. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c7238bfc0 00007f4c72376929 00007f4c723763d2 00007f4c6b8edbd4
0709 13:18:08.656994 (+   205us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:08.938777 (+281783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:08.944291 (+  5514us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:09.082813 (+138522us) log.cc:844] Serialized 26 byte log entry
0709 13:18:09.843617 (+760804us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:09.843663 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:09.843740 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:09.843796 (+    56us) raft_consensus.cc:1685] finished
0709 13:18:09.843807 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:09.844012 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":255774,"raft.queue_time_us":272536,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"spinlock_wait_cycles":15385984,"thread_start_us":714984,"threads_started":1,"wal-append.queue_time_us":698255}
I20260709 13:18:09.802668 14151 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:09.953779 13384 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:18:09.788516 14182 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:09.810240 14041 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:09.811554 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:09.839112 14119 consensus_meta.cc:287] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.746s	user 0.003s	sys 0.008s
I20260709 13:18:09.838487 14087 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:09.968189 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:09.968546 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:09.969640 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:09.970345 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:09.805508 13380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 319) took 1078 ms. Trace:
W20260709 13:18:09.852294 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:09.978360 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:09.978478 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:09.979601 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:09.853981 14153 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:09.854247 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:09.813676 12774 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.864517 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:09.855340 14052 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:09.869951 14145 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:09.856921 12703 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:09.837960 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:09.870147 14155 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:09.870930 12779 consensus_meta.cc:287] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 1.413s	user 0.012s	sys 0.007s
I20260709 13:18:09.993623 14136 raft_consensus.cc:3055] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:09.993897 12779 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
W20260709 13:18:09.994274 14196 log.cc:927] Time spent T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 13:18:09.994599 12779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36478 (request call id 1625) took 1761 ms (client timeout 1716 ms). Trace:
I20260709 13:18:09.804709 14181 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:09.871520 13385 rpcz_store.cc:276] 0709 13:18:07.744527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.744720 (+   193us) service_pool.cc:224] Handling call
0709 13:18:08.177986 (+433266us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b5800095198. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c7238bfc0 00007f4c72376929 00007f4c723763d2 00007f4c6b8edbd4
0709 13:18:08.178184 (+   198us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:08.178546 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:08.178572 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:08.689429 (+510857us) spinlock_profiling.cc:243] Waited 27.5 ms on lock 0x7b4c00044940. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6b67ea20 00007f4c6b68fa29 00007f4c6b655644 00007f4c6b67cf2d 00007f4c6b67cb93 00007f4c6d3ad18b 00007f4c6d3adcb5 00007f4c6d3af004 00007f4c6d3ae729 00007f4c6d3b3d96 00007f4c6d3a8adb
0709 13:18:08.857697 (+168268us) log.cc:844] Serialized 556 byte log entry
0709 13:18:08.986543 (+128846us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:08.986583 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:08.986650 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:09.789643 (+802993us) raft_consensus.cc:1685] finished
0709 13:18:09.789662 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:09.789775 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86251264,"thread_start_us":128485,"threads_started":1,"wal-append.queue_time_us":231183}
I20260709 13:18:09.847609 14193 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:09.994983 12779 rpcz_store.cc:269] 0709 13:18:08.233146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.241651 (+  8505us) service_pool.cc:224] Handling call
0709 13:18:09.994533 (+1752882us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:09.817729 14037 consensus_queue.cc:1048] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
W20260709 13:18:09.874963 14138 consensus_meta.cc:287] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.923s	user 0.008s	sys 0.015s
W20260709 13:18:09.891182 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 124669us
I20260709 13:18:09.844820 12701 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:18:09.817643 14189 consensus_queue.cc:1059] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 13:18:09.882335 12708 raft_consensus.cc:2955] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:09.904604 12673 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:10.018361 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 113840us
I20260709 13:18:10.019443 12673 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:10.020990 12673 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:09.905408 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:10.022045 12706 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
W20260709 13:18:09.910869 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 134761us
I20260709 13:18:09.916795 13899 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:09.918606 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:10.022812 12747 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:10.031209 12709 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:18:09.887255 14031 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:09.883414 14026 raft_consensus.cc:697] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:09.894992 13976 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.952844 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 923) took 1025 ms. Trace:
I20260709 13:18:09.906823 13377 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:18:09.955031 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 81578.8us
I20260709 13:18:09.908468 14118 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:09.963555 14013 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.046568 14013 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:09.978150 13380 rpcz_store.cc:276] 0709 13:18:08.726821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.728665 (+  1844us) service_pool.cc:224] Handling call
0709 13:18:08.729075 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:09.248020 (+518945us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:09.248088 (+    68us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:09.248158 (+    70us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:09.248211 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:09.805262 (+557051us) spinlock_profiling.cc:243] Waited 519 ms on lock 0x7b54005d9654. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:09.805310 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:09.805449 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1192813312}
I20260709 13:18:10.048264 13854 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:10.048746 14013 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:09.894073 14115 consensus_queue.cc:1175] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "639ab2db265f4747bb7afb93e833f530" responder_term: 3 status { last_received { term: 1 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:18:09.885926 14148 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:09.971704 12633 raft_consensus.cc:1240] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:10.051628 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:09.972591 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:10.053591 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:10.054468 14065 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:10.054877 12704 raft_consensus.cc:2368] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3.
I20260709 13:18:09.973007 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:10.055626 12626 raft_consensus.cc:2393] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:10.055480 14026 consensus_queue.cc:237] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:10.056440 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:10.057123 12626 raft_consensus.cc:2368] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3.
I20260709 13:18:09.968907 12632 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:10.058136 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:10.058646 12632 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:10.063110 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:09.839675 14186 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:09.981081 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:10.081269 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 100235us
I20260709 13:18:09.981752 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:09.898726 14131 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:09.984340 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:10.083325 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:09.988763 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:09.985945 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:09.948429 14096 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:10.088459 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:09.985822 13383 raft_consensus.cc:2410] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:18:09.994165 14136 raft_consensus.cc:740] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:09.886531 12782 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:10.041702 14153 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:09.968374 14119 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:09.990875 14088 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:10.031857 12597 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:10.021715 13376 raft_consensus.cc:2393] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 2: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:10.036221 12673 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:10.005968 12703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 938) took 1036 ms. Trace:
I20260709 13:18:10.100723 12703 rpcz_store.cc:276] 0709 13:18:08.959009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.110161 (+151152us) service_pool.cc:224] Handling call
0709 13:18:09.995727 (+885566us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:10.040115 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:10.104565 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 64514us
I20260709 13:18:10.105299 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:10.106230 12747 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:10.041518 12702 rpcz_store.cc:276] 0709 13:18:08.927433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.927583 (+   150us) service_pool.cc:224] Handling call
0709 13:18:08.928057 (+   474us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:08.928387 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:08.928415 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:08.929340 (+   925us) log.cc:844] Serialized 556 byte log entry
0709 13:18:09.123991 (+194651us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:09.124026 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:09.124091 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:09.952620 (+828529us) raft_consensus.cc:1685] finished
0709 13:18:09.952639 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:09.952782 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":194358,"threads_started":1,"wal-append.queue_time_us":358339}
I20260709 13:18:10.042937 13967 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:10.113139 12782 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:10.072479 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:10.113808 12782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1369) took 1176 ms. Trace:
I20260709 13:18:10.114140 12626 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:10.073669 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:10.075855 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:10.077344 13972 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.077607 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:10.078244 14118 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.078890 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:10.078786 14156 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:10.087389 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:10.087675 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:10.081900 12672 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:10.133549 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 51722.6us
I20260709 13:18:10.091919 12632 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:10.093245 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:10.034778 14181 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:10.040396 14145 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:09.982342 14033 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:09.870936 14112 raft_consensus.cc:2955] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:10.022796 14138 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:10.116086 12629 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:10.110100 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 78309.8us
W20260709 13:18:10.117383 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 81283.6us
I20260709 13:18:10.114169 12782 rpcz_store.cc:276] 0709 13:18:08.937793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.877317 (+939524us) service_pool.cc:224] Handling call
0709 13:18:09.886415 (+  9098us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:10.113589 (+227174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:10.113757 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:10.116204 13382 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:10.147782 12673 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:10.088933 13979 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.119820 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:10.088814 14013 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.093451 14126 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:10.122041 12702 raft_consensus.cc:1275] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:10.132588 14192 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:18:10.099643 14204 log.cc:927] Time spent T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:18:10.130908 14203 log.cc:927] Time spent T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:18:10.131771 14052 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:10.138610 12598 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:10.139552 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:10.141986 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:10.142159 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:10.193058 12632 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:10.193536 12634 raft_consensus.cc:2393] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:10.127506 14088 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:10.121407 14189 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:10.113102 13967 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:10.129314 14085 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:10.150435 12673 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:10.153002 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:10.153375 14033 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:10.153410 13383 raft_consensus.cc:2955] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:10.164323 12629 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:10.125156 14186 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:10.132556 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:10.135903 14156 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:10.179790 12626 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:10.177001 12776 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:10.135716 14041 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:10.183907 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:10.184845 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:10.132601 14206 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:10.145171 14145 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:10.196290 12631 raft_consensus.cc:1240] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:18:10.165539 13381 raft_consensus.cc:2393] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 3: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:10.137975 12705 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:10.129902 14118 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:10.128789 13972 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:10.143923 13996 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:10.143136 14177 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:10.143201 14136 consensus_queue.cc:260] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:10.574505 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 425257us
I20260709 13:18:10.254261 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:10.441956 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 303443us
I20260709 13:18:10.575340 12673 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:10.165843 14153 raft_consensus.cc:2764] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 13:18:10.239086 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 81230.7us
I20260709 13:18:10.215624 12632 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:10.172503 14096 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:10.249380 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:10.179891 14126 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:10.189406 14040 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.672003 12628 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 13:18:10.455945 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 349757us
I20260709 13:18:10.575841 12634 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:10.319463 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:10.655779 14156 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:10.657773 14145 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:10.658432 12782 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:10.180886 13979 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:10.576216 13967 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:10.479447 14159 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:10.481441 14182 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:10.353755 14157 consensus_meta.cc:287] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 1.085s	user 0.009s	sys 0.003s
I20260709 13:18:10.687068 14159 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:10.690697 14159 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:10.690603 14157 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:10.691856 13979 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:10.692662 14157 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:10.682293 14040 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:10.693459 14050 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:10.673199 14126 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:10.572784 13382 raft_consensus.cc:1240] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:10.671336 13996 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:18:10.639827 12780 consensus_meta.cc:287] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 1.554s	user 0.012s	sys 0.009s
I20260709 13:18:10.699550 12780 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
W20260709 13:18:10.430722 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:10.697449 14118 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:10.701962 14198 log.cc:927] Time spent T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.731s	user 0.000s	sys 0.003s
I20260709 13:18:10.702502 14205 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.702979 14205 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:10.659775 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:10.705282 13381 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:10.705484 13968 raft_consensus.cc:2764] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:18:10.708628 14205 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:10.256618 13969 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:10.573174 14131 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:10.499675 13923 raft_consensus.cc:3055] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:10.709960 14155 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:10.710183 13923 raft_consensus.cc:740] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:10.710901 14033 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:10.711025 14139 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:10.711375 13923 consensus_queue.cc:260] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:10.712831 13923 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:10.249586 13976 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:10.713940 13976 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:10.715915 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:10.535847 13899 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.716922 13899 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:10.719406 13899 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:10.200351 14088 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:10.575577 14200 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.250646 13983 raft_consensus.cc:3055] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:10.720017 14088 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:10.720216 13983 raft_consensus.cc:740] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:10.720258 14200 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:10.721266 13983 consensus_queue.cc:260] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:10.575397 14181 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:10.722306 13983 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:10.253849 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:10.537762 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:10.451656 14087 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.723541 14087 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:10.181182 14192 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:10.725773 14192 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:10.190784 14013 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:10.730082 14212 log.cc:927] Time spent T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.298s	user 0.001s	sys 0.000s
I20260709 13:18:10.732268 14013 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:10.732430 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:10.733614 14087 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:10.735324 14119 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:10.737320 12776 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36478 (request call id 1614) took 2667 ms (client timeout 1850 ms). Trace:
W20260709 13:18:10.737737 12776 rpcz_store.cc:269] 0709 13:18:08.069836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.122911 (+ 53075us) service_pool.cc:224] Handling call
0709 13:18:10.737152 (+2614241us) spinlock_profiling.cc:243] Waited 2.01 s on lock 0x7b540044cad4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:10.737254 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4633759616}
W20260709 13:18:10.480790 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:10.694659 13972 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:10.674022 14138 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:10.195153 14151 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.491s
I20260709 13:18:10.683121 14136 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:10.697974 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:10.699087 14148 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:10.716202 13976 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:10.472170 14183 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.041s	sys 0.069s
I20260709 13:18:10.733757 14040 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:10.740991 14159 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:10.255460 14202 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:10.664276 14206 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:10.807806 13382 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:10.808831 12783 raft_consensus.cc:2368] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 2. Current term is 3.
I20260709 13:18:10.686564 14156 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:10.763955 14088 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:10.741182 14192 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:10.743858 14110 consensus_meta.cc:287] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.900s	user 0.010s	sys 0.000s
I20260709 13:18:10.741130 14140 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:10.769109 14200 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:10.749471 14166 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:10.754747 14031 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:10.822881 14206 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:10.750312 14134 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:10.836153 14188 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.565s	user 0.007s	sys 0.017s
W20260709 13:18:10.836340 14154 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.002s	sys 0.009s
I20260709 13:18:10.767948 14126 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:10.825707 14202 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:10.855875 14192 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:10.853315 14154 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.004s	sys 0.015s
I20260709 13:18:10.895854 12630 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:18:10.897312 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 321860us
I20260709 13:18:10.640007 12598 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:10.969897 12705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1275) took 1193 ms. Trace:
W20260709 13:18:10.970723 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 330698us
I20260709 13:18:10.972262 12598 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:10.899711 12599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.009s	sys 0.034s
W20260709 13:18:10.973708 12599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.009s	sys 0.035s
W20260709 13:18:10.678119 12700 consensus_meta.cc:287] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.696s	user 0.009s	sys 0.001s
I20260709 13:18:10.976210 12700 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:10.977120 12700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1285) took 1193 ms. Trace:
I20260709 13:18:10.321708 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:10.978407 12703 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:10.671954 12707 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:10.981957 14159 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:10.982614 12707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1283) took 1200 ms. Trace:
I20260709 13:18:10.983134 12707 rpcz_store.cc:276] 0709 13:18:09.781724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.577206 (+795482us) service_pool.cc:224] Handling call
0709 13:18:10.671774 (+ 94568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:10.982063 (+310289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:10.982384 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:10.985003 12707 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:10.682906 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:10.985736 14145 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:10.986482 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 303544us
I20260709 13:18:10.987201 12701 raft_consensus.cc:1240] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:10.987777 12747 leader_election.cc:400] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3.
I20260709 13:18:10.988175 12747 leader_election.cc:403] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:10.760284 12778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1362) took 1828 ms. Trace:
I20260709 13:18:10.988497 14145 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:10.970502 12705 rpcz_store.cc:276] 0709 13:18:09.776486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.778422 (+  1936us) service_pool.cc:224] Handling call
0709 13:18:10.969370 (+1190948us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b540045f6d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:10.969649 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434547456}
I20260709 13:18:10.988963 12778 rpcz_store.cc:276] 0709 13:18:08.932060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.952965 (+ 20905us) service_pool.cc:224] Handling call
0709 13:18:08.953223 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:09.252288 (+299065us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:18:09.735177 (+482889us) log.cc:844] Serialized 14 byte log entry
0709 13:18:09.881740 (+146563us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:09.881792 (+    52us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:09.881844 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:10.701400 (+819556us) spinlock_profiling.cc:243] Waited 2.06 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:18:10.721206 (+ 19806us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b180041e7e0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2932a6 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419
0709 13:18:10.760022 (+ 38816us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b54001ecb54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:10.760071 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:10.760216 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":84,"raft.run_cpu_time_us":1489,"raft.run_wall_time_us":3701,"spinlock_wait_cycles":727340544,"thread_start_us":123547,"threads_started":1,"wal-append.queue_time_us":151011}
W20260709 13:18:10.700165 12780 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36478 (request call id 1624) took 2469 ms (client timeout 2213 ms). Trace:
W20260709 13:18:10.992246 12780 rpcz_store.cc:269] 0709 13:18:08.230377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.422812 (+192435us) service_pool.cc:224] Handling call
0709 13:18:10.700076 (+2277264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:10.822992 12782 raft_consensus.cc:1240] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:10.904157 12777 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:10.905210 12625 raft_consensus.cc:2393] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:10.666836 12632 raft_consensus.cc:1240] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:10.994907 12632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 112) took 1003 ms. Trace:
W20260709 13:18:10.995286 12777 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36478 (request call id 1604) took 3284 ms (client timeout 1730 ms). Trace:
W20260709 13:18:10.995874 12777 rpcz_store.cc:269] 0709 13:18:07.710997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.839476 (+128479us) service_pool.cc:224] Handling call
0709 13:18:08.088267 (+248791us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b5800193e98. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c723780a0 00007f4c72377984 00007f4c6b8f26d4 00007f4c6b8f2662
0709 13:18:10.994855 (+2906588us) spinlock_profiling.cc:243] Waited 2.81 s on lock 0x7b54001fd954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:10.995162 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6475882112}
I20260709 13:18:10.995312 12632 rpcz_store.cc:276] 0709 13:18:09.990999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.666004 (+675005us) service_pool.cc:224] Handling call
0709 13:18:10.666366 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:10.994307 (+327941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:10.994687 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:10.998596 12706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1282) took 1217 ms. Trace:
W20260709 13:18:10.906776 12774 consensus_meta.cc:287] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.911s	user 0.005s	sys 0.005s
I20260709 13:18:10.999619 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:10.907979 13380 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:10.771627 13381 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:11.002465 14192 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.910250 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:11.005358 14200 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:10.912413 14088 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:10.916966 14126 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:10.853586 14188 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.587s	user 0.007s	sys 0.018s
I20260709 13:18:10.859997 14202 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:10.932996 12704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 935) took 1974 ms (client timeout 1823 ms). Trace:
I20260709 13:18:10.957932 14031 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:10.785522 13967 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:10.961709 12746 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:11.000002 12774 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:10.778357 12779 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:11.003404 14192 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:11.004642 14110 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:11.017797 14110 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:11.018335 14110 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:11.019757 13967 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:11.008692 12631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 921) took 1034 ms. Trace:
I20260709 13:18:10.977499 12700 rpcz_store.cc:276] 0709 13:18:09.783970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.811258 (+ 27288us) service_pool.cc:224] Handling call
0709 13:18:10.977024 (+1165766us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:11.021055 12631 rpcz_store.cc:276] 0709 13:18:09.972599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.973635 (+  1036us) service_pool.cc:224] Handling call
0709 13:18:10.088164 (+114529us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:10.576630 (+488466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.004155 (+427525us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54002ecf54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:11.006875 (+  2720us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":247954688}
I20260709 13:18:11.024252 14192 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:11.012317 12704 rpcz_store.cc:269] 0709 13:18:08.958301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.032498 (+ 74197us) service_pool.cc:224] Handling call
0709 13:18:10.932385 (+1899887us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540006de54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:10.932715 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2341885184}
I20260709 13:18:11.026856 14188 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.010073 12673 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:11.010995 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:11.028016 14188 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:11.029210 14145 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:11.029999 14214 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:11.030673 14214 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.031111 14214 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:11.031860 12529 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40528 (request call id 133) took 6691 ms. Trace:
I20260709 13:18:10.933670 14219 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:11.014236 12707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1286) took 1223 ms. Trace:
I20260709 13:18:11.034446 12707 rpcz_store.cc:276] 0709 13:18:09.784220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.984171 (+1199951us) service_pool.cc:224] Handling call
0709 13:18:10.984652 (+   481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:10.985380 (+   728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:10.985548 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:10.992722 12705 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:10.999313 14166 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.037533 14166 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:11.040449 14166 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:11.041750 14219 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:11.044620 14219 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:11.047169 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 133912us
I20260709 13:18:11.047428 14192 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:11.047961 14192 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.048431 14192 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:11.049973 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:11.050818 13380 raft_consensus.cc:2376] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3. Re-sending same reply.
I20260709 13:18:11.055008 12531 catalog_manager.cc:5697] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 1 to 2, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 2 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:11.057390 12531 catalog_manager.cc:5697] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 1 to 2, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 2 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:11.060070 14026 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:18:11.061817 12774 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36478 (request call id 1629) took 1913 ms (client timeout 2046 ms). Trace:
I20260709 13:18:11.067595 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:10.981513 12703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1284) took 1197 ms. Trace:
I20260709 13:18:10.999166 12706 rpcz_store.cc:276] 0709 13:18:09.781451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.552282 (+770831us) service_pool.cc:224] Handling call
0709 13:18:10.553056 (+   774us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:10.553527 (+   471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:10.553589 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:10.666035 (+112446us) log.cc:844] Serialized 26 byte log entry
0709 13:18:10.685588 (+ 19553us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:10.930376 (+244788us) log.cc:844] Serialized 14 byte log entry
0709 13:18:10.966344 (+ 35968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:10.998066 (+ 31722us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:10.998147 (+    81us) raft_consensus.cc:1685] finished
0709 13:18:10.998166 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:10.998341 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":243418,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":114,"raft.queue_time_us":43087,"raft.run_cpu_time_us":505,"raft.run_wall_time_us":99727,"thread_start_us":40384,"threads_started":2,"wal-append.queue_time_us":26194}
I20260709 13:18:11.073091 14026 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:11.088361 14026 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:11.074853 12774 rpcz_store.cc:269] 0709 13:18:09.148592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.325983 (+177391us) service_pool.cc:224] Handling call
0709 13:18:11.061664 (+1735681us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:11.093961 14188 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:11.095746 12597 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.005s	sys 0.001s
W20260709 13:18:11.096159 12597 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.917s	user 0.005s	sys 0.001s
I20260709 13:18:11.081162 12703 rpcz_store.cc:276] 0709 13:18:09.783663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.979993 (+1196330us) service_pool.cc:224] Handling call
0709 13:18:10.980746 (+   753us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:10.981115 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:10.981145 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:10.981212 (+    67us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:10.981252 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:10.981316 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:10.981423 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:11.081609 14166 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.102469 14166 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:11.104120 14159 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:11.108807 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.110250 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:11.116524 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.071275 14192 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:11.032341 12529 rpcz_store.cc:276] 0709 13:18:04.339914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.340084 (+   170us) service_pool.cc:224] Handling call
0709 13:18:11.031575 (+6691491us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:07.553749 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:07.554622 (+   873us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:07.554658 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:18:07.554677 (+    19us) tablet.cc:662] Decoding operations
0709 13:18:07.555264 (+   587us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:07.555379 (+   115us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:07.555424 (+    45us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:18:07.555882 (+   458us) tablet.cc:701] Row locks acquired
0709 13:18:07.555910 (+    28us) write_op.cc:260] PREPARE: finished
0709 13:18:07.556101 (+   191us) write_op.cc:270] Start()
0709 13:18:07.556256 (+   155us) write_op.cc:276] Timestamp: P: 1783603087556045 usec, L: 0
0709 13:18:07.556289 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 13:18:07.557256 (+   967us) log.cc:844] Serialized 913 byte log entry
0709 13:18:08.173101 (+615845us) spinlock_profiling.cc:243] Waited 20.7 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6a2a80ac 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b
0709 13:18:09.030190 (+857089us) op_driver.cc:464] REPLICATION: finished
0709 13:18:09.811807 (+781617us) write_op.cc:301] APPLY: starting
0709 13:18:09.811946 (+   139us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:09.812067 (+   121us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:09.812092 (+    25us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:09.812548 (+   456us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:09.812731 (+   183us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:18:09.812784 (+    53us) write_op.cc:312] APPLY: finished
0709 13:18:09.812986 (+   202us) log.cc:844] Serialized 29 byte log entry
0709 13:18:09.948848 (+135862us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:09.951479 (+  2631us) write_op.cc:454] Released schema lock
0709 13:18:09.951550 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:09.986333 (+ 34783us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:09.989512 (+  3179us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1521635,"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":768157,"apply.run_cpu_time_us":74183,"apply.run_wall_time_us":166373,"num_ops":2,"prepare.queue_time_us":291326,"prepare.run_cpu_time_us":43003,"prepare.run_wall_time_us":875300,"raft.queue_time_us":623693,"raft.run_cpu_time_us":39337,"raft.run_wall_time_us":236202,"replication_time_us":1473713,"spinlock_wait_cycles":47687680,"thread_start_us":1487199,"threads_started":5,"wal-append.queue_time_us":991913,"wal-append.run_cpu_time_us":12684,"wal-append.run_wall_time_us":1071235}]]}
W20260709 13:18:11.113396 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 114572us
I20260709 13:18:11.114363 13967 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:11.114961 12746 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:11.115329 12775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36478 (request call id 1632) took 1134 ms. Trace:
I20260709 13:18:11.139122 12775 rpcz_store.cc:276] 0709 13:18:09.980407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.108161 (+1127754us) service_pool.cc:224] Handling call
0709 13:18:11.115244 (+  7083us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:11.139478 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:11.140720 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.144565 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:11.145329 12709 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:11.145725 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.146361 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:11.147598 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.125001 14126 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:11.148854 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.125612 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.149982 12777 raft_consensus.cc:2393] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:11.150812 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:11.127066 12780 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:11.153720 12703 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 1.
I20260709 13:18:11.154863 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.155602 12780 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:11.156571 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.156351 14110 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:11.157162 12780 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:11.139076 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:11.118618 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 87160.1us
I20260709 13:18:11.135735 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:11.137004 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:11.136857 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.138222 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:11.138587 13967 raft_consensus.cc:697] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:11.147665 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:11.148578 14126 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:11.125032 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:11.156832 14206 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:11.133106 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.134447 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:11.173674 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 58921.4us
I20260709 13:18:11.138288 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:11.171617 12704 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:11.182072 12701 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:11.182973 12630 raft_consensus.cc:1240] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:11.190181 12778 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:11.177327 14088 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:11.196965 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.199940 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:11.200873 12704 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 1.
W20260709 13:18:11.205615 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.206265 12781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1375) took 1021 ms. Trace:
W20260709 13:18:11.221065 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 82947.1us
I20260709 13:18:11.217729 12707 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:11.224701 14096 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:11.225256 14096 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:11.225670 14096 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:11.227118 13967 consensus_queue.cc:237] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:11.230398 14110 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:11.175665 14026 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:11.233055 14026 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:11.208729 12706 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:11.239934 14088 raft_consensus.cc:697] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:11.240091 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:11.241679 14088 consensus_queue.cc:237] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:11.242458 14126 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:11.198707 14192 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:11.209635 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:11.246423 13380 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:11.258634 12747 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:11.260133 13923 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:11.260571 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:11.261691 14110 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:11.262143 14192 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:11.260641 13923 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.262636 14192 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:11.262785 13923 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:11.263110 14192 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:11.263935 14231 log.cc:927] Time spent T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:18:11.269440 12777 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:11.302577 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.214964 12531 catalog_manager.cc:5697] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 1 to 2, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 2 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:11.305403 12531 catalog_manager.cc:5697] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 1 to 2, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 2 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:11.307180 12531 catalog_manager.cc:5697] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 1 to 2, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 2 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:11.309887 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:11.310650 12783 raft_consensus.cc:2376] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3. Re-sending same reply.
I20260709 13:18:11.310287 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.311661 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:11.313956 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.206713 12781 rpcz_store.cc:276] 0709 13:18:10.184656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.682804 (+498148us) service_pool.cc:224] Handling call
0709 13:18:10.683115 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:10.683303 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:18:11.053922 (+370619us) log.cc:844] Serialized 14 byte log entry
0709 13:18:11.187451 (+133529us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:11.191268 (+  3817us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:11.205806 (+ 14538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.205924 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.206091 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":177645,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":7298,"thread_start_us":475372,"threads_started":2}
I20260709 13:18:11.314107 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.314774 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:11.318169 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:11.325641 12778 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:18:11.321978 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.316220 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:11.315737 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.314697 12777 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
W20260709 13:18:11.337512 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.339681 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.340036 12776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1379) took 1014 ms. Trace:
I20260709 13:18:11.340261 14031 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:11.342705 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 166190us
I20260709 13:18:11.343786 12673 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:11.345785 14126 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:11.343802 14031 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:11.346835 14156 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:11.348943 14156 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:11.349217 13923 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:11.349900 14136 consensus_meta.cc:287] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.527s	user 0.006s	sys 0.007s
I20260709 13:18:11.350512 12782 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:11.350626 12633 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:11.350172 12781 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:11.351030 12707 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:11.352162 14206 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:11.223533 12705 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:11.352460 12632 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:11.352208 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:18:11.356783 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.345458 12673 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:11.361475 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:11.362649 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:11.363126 14166 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:11.364904 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:11.295403 14214 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:11.366115 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.366679 12782 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:11.367517 14214 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:11.371507 12778 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
W20260709 13:18:11.372226 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 54213.8us
I20260709 13:18:11.325515 12775 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:11.371409 12782 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:11.374788 12673 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:11.376089 14110 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:11.379065 14110 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.378897 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:11.379508 14110 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:11.379417 12747 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:11.379858 12705 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:11.380966 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.381707 12630 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:11.377300 12597 leader_election.cc:400] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 2. Current term is 3.
I20260709 13:18:11.384479 12597 leader_election.cc:403] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:18:11.387176 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.383608 12781 raft_consensus.cc:2376] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3. Re-sending same reply.
I20260709 13:18:11.390177 14214 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.395627 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.394083 14031 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:11.398371 12747 leader_election.cc:400] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3.
I20260709 13:18:11.398905 12747 leader_election.cc:403] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:11.397363 14031 raft_consensus.cc:697] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:11.401481 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:11.402959 14031 consensus_queue.cc:237] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:11.405645 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1281) took 1548 ms. Trace:
I20260709 13:18:11.406054 12702 rpcz_store.cc:276] 0709 13:18:09.781191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.027085 (+1245894us) service_pool.cc:224] Handling call
0709 13:18:11.027696 (+   611us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:11.028248 (+   552us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:11.028278 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:11.028975 (+   697us) log.cc:844] Serialized 26 byte log entry
0709 13:18:11.173985 (+145010us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:11.297258 (+123273us) log.cc:844] Serialized 14 byte log entry
0709 13:18:11.327312 (+ 30054us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.327430 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:11.327489 (+    59us) raft_consensus.cc:1685] finished
0709 13:18:11.327507 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.329946 (+  2439us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":126554,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":29716,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":4995,"thread_start_us":144001,"threads_started":1,"wal-append.queue_time_us":160763}
I20260709 13:18:11.409840 14166 consensus_queue.cc:1059] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:11.411545 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.412237 12775 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:11.412513 14166 raft_consensus.cc:3055] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:11.412923 14166 raft_consensus.cc:740] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:11.409839 12632 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:11.376442 14156 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:11.396189 14214 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:11.414073 12778 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:11.383212 14126 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:11.419960 12673 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:11.426079 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:11.424753 14156 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:11.433760 14200 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3.
I20260709 13:18:11.433987 12673 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:11.420848 14192 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:11.435684 12673 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:11.436926 14219 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:11.437299 14166 consensus_queue.cc:260] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:11.439244 14166 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:11.439471 12673 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:11.441799 13923 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:11.442418 14112 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3.
I20260709 13:18:11.442721 14156 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:11.433996 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.421078 12775 raft_consensus.cc:1240] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:11.444420 12783 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:11.446095 12781 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 2: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:11.447394 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.424005 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:11.437247 14192 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:11.450513 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.452170 12673 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:11.446067 14126 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:11.437577 14219 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:18:11.454687 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 100201us
I20260709 13:18:11.434131 14207 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 2. Current term is 3.
I20260709 13:18:11.457151 14145 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:11.457477 12777 raft_consensus.cc:2393] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 3: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:11.457753 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.458215 12781 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:11.462297 14214 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:11.471920 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.477854 12673 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:11.483054 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.484009 12780 raft_consensus.cc:2393] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:11.466290 14126 raft_consensus.cc:697] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:11.451606 14096 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:11.491602 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:11.492288 12783 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:11.502933 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 51979.5us
I20260709 13:18:11.503399 14219 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:11.504616 12673 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:11.504957 12597 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:11.506810 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:11.514933 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:11.507308 14228 log.cc:927] Time spent T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.174s	user 0.003s	sys 0.000s
I20260709 13:18:11.496521 14207 consensus_queue.cc:1059] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:18:11.510032 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:11.502092 14126 consensus_queue.cc:237] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:11.515708 12780 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:11.521782 14136 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:11.522011 12777 raft_consensus.cc:1240] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:11.522570 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:11.523126 12781 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:11.524479 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:11.525094 14233 log.cc:927] Time spent T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 13:18:11.537889 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:11.538702 12777 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:11.530081 12774 raft_consensus.cc:1240] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:11.530148 12672 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:11.532485 14136 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.550729 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:11.549676 14145 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:11.553229 12781 raft_consensus.cc:1240] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:11.553391 14145 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.545318 12775 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:11.559808 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.560508 12781 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:11.557991 12597 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:18:11.556753 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 140629us
I20260709 13:18:11.558131 12782 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:11.551795 12598 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.637s (SENT) [suppressed 13 similar messages]
I20260709 13:18:11.564779 12777 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:11.551854 14136 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:11.560736 12774 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:11.567281 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.548635 14156 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:18:11.569588 14096 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:11.569689 13383 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:11.561252 14145 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:11.563712 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:11.575932 14154 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:11.570432 12775 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:11.587873 14166 consensus_queue.cc:1059] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:11.588356 12783 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:11.588464 13349 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:11.589141 14166 raft_consensus.cc:3055] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:11.582098 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.590615 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.582684 14154 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.594985 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.596199 14154 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:11.597007 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:11.601826 12631 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:11.603713 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.595274 13380 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:11.592113 14166 raft_consensus.cc:740] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:11.598405 14219 raft_consensus.cc:697] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:11.604539 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:11.594184 14188 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:11.582856 14110 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:11.606762 12597 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:11.608163 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:11.606508 12632 raft_consensus.cc:2376] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3. Re-sending same reply.
I20260709 13:18:11.607492 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:11.612582 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.616050 12633 raft_consensus.cc:2368] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3.
I20260709 13:18:11.617784 14241 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:11.340545 12776 rpcz_store.cc:276] 0709 13:18:10.326014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.739529 (+413515us) service_pool.cc:224] Handling call
0709 13:18:10.739788 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:10.740155 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:18:11.085530 (+345375us) log.cc:844] Serialized 14 byte log entry
0709 13:18:11.332665 (+247135us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:11.332740 (+    75us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:11.332837 (+    97us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.336213 (+  3376us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.339866 (+  3653us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3249,"prepare.queue_time_us":121075,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":32385,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":110,"thread_start_us":553793,"threads_started":2}
W20260709 13:18:11.618012 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 87949.9us
I20260709 13:18:11.618295 14241 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.618453 12780 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 3: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:11.620190 12774 raft_consensus.cc:1275] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:11.621369 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:11.622071 12775 raft_consensus.cc:2376] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3. Re-sending same reply.
I20260709 13:18:11.622124 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.623889 14214 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.624617 14214 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:11.624864 12780 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
W20260709 13:18:11.626776 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.626863 14214 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:11.627159 14206 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:18:11.627457 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.627571 14206 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.628115 14206 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:11.629778 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:11.639206 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:11.641533 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.646925 12782 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:11.648260 12781 raft_consensus.cc:1240] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:11.650647 14242 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:11.651146 14242 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.651526 14242 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:11.651898 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:11.653713 14224 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:11.654203 14224 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.654670 14224 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:11.618711 14241 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:11.660940 14214 consensus_queue.cc:1059] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:11.662496 14214 raft_consensus.cc:3055] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:11.662927 14214 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:11.662774 14242 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:11.664247 14214 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:11.667232 14219 consensus_queue.cc:237] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:11.645262 14188 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:11.670079 14188 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.670604 14188 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:11.679898 12673 leader_election.cc:400] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3.
I20260709 13:18:11.680462 12673 leader_election.cc:403] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:18:11.684015 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:11.684521 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.684815 12705 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:18:11.684827 14040 raft_consensus.cc:3055] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:11.685937 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.686425 14040 raft_consensus.cc:740] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:11.686864 12629 raft_consensus.cc:2393] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:11.687811 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.687510 14040 consensus_queue.cc:260] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:11.688795 14040 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:11.690359 14156 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:18:11.691083 14232 log.cc:927] Time spent T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.367s	user 0.002s	sys 0.000s
I20260709 13:18:11.695320 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:11.698273 12701 raft_consensus.cc:1240] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:18:11.699251 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:11.699710 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.700053 12783 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 2: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:11.700520 12628 raft_consensus.cc:2393] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:11.699961 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.701071 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:11.701660 12707 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:11.700520 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.612748 14026 consensus_queue.cc:1059] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.064s
I20260709 13:18:11.702015 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:11.702838 14026 raft_consensus.cc:3055] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:11.703063 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:11.703233 14026 raft_consensus.cc:740] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:11.704062 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.704218 14026 consensus_queue.cc:260] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:11.705334 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.694602 12702 raft_consensus.cc:2955] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:11.706007 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.707584 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:11.716329 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:11.717840 12629 raft_consensus.cc:2393] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:11.708417 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:11.724097 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.724705 12630 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:11.724799 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:11.705471 14026 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:11.727440 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.704167 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:18:11.727617 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 51045.3us
I20260709 13:18:11.728370 12627 raft_consensus.cc:2368] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3.
I20260709 13:18:11.731879 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.734195 12597 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:11.713719 12779 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:11.703111 12703 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:11.737414 14242 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:11.737658 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:11.738279 14183 consensus_queue.cc:1059] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:18:11.739038 12774 raft_consensus.cc:2393] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 3: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:11.739589 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:11.746721 12633 raft_consensus.cc:2368] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3.
I20260709 13:18:11.739725 14154 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:11.740217 14207 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:11.740815 14126 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:11.749152 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:11.753021 12705 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:18:11.737668 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.741905 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:11.754165 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:11.754436 12632 raft_consensus.cc:2376] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3. Re-sending same reply.
I20260709 13:18:11.754935 14110 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:11.755295 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:11.757450 14183 raft_consensus.cc:3055] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:11.757992 14183 raft_consensus.cc:740] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:11.759074 14183 consensus_queue.cc:260] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:11.752786 14126 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:11.766396 14126 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:11.742499 14249 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:11.770006 14249 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.770397 14249 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:11.761359 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:11.774232 12630 raft_consensus.cc:2393] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:11.727476 14214 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:11.774509 14188 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:11.763051 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:11.748041 14224 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:11.778333 14145 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:11.781299 14145 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:11.746853 14166 consensus_queue.cc:260] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:11.784006 12778 raft_consensus.cc:1240] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:18:11.783763 14136 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:11.791524 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:11.748229 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 129335us
I20260709 13:18:11.762624 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.798249 12673 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:11.801604 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 155204us
I20260709 13:18:11.808101 12629 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:18:11.818905 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.819098 12634 raft_consensus.cc:2393] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:18:11.819536 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.776590 14188 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:11.820768 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 86647.2us
I20260709 13:18:11.821738 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:11.821420 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:11.824410 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:11.827787 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:11.830513 12628 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:18:11.837636 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 93393.8us
I20260709 13:18:11.837860 13349 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:11.839579 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:11.842003 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:11.844707 14183 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:11.845718 12630 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:11.851598 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:11.853377 12626 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:11.855775 12707 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
W20260709 13:18:11.867236 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.867504 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:11.868260 12706 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:11.859514 12673 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:11.834851 12747 connection.cc:256] RPC call timeout handler was delayed by -3.2641322669999226s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20790000000000003s for a response
W20260709 13:18:11.885768 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:11.880312 12747 connection.cc:256] RPC call timeout handler was delayed by -3.2542322669999066s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.209s for a response
I20260709 13:18:11.842206 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:11.858316 14166 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:11.832733 14242 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:18:11.854199 13380 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:11.902175 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:11.907661 12629 raft_consensus.cc:2393] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 4: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:18:11.899879 12747 connection.cc:256] RPC call timeout handler was delayed by -3.2425547780001125s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1733s for a response
I20260709 13:18:11.886376 14219 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:11.917138 12672 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
W20260709 13:18:11.919847 12747 connection.cc:256] RPC call timeout handler was delayed by -3.1938213030000497s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2848s for a response
W20260709 13:18:11.920300 12747 connection.cc:256] RPC call timeout handler was delayed by -2.9734547780001321s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20320000000000002s for a response
W20260709 13:18:11.920610 12747 connection.cc:256] RPC call timeout handler was delayed by -2.7196273960003055s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1807s for a response
I20260709 13:18:11.921813 12630 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:11.905295 14145 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:18:11.924762 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 85584.4us
I20260709 13:18:11.906349 12705 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:11.929236 12673 leader_election.cc:400] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3.
I20260709 13:18:11.929730 12673 leader_election.cc:403] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:18:11.938508 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:11.939479 14156 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:11.939654 12672 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:11.939718 14245 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:11.939965 14156 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:11.940490 14245 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:11.940652 14156 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:11.940686 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:11.940943 14245 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:11.940196 14246 log.cc:927] Time spent T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:18:11.912793 14219 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.918305 14207 consensus_queue.cc:1059] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:18:11.919543 14241 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:11.918844 14243 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:11.941526 12672 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:11.941599 12703 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:11.953639 14156 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:11.941735 14126 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:11.997895 14126 raft_consensus.cc:697] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:18:11.999126 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.998838 14126 consensus_queue.cc:237] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:11.945838 14219 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:11.945961 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:12.005915 12707 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:12.009003 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:11.946033 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:12.010015 12709 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:11.946372 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:12.010493 14241 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:12.010864 12700 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:11.956310 14224 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:11.957171 14136 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:11.964955 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:11.955811 14166 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:11.965262 14242 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:11.964102 14250 raft_consensus.cc:3055] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:12.018851 14250 raft_consensus.cc:740] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:12.019975 14250 consensus_queue.cc:260] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:11.948027 14154 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:11.953891 12747 connection.cc:256] RPC call timeout handler was delayed by -2.2579547780001121s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2827s for a response
W20260709 13:18:12.021456 12747 connection.cc:256] RPC call timeout handler was delayed by -2.2251164850001715s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1674s for a response
I20260709 13:18:11.949124 14245 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:12.021808 12747 connection.cc:256] RPC call timeout handler was delayed by -2.1693164850000812s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
W20260709 13:18:12.022135 12747 connection.cc:256] RPC call timeout handler was delayed by -2.091016485000182s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18230000000000002s for a response
I20260709 13:18:11.947805 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:11.948248 12625 raft_consensus.cc:2410] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:11.997300 14156 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:12.018002 12702 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:12.022435 12747 connection.cc:256] RPC call timeout handler was delayed by -2.0874164850001762s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1827s for a response
W20260709 13:18:12.023233 12747 connection.cc:256] RPC call timeout handler was delayed by -2.07121648500015s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1845s for a response
I20260709 13:18:11.965063 12673 leader_election.cc:400] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3.
W20260709 13:18:12.023545 12747 connection.cc:256] RPC call timeout handler was delayed by -1.978516485s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1948s for a response
I20260709 13:18:11.954077 14200 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.023975 12673 leader_election.cc:403] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260709 13:18:12.024029 12747 connection.cc:256] RPC call timeout handler was delayed by -1.9173164850001285s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2016s for a response
W20260709 13:18:12.024665 12747 connection.cc:256] RPC call timeout handler was delayed by -1.908316485000114s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2026s for a response
W20260709 13:18:12.025031 12747 connection.cc:256] RPC call timeout handler was delayed by -1.8156164850001915s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2129s for a response
W20260709 13:18:12.025378 12747 connection.cc:256] RPC call timeout handler was delayed by -1.8093164850001813s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2136s for a response
W20260709 13:18:12.025750 12747 connection.cc:256] RPC call timeout handler was delayed by -1.7967164850001609s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.215s for a response
W20260709 13:18:12.026084 12747 connection.cc:256] RPC call timeout handler was delayed by -1.713916485000027s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2242s for a response
W20260709 13:18:12.026379 12747 connection.cc:256] RPC call timeout handler was delayed by -1.3269164850000834s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26720000000000005s for a response
W20260709 13:18:12.026737 12747 connection.cc:256] RPC call timeout handler was delayed by -1.2342164850001609s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2775s for a response
I20260709 13:18:12.033983 14145 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:12.034690 14245 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:12.018006 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 76564.6us
I20260709 13:18:12.047542 12707 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
W20260709 13:18:12.052188 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 87255.4us
I20260709 13:18:12.053218 12673 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:12.055124 12706 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
W20260709 13:18:12.062536 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 234816us
I20260709 13:18:12.062937 12747 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:11.989507 14243 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3.
I20260709 13:18:12.018021 14166 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:12.023058 12705 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:12.074525 12709 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:12.082538 12702 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:12.064652 14156 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:12.088802 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 268478us
I20260709 13:18:12.071007 14245 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:12.090960 12746 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:12.074124 14166 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:12.070871 14154 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.051044 14200 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.102131 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:12.084723 12597 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:12.073801 14224 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.094079 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:12.090937 14110 raft_consensus.cc:2764] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 13:18:12.070161 14249 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:12.104202 12625 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:12.102771 12634 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:12.104694 12597 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:18:12.110399 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 50874.2us
I20260709 13:18:12.074292 14206 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:12.111562 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.111642 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:12.114646 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:12.117020 13382 raft_consensus.cc:2368] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3.
W20260709 13:18:12.117668 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.121723 14206 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:12.112092 14224 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.127756 14224 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:12.129689 14250 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:12.130357 14040 consensus_queue.cc:1059] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:12.100292 14031 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:12.129768 14214 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:12.131302 12597 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:12.106338 14026 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:12.104240 14245 raft_consensus.cc:697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:12.144023 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:12.144954 12781 raft_consensus.cc:2368] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3.
I20260709 13:18:12.090785 14255 consensus_queue.cc:1059] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:18:12.145929 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:12.146943 12625 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:12.149256 12776 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:12.111169 14259 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:12.125360 12703 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:12.129313 14219 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:12.147400 14031 raft_consensus.cc:697] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:12.116676 14154 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:12.161278 14026 raft_consensus.cc:697] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:18:12.185712 12672 connection.cc:256] RPC call timeout handler was delayed by -1.0666910969998753s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2046s for a response
I20260709 13:18:12.164471 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:12.172120 12704 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:12.164805 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:12.180013 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 62197us
I20260709 13:18:12.195595 12776 raft_consensus.cc:2393] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:12.154489 14214 raft_consensus.cc:697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:12.193672 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:12.156215 14207 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:12.209180 14026 consensus_queue.cc:237] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:12.209950 14261 raft_consensus.cc:3055] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:12.210371 14261 raft_consensus.cc:740] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:12.211273 14261 consensus_queue.cc:260] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:12.154109 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:12.212378 14261 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:12.203672 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:12.204025 14031 consensus_queue.cc:237] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.204735 14249 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:12.202091 14096 consensus_queue.cc:1059] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:18:12.206748 14219 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:12.206470 14245 consensus_queue.cc:237] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.166566 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:12.161749 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:12.250763 12774 raft_consensus.cc:2393] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 2: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:12.207679 12634 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:12.169674 14251 consensus_queue.cc:1059] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 13:18:12.241173 14154 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:12.231403 12781 raft_consensus.cc:2393] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 2: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:12.226931 12632 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:12.233736 12747 leader_election.cc:400] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3.
I20260709 13:18:12.234130 14248 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:12.157737 14259 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.237138 12700 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:12.268380 14156 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3.
I20260709 13:18:12.201325 14255 consensus_queue.cc:1059] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 13:18:12.269933 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:12.270582 14259 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:12.271548 14259 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:12.272046 13349 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:12.239255 14166 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.272946 13349 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:12.274132 14166 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:12.275074 14166 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:12.275542 14166 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:12.243256 14096 raft_consensus.cc:3055] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:12.277077 14096 raft_consensus.cc:740] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:12.202560 14145 raft_consensus.cc:697] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:12.214274 12783 raft_consensus.cc:2393] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
W20260709 13:18:12.305827 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 148737us
I20260709 13:18:12.318368 12777 raft_consensus.cc:1240] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:12.193648 14183 consensus_queue.cc:1059] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.098s
I20260709 13:18:12.252482 14251 raft_consensus.cc:3055] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:12.233433 14200 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:12.270048 14154 raft_consensus.cc:3055] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:12.236922 14214 consensus_queue.cc:237] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:12.216671 14207 raft_consensus.cc:697] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:18:12.279816 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 125972us
W20260709 13:18:12.331995 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.265838 14248 raft_consensus.cc:697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:12.335717 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:12.265936 14242 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3.
I20260709 13:18:12.275897 14166 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:12.281097 14241 raft_consensus.cc:2764] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 13:18:12.339224 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 286105us
I20260709 13:18:12.333914 13349 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
W20260709 13:18:12.343389 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 222476us
I20260709 13:18:12.345587 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:12.254431 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 53489.2us
W20260709 13:18:12.355412 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.417886 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:12.422960 12776 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:12.331071 14261 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.422520 12781 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:12.361128 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:12.361212 12777 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:12.364413 14248 consensus_queue.cc:237] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.366794 14145 consensus_queue.cc:237] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.306429 13380 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
W20260709 13:18:12.379809 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 114276us
W20260709 13:18:12.396986 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.345949 14154 raft_consensus.cc:740] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:12.334110 14251 raft_consensus.cc:740] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:12.402825 14268 consensus_queue.cc:1059] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:12.402920 14207 consensus_queue.cc:237] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.278160 14096 consensus_queue.cc:260] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:12.412672 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.413126 14255 consensus_queue.cc:1059] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:12.416397 14267 consensus_queue.cc:1059] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:18:12.417016 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.417600 12673 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:12.379603 14136 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:18:12.424252 12780 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:12.428125 12672 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:12.438174 14261 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:12.455947 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:12.456610 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:12.445394 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:12.453812 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 125797us
I20260709 13:18:12.458169 13348 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:12.459614 13348 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:12.462046 14154 consensus_queue.cc:260] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:12.463693 14154 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:12.464406 14126 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:12.464951 14126 raft_consensus.cc:697] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:12.466488 14259 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:12.466894 14259 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:12.467237 14259 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:12.468382 14251 consensus_queue.cc:260] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:12.469795 14251 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:12.469918 14200 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:12.471931 12777 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:12.473727 12781 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
W20260709 13:18:12.473953 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 45952.2us
I20260709 13:18:12.475462 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:12.476202 12634 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:12.476774 12776 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:12.480176 12672 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:12.484935 14206 raft_consensus.cc:3055] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:12.485442 14206 raft_consensus.cc:740] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:12.452104 14096 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:12.490491 12597 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:12.491345 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
W20260709 13:18:12.494962 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 47358.3us
I20260709 13:18:12.496172 14156 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:12.496385 14040 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:12.486196 14258 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:12.493832 12597 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:12.475455 14249 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.505908 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:12.500380 14243 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:12.484987 14255 raft_consensus.cc:3055] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:12.490495 14031 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.475533 14273 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:12.518344 12672 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:12.526381 12634 raft_consensus.cc:1275] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:12.520954 13382 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:12.520398 14255 raft_consensus.cc:740] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:12.525640 13349 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:12.525516 14249 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.525202 12597 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:18:12.526264 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.512878 14258 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:12.528048 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:12.513463 14040 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:12.510783 14267 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:12.514070 12747 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:12.493377 14188 consensus_queue.cc:1048] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:18:12.528957 14224 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:12.552958 13384 raft_consensus.cc:2393] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 4: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:12.543525 12597 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:12.554164 13380 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:12.547137 14258 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:12.565733 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:12.567901 14270 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.568446 14270 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:12.570649 14270 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:12.519058 14126 consensus_queue.cc:237] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:12.576555 14251 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:12.525317 14282 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:18:12.580420 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 81905.8us
I20260709 13:18:12.563273 14110 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:12.564992 14255 consensus_queue.cc:260] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:12.545205 14261 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:12.573319 12780 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:12.551074 14040 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:12.588534 14200 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:12.559396 14224 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:12.589651 14224 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:12.547755 14031 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.591249 12701 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:18:12.592311 12701 raft_consensus.cc:606] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:12.592702 12701 raft_consensus.cc:614] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:12.578676 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:12.594308 14241 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:12.547187 14136 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:12.598548 14136 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:12.558635 14156 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:12.565455 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:12.561458 14250 raft_consensus.cc:3055] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:12.613679 14250 raft_consensus.cc:740] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:12.613813 13385 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
W20260709 13:18:12.613838 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 52343.6us
I20260709 13:18:12.614988 14250 consensus_queue.cc:260] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:12.565003 14260 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:12.616995 12672 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:12.577284 14251 raft_consensus.cc:697] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:12.620882 14282 raft_consensus.cc:697] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:12.620533 14251 consensus_queue.cc:237] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:12.621338 14275 log.cc:927] Time spent T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:18:12.622560 12747 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:12.623414 12597 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:12.624197 12747 leader_election.cc:400] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3.
I20260709 13:18:12.625198 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:12.625986 12631 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:12.626703 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:12.627712 12625 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 4: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:12.631402 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:12.632478 12780 raft_consensus.cc:2410] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:18:12.561873 14245 consensus_queue.cc:1048] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 13:18:12.608867 14136 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:12.617045 14250 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:12.580703 14242 consensus_queue.cc:1059] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:18:12.653723 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:12.559355 14206 consensus_queue.cc:260] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:12.619586 14260 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:12.554451 14267 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:12.660727 14207 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:12.696563 14206 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:12.697208 14207 raft_consensus.cc:697] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:18:12.698467 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.698271 14207 consensus_queue.cc:237] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:12.699848 14267 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:12.668840 14031 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:12.669162 14126 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:12.701308 14126 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:12.701771 14126 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:12.701963 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:12.669835 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 45820.8us
I20260709 13:18:12.671060 14242 raft_consensus.cc:3055] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:12.672950 12625 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:18:12.673219 12630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:18:12.672972 12626 raft_consensus.cc:1240] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:12.690613 14156 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:12.695149 14249 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:12.608464 14259 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.696398 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:12.696339 14260 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:12.730011 12780 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:12.646008 14110 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:12.740496 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:12.702699 12628 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
W20260709 13:18:12.777642 13348 leader_election.cc:336] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.666s (SENT)
I20260709 13:18:12.776995 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:18:12.781141 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.693156 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:12.785444 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.646921 14183 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.131s
I20260709 13:18:12.663338 14285 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:12.693981 12779 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:12.706327 14242 raft_consensus.cc:740] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:12.690742 14040 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.790110 12747 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:12.786139 14154 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:12.810357 12631 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:12.814025 13381 raft_consensus.cc:1240] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:12.816376 14040 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:12.798638 14285 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:12.742162 14156 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.776826 14282 consensus_queue.cc:237] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:12.826220 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.828086 12630 raft_consensus.cc:606] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:12.845152 12630 raft_consensus.cc:614] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:12.832659 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:12.852617 12780 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:12.792979 14289 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:12.834129 12747 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:12.818218 14031 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:12.831840 14156 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:12.817323 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:12.817752 12775 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:12.828985 14285 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:12.859773 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 243519us
I20260709 13:18:12.831894 14154 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:12.867472 14154 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:12.827033 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:12.841146 14264 raft_consensus.cc:3055] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:12.866799 12780 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:12.833328 14272 raft_consensus.cc:3055] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:12.838384 14242 consensus_queue.cc:260] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.894075 14126 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:12.877971 12747 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:12.864713 14258 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.897454 12747 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:12.898128 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:12.863934 14289 raft_consensus.cc:697] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:12.898824 13376 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:12.899224 14258 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:12.864228 14206 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.899484 14289 consensus_queue.cc:237] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.900372 13376 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:12.900089 14206 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:12.829332 14166 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.902998 14206 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:12.903059 14166 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:12.828080 14224 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.903235 14154 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.906057 13385 raft_consensus.cc:2393] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:12.905722 14154 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:12.906872 14200 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:12.907219 14110 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:12.865854 14096 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:12.908595 14096 raft_consensus.cc:697] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:12.910058 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:12.910760 13380 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:12.911125 12747 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:12.912793 12747 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:12.914860 14126 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:12.915983 14200 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:12.916134 14278 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:12.882490 14031 consensus_queue.cc:1059] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:12.917207 14031 raft_consensus.cc:3055] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:12.885126 14259 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:12.885741 13382 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:18:12.886628 13379 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:18:12.867175 13378 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:12.917589 12747 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:12.919567 14096 consensus_queue.cc:237] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.920557 12747 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:12.880023 14285 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.922070 14278 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:12.922984 14278 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:12.917608 14031 raft_consensus.cc:740] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:12.924413 14285 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:12.891892 12672 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:12.891950 14272 raft_consensus.cc:740] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:12.926041 12672 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:12.925854 14272 consensus_queue.cc:260] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.893932 12597 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:12.927322 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:12.927909 12597 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:12.928215 13384 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:18:12.928789 14280 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:12.929172 14280 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:12.929512 14280 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:12.891938 14264 raft_consensus.cc:740] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:18:12.894810 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.894573 14292 consensus_queue.cc:1059] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:18:12.896095 14242 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:12.930850 14292 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:12.932673 14292 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:12.924666 14031 consensus_queue.cc:260] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:12.933027 14292 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:12.933452 14292 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:12.932691 12746 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:18:12.881055 14279 log.cc:927] Time spent T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.315s	user 0.002s	sys 0.000s
I20260709 13:18:12.905735 14224 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:12.891265 12673 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:12.889165 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:12.931774 12634 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:12.939692 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:12.940227 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:18:12.942229 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 51036.9us
I20260709 13:18:12.930181 14126 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:12.944597 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:12.944758 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:12.945241 12776 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:12.946252 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:12.946465 14156 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:12.946822 13384 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:12.946898 14156 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:12.947460 14156 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:12.947630 14136 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:12.948829 12629 raft_consensus.cc:1240] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:18:12.929417 12672 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:12.954465 12775 raft_consensus.cc:1275] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:12.927315 14272 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:12.939738 14214 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:12.958791 12597 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:12.939778 14200 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:12.895816 14260 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:12.960295 13348 leader_election.cc:336] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 2.182s (SENT)
W20260709 13:18:12.961141 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.960103 14285 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:12.958952 14219 consensus_queue.cc:1059] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:18:12.959035 14206 consensus_queue.cc:1059] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:18:12.964483 14206 raft_consensus.cc:3055] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:12.965081 14206 raft_consensus.cc:740] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:12.966169 14206 consensus_queue.cc:260] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.941704 14207 consensus_queue.cc:1059] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:18:12.967603 14206 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:12.967893 14207 raft_consensus.cc:3055] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:12.968077 14126 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:12.968359 14207 raft_consensus.cc:740] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:12.969630 14136 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:12.970320 14214 raft_consensus.cc:697] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:12.970578 14183 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:12.971050 14183 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:12.971457 14183 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:12.971884 14110 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:12.972481 14110 raft_consensus.cc:697] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:18:12.976272 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.967618 14255 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:12.979197 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.981688 13349 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:12.983083 14273 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:12.981662 13381 raft_consensus.cc:2368] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4.
I20260709 13:18:12.983634 14273 raft_consensus.cc:697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:12.983795 12628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:18:12.984407 12628 raft_consensus.cc:606] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Received request to transfer leadership to TS 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:12.984758 14273 consensus_queue.cc:237] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.985685 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:12.988436 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:12.936945 14145 consensus_queue.cc:1059] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:18:12.990639 14145 raft_consensus.cc:3055] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:12.984793 12628 raft_consensus.cc:614] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:12.962467 14200 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:12.982750 14264 consensus_queue.cc:260] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.990974 14145 raft_consensus.cc:740] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:13.000093 14145 consensus_queue.cc:260] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:13.001583 14145 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:13.002135 14264 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:12.995867 14200 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:12.969340 14207 consensus_queue.cc:260] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.950939 14278 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:18:13.001133 12528 catalog_manager.cc:5697] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 1 to 2, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 2 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:12.971467 14214 consensus_queue.cc:237] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:12.959311 12672 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:13.020910 13384 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:13.033152 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:13.037459 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.038004 14268 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.038677 14268 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:13.039989 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 99849.5us
W20260709 13:18:13.041819 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 82529.2us
I20260709 13:18:13.044183 13383 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:12.960965 14243 consensus_queue.cc:1059] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.073s
I20260709 13:18:13.060638 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:13.041344 14292 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:13.042126 14040 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:13.047832 14126 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:13.086258 14243 raft_consensus.cc:3055] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:13.087791 14243 raft_consensus.cc:740] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:13.088253 14040 raft_consensus.cc:697] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:13.088879 14243 consensus_queue.cc:260] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:13.089525 14040 consensus_queue.cc:237] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:13.090566 14243 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:13.090974 14260 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:12.988437 14110 consensus_queue.cc:237] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.079030 14299 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:13.130497 14260 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.131024 14268 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:13.131160 14260 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:13.093158 14126 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:13.132512 14126 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:13.098676 13380 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:13.145195 14200 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:13.147686 14200 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:13.069554 14292 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:13.163089 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 89287.6us
I20260709 13:18:13.153942 14260 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:18:13.161944 14299 raft_consensus.cc:697] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:13.157462 14280 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.173750 14183 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:13.183754 14156 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.175482 14280 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:13.190029 14156 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:13.201776 14183 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:13.193203 14126 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.205922 14126 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:13.209772 12528 catalog_manager.cc:5697] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 1 to 2, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 2 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:13.213482 12528 catalog_manager.cc:5697] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 2 to 3, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 3 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:13.211321 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:13.217186 13380 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:13.226481 12528 catalog_manager.cc:5697] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 1 to 2, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 2 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:13.208225 14299 consensus_queue.cc:237] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.241756 13380 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:13.254957 12746 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:13.256299 14110 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:13.256981 14110 raft_consensus.cc:697] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:13.259294 14110 consensus_queue.cc:237] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:13.302270 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:13.315594 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.318218 13348 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:13.318655 13380 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:13.319921 14260 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:13.320042 14096 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:13.324889 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:13.330353 12747 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:13.331200 14096 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:13.332588 12747 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:13.333417 14110 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:13.329957 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:13.333850 14110 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:13.334031 12774 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:13.334244 14110 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:13.335166 14183 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.335647 14183 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.337915 14183 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:13.344781 14110 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.352346 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:13.362731 14299 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:13.365973 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:13.367871 14126 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:13.368202 14110 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:13.368503 14126 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.369498 14096 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:13.369951 14096 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:13.370340 14096 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:13.372159 12702 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:18:13.378521 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:13.382097 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.384435 12774 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:13.390158 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:13.390379 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:13.391616 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:13.392405 12774 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
W20260709 13:18:13.392477 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.393296 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:13.394516 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:13.395107 12774 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:13.396163 12700 raft_consensus.cc:1275] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:13.396368 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:13.399186 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:13.403033 12774 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:13.404235 12632 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:13.411312 12531 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54792 (request call id 224) took 6850 ms. Trace:
I20260709 13:18:13.411777 12531 rpcz_store.cc:276] 0709 13:18:06.560481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.560682 (+   201us) service_pool.cc:224] Handling call
0709 13:18:13.411225 (+6850543us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:11.597178 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:11.634070 (+ 36892us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:11.634134 (+    64us) write_op.cc:435] Acquired schema lock
0709 13:18:11.634155 (+    21us) tablet.cc:662] Decoding operations
0709 13:18:11.733301 (+ 99146us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:11.740791 (+  7490us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:11.740827 (+    36us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:18:11.741533 (+   706us) tablet.cc:701] Row locks acquired
0709 13:18:11.741683 (+   150us) write_op.cc:260] PREPARE: finished
0709 13:18:11.741860 (+   177us) write_op.cc:270] Start()
0709 13:18:11.742004 (+   144us) write_op.cc:276] Timestamp: P: 1783603091741800 usec, L: 0
0709 13:18:11.742040 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 13:18:11.770076 (+ 28036us) log.cc:844] Serialized 2070 byte log entry
0709 13:18:11.990155 (+220079us) op_driver.cc:464] REPLICATION: finished
0709 13:18:12.277960 (+287805us) write_op.cc:301] APPLY: starting
0709 13:18:12.278059 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:12.278245 (+   186us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:12.351879 (+ 73634us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:12.500984 (+149105us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:12.501292 (+   308us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:18:12.501356 (+    64us) write_op.cc:312] APPLY: finished
0709 13:18:12.501722 (+   366us) log.cc:844] Serialized 47 byte log entry
0709 13:18:12.501925 (+   203us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:12.502032 (+   107us) write_op.cc:454] Released schema lock
0709 13:18:12.502092 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:12.562200 (+ 60108us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:12.563256 (+  1056us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":46791,"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":226905,"apply.run_cpu_time_us":4589,"apply.run_wall_time_us":264643,"num_ops":5,"prepare.queue_time_us":199267,"prepare.run_cpu_time_us":17049,"prepare.run_wall_time_us":255121,"raft.queue_time_us":139080,"raft.run_cpu_time_us":10037,"raft.run_wall_time_us":293117,"replication_time_us":247957,"spinlock_wait_cycles":2944,"thread_start_us":471250,"threads_started":4,"wal-append.queue_time_us":43037}]]}
W20260709 13:18:13.421082 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 359723us
I20260709 13:18:13.422128 12672 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:13.423529 12672 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:13.425539 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:13.426539 12672 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:13.427742 12672 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:13.427907 14285 raft_consensus.cc:3055] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:13.428303 14285 raft_consensus.cc:740] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:13.429296 14285 consensus_queue.cc:260] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.430699 14285 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:13.438056 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:13.438791 12782 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:13.438897 12672 leader_election.cc:400] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3.
I20260709 13:18:13.440834 12672 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:13.443523 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.448753 14285 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:13.449975 14285 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:13.450572 14285 raft_consensus.cc:697] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:13.451618 14285 consensus_queue.cc:237] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.442169 12672 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:13.453485 12672 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:13.454165 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:13.454820 12777 raft_consensus.cc:2376] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3. Re-sending same reply.
I20260709 13:18:13.455972 12701 raft_consensus.cc:1240] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:13.461200 12634 raft_consensus.cc:2410] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:18:13.456682 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:13.461846 12702 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:13.458109 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:13.462482 12708 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:13.459765 12705 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:13.460362 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:13.464663 12700 raft_consensus.cc:2393] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 3: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
W20260709 13:18:13.457276 13348 leader_election.cc:336] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.679s (SENT)
I20260709 13:18:13.465111 13348 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:13.466125 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 86856.5us
I20260709 13:18:13.466878 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
W20260709 13:18:13.467511 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 48383us
I20260709 13:18:13.468578 14304 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:13.469512 14317 raft_consensus.cc:2764] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:18:13.469875 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:13.470474 12628 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:13.471588 12747 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:13.472203 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:13.472493 14304 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:13.473224 14304 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:13.473855 14304 raft_consensus.cc:697] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:13.474799 14304 consensus_queue.cc:237] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.473114 12703 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:13.478296 12704 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:13.480154 14183 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.480715 14183 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.473309 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:13.482345 14110 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:13.482745 14183 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:13.482844 14110 raft_consensus.cc:697] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:13.484146 14110 consensus_queue.cc:237] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.487680 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:13.488384 12707 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:13.488862 12747 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:13.489710 14317 raft_consensus.cc:2764] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:18:13.490142 12747 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:13.490424 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:13.491242 12707 raft_consensus.cc:2410] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:13.492440 14317 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:13.495047 12707 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:13.497061 14304 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:13.510845 14031 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:13.512418 14304 consensus_queue.cc:1048] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:13.499599 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:13.513895 12707 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:13.499826 14166 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:13.502296 12703 raft_consensus.cc:1240] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:13.506105 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:13.517084 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.506982 12700 raft_consensus.cc:1240] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:13.510844 14285 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:13.523250 14285 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:13.523727 14285 raft_consensus.cc:697] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:13.524605 14285 consensus_queue.cc:237] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.525401 12779 raft_consensus.cc:1240] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:13.526796 14242 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:13.527554 14242 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:13.530042 14243 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:18:13.535434 12598 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.015s	sys 0.010s
W20260709 13:18:13.535815 12598 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.015s	sys 0.010s
I20260709 13:18:13.535842 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:13.510006 14096 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:13.499176 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.536319 14183 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.535899 12747 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:13.536686 12700 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:13.535987 14317 raft_consensus.cc:697] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:13.536819 14183 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.537703 14317 consensus_queue.cc:237] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.539021 14183 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:13.539403 14096 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:13.539950 14096 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.540378 14096 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.542431 14096 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:13.542502 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:13.543121 14096 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:13.544648 12700 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:13.545634 12703 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:18:13.546046 12703 pending_rounds.cc:85] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 4
I20260709 13:18:13.546307 12703 pending_rounds.cc:107] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:18:13.546602 12703 raft_consensus.cc:2889] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:13.547868 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:13.548045 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:13.548717 12707 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 4: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:13.549196 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:13.549532 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:13.550154 12707 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:13.554898 12700 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:13.555987 12705 raft_consensus.cc:1240] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:13.556308 12704 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:13.556996 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:13.557143 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:13.550150 12703 raft_consensus.cc:2410] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:18:13.557626 12705 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:13.557366 12707 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:13.561661 12708 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:13.562386 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:13.562930 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:13.557964 12704 raft_consensus.cc:2410] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:18:13.566633 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:13.567888 12706 raft_consensus.cc:2410] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:18:13.569260 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:13.569355 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:13.570674 12706 raft_consensus.cc:2410] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:13.575116 12704 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:13.575222 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:13.575654 13348 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:13.576723 13348 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:13.577857 14166 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:18:13.578027 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.577970 14288 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.578434 14166 raft_consensus.cc:697] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:13.578764 14288 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.579936 14166 consensus_queue.cc:237] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:13.581343 14288 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:13.582118 12631 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:13.582954 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:18:13.583827 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.583739 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:18:13.584408 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.587806 14206 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.588303 14206 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.588984 14154 consensus_queue.cc:1048] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:13.584414 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:13.580449 14289 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:13.590859 12626 raft_consensus.cc:2393] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:13.589983 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:13.592819 12632 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:13.594408 12776 raft_consensus.cc:1240] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:13.595201 14145 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:13.596575 14299 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:13.542475 14278 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:13.598001 14278 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:13.598379 14278 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:13.598433 14299 consensus_queue.cc:1059] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:13.599093 12673 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:13.599632 14110 raft_consensus.cc:3055] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:13.600029 14110 raft_consensus.cc:740] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:13.600874 14110 consensus_queue.cc:260] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.601799 14145 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:13.602190 14145 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:13.602547 14145 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:13.605118 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:13.605759 12777 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:18:13.605868 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:13.607162 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:13.608513 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:13.609874 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:13.610468 12782 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:13.610209 14026 consensus_queue.cc:1059] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:13.611272 12628 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:13.611804 14026 raft_consensus.cc:3055] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:13.612534 14026 raft_consensus.cc:740] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:13.612637 12673 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:13.613605 14288 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:13.614084 14288 raft_consensus.cc:697] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 2 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:18:13.615397 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.613792 14026 consensus_queue.cc:260] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.616540 14026 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:13.619429 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:13.508100 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:13.536085 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 543124us
I20260709 13:18:13.625713 12781 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:13.627641 12781 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:18:13.628708 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 52071.9us
I20260709 13:18:13.629499 13348 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:13.630738 13348 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:13.658808 12782 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:13.660602 12777 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:13.666934 12709 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:13.673956 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:13.675298 12706 raft_consensus.cc:2410] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:18:13.680368 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:13.681239 12709 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:13.684964 12597 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:13.691572 12774 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:13.695631 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:13.673082 14289 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:13.721199 14289 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:13.722132 14289 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:13.556018 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:13.724236 12700 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 4: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
W20260709 13:18:13.726338 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:13.682962 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 50628.1us
I20260709 13:18:13.741703 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:13.704107 14200 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.730566 12779 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:18:13.796522 12702 raft_consensus.cc:1240] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:13.793256 14183 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:18:13.798749 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 51284.2us
I20260709 13:18:13.750429 14166 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:13.810470 12705 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:13.813045 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:13.813936 12705 raft_consensus.cc:2410] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:13.746798 14282 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 13:18:13.749752 14288 consensus_queue.cc:237] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.370831 14126 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:13.744993 14145 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.792351 14200 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.811862 12779 raft_consensus.cc:606] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
W20260709 13:18:13.841636 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 143846us
I20260709 13:18:13.845686 12704 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:13.845924 12597 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:13.824944 14166 raft_consensus.cc:697] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 3 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:13.846801 12779 raft_consensus.cc:614] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:13.854414 12778 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:18:13.858961 12705 raft_consensus.cc:3055] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:13.859651 12705 raft_consensus.cc:740] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:13.829048 14206 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:13.867789 14250 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:13.869772 14250 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:13.870352 14250 raft_consensus.cc:697] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:13.871505 14250 consensus_queue.cc:237] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:13.872644 14166 consensus_queue.cc:237] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:13.873777 14255 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:13.874859 13383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 278) took 1293 ms. Trace:
I20260709 13:18:13.875391 13383 rpcz_store.cc:276] 0709 13:18:12.581870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.582930 (+  1060us) service_pool.cc:224] Handling call
0709 13:18:12.630060 (+ 47130us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:13.044871 (+414811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.874585 (+829714us) spinlock_profiling.cc:243] Waited 402 ms on lock 0x7b54002652d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:13.874781 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":924424064}
I20260709 13:18:13.877517 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:13.817584 14183 raft_consensus.cc:697] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:13.878307 12709 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 4: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:13.862334 12705 consensus_queue.cc:260] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:13.880142 12705 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:13.881626 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:13.883622 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:13.883677 12709 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:13.884234 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.884374 12776 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:13.885713 14183 consensus_queue.cc:237] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:13.894793 12709 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
W20260709 13:18:13.906885 14333 log.cc:927] Time spent T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:18:13.908730 12780 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:13.908922 12701 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:13.902207 14206 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:13.909734 12706 raft_consensus.cc:1275] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:13.910065 14206 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:13.982719 12776 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fa34731f37084c608660f0088a2dba09"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:18:13.983592 12776 raft_consensus.cc:606] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:18:13.984016 12776 raft_consensus.cc:614] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:13.880695 14336 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.903190 12703 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:18:14.018172 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:14.036237 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:14.036681 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:14.037107 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.041232 12705 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:18:14.044256 12709 raft_consensus.cc:1275] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:14.052634 12705 pending_rounds.cc:85] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 4
I20260709 13:18:14.053025 12705 pending_rounds.cc:107] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:18:14.044749 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:14.036456 14280 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.046195 14145 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:14.048573 12707 raft_consensus.cc:1275] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:18:14.049283 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:14.060483 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:14.051270 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.053344 12705 raft_consensus.cc:2889] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:14.004994 14207 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:14.068699 12626 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:18:14.069154 12626 pending_rounds.cc:85] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 1
I20260709 13:18:14.069440 12626 pending_rounds.cc:107] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:18:14.069706 12626 raft_consensus.cc:2889] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:14.071130 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:14.071954 12628 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:14.015167 14336 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:14.102737 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.103428 14268 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:14.106482 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.104362 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:14.090253 14288 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:18:14.101922 12628 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:14.106236 14268 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:14.115355 12673 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:14.116515 14264 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:14.117089 14264 raft_consensus.cc:697] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:14.117771 14268 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:14.090579 14304 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:14.118485 14304 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:14.118198 14264 consensus_queue.cc:237] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:14.087671 14334 consensus_queue.cc:1048] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:18:14.120602 14304 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:14.106635 14096 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:14.073650 14206 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:14.112437 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:14.077015 14145 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.058954 14317 consensus_queue.cc:1048] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:18:14.063025 14280 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:14.130820 12634 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:14.054020 14278 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:14.145623 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:14.155053 12706 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:14.114641 14031 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.161269 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:14.163677 12628 raft_consensus.cc:1275] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:14.130311 12775 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:14.164053 12634 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:14.132148 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:14.171979 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:14.167869 14242 consensus_meta.cc:287] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.635s	user 0.011s	sys 0.000s
W20260709 13:18:14.173583 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.174167 12628 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:18:14.175374 12634 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:14.180380 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:14.180416 12628 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:14.181016 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:14.182520 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:14.184188 12625 raft_consensus.cc:1275] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:14.185602 12625 raft_consensus.cc:1240] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:14.187116 12625 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:14.179970 12634 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:14.189992 12630 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:14.134047 14299 consensus_queue.cc:1059] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:18:14.190394 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:14.194352 12704 raft_consensus.cc:2410] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:18:14.188310 12703 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:14.194902 12631 raft_consensus.cc:2955] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:14.196307 14280 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:14.196235 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.196205 14096 raft_consensus.cc:3055] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:14.197248 14096 raft_consensus.cc:740] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:14.198360 14096 consensus_queue.cc:260] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:14.191880 14325 log.cc:927] Time spent T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.310s	user 0.001s	sys 0.000s
I20260709 13:18:14.189064 14242 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:14.203122 14347 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:14.203742 14347 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:14.169519 14126 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.226645 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:14.227469 12709 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:14.134944 14145 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:14.232347 12625 raft_consensus.cc:1275] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:18:14.234381 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 60053.7us
I20260709 13:18:14.235000 12701 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:14.234815 14126 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:14.225256 14289 consensus_meta.cc:287] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.002s
I20260709 13:18:14.168926 14031 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:14.241746 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:14.249830 12706 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
W20260709 13:18:14.261869 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 173038us
I20260709 13:18:14.270205 12775 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:14.270833 12775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36488 (request call id 1429) took 1059 ms. Trace:
I20260709 13:18:14.271216 12775 rpcz_store.cc:276] 0709 13:18:13.211210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.093171 (+881961us) service_pool.cc:224] Handling call
0709 13:18:14.270770 (+177599us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:14.275491 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:14.276165 12706 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:14.278471 14343 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:14.280565 14343 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:14.282335 14278 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:14.282668 14336 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:14.264246 14347 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:14.281190 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:14.267829 12700 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:14.311453 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:14.312106 12700 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:14.305189 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:14.313287 12708 raft_consensus.cc:2393] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 2: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:14.314908 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:14.325340 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:14.326519 12777 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:14.393108 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:14.393476 12775 raft_consensus.cc:1240] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:14.394830 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:14.397764 12804 debug-util.cc:398] Leaking SignalData structure 0x7b0800061020 after lost signal to thread 11520
W20260709 13:18:14.399322 12804 debug-util.cc:398] Leaking SignalData structure 0x7b08001713a0 after lost signal to thread 12566
I20260709 13:18:14.396191 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:14.400815 12708 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:14.409070 12780 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:14.411048 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:14.414855 12804 debug-util.cc:398] Leaking SignalData structure 0x7b0800056560 after lost signal to thread 12658
I20260709 13:18:14.415177 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:14.415840 12804 debug-util.cc:398] Leaking SignalData structure 0x7b08001e9240 after lost signal to thread 12659
I20260709 13:18:14.248713 14289 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:14.416584 12776 raft_consensus.cc:2393] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 2: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:14.285728 14317 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.288029 14348 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.423769 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:14.428670 12706 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
W20260709 13:18:14.430948 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:14.443136 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.419896 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:18:14.448427 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.341711 14278 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.398523 14334 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:14.288303 14349 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:18:14.414667 14166 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.027s	sys 0.085s
I20260709 13:18:14.453595 12706 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:14.445498 14317 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:14.458817 12776 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:18:14.459904 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 220022us
I20260709 13:18:14.464906 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:14.470310 12780 raft_consensus.cc:2368] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3.
I20260709 13:18:14.474184 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:14.474368 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:14.463419 14285 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.421820 14031 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:14.463882 12706 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:14.477900 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:14.464829 14166 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.028s	sys 0.085s
I20260709 13:18:14.450759 12701 raft_consensus.cc:2393] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 4: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:14.415102 14304 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:18:14.450426 14336 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.465745 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:14.475190 12780 raft_consensus.cc:2393] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:14.485936 12783 raft_consensus.cc:2368] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3.
I20260709 13:18:14.451660 14348 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:14.461449 12672 leader_election.cc:400] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3.
I20260709 13:18:14.487057 12672 leader_election.cc:403] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:18:14.464439 14334 consensus_queue.cc:1048] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
W20260709 13:18:14.467940 14358 log.cc:927] Time spent T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 13:18:14.479616 13349 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:14.493230 13349 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:14.479869 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:14.455687 14278 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:14.483387 12706 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:14.491567 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:14.495185 12672 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:14.474033 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:14.497813 12783 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:14.462042 14349 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:14.496469 14352 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:18:14.506829 12780 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:18:14.507306 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 99069.3us
I20260709 13:18:14.512298 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:14.515791 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:14.516570 12775 raft_consensus.cc:2368] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3.
I20260709 13:18:14.519924 12777 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:14.499305 14219 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3.
I20260709 13:18:14.483922 14285 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:14.532545 12672 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:14.506829 14360 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:14.492211 14336 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:14.507916 14166 raft_consensus.cc:2764] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:18:14.515133 14145 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:14.548187 14348 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:14.550088 14273 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:14.519562 14347 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:18:14.556555 14356 log.cc:927] Time spent T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:18:14.557874 14336 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:14.558007 12626 raft_consensus.cc:2393] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:14.597150 14207 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.598029 14207 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:14.598685 14126 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:14.600008 14348 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.600546 14348 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:14.558806 14289 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:18:14.559804 14317 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:14.560334 14304 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:14.561470 14349 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:14.545073 12779 raft_consensus.cc:2393] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 2: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:14.561906 14362 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:14.546146 12672 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:14.562386 14285 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:14.563082 14278 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:14.557401 14357 log.cc:927] Time spent T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:18:14.522758 14292 raft_consensus.cc:2764] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:18:14.566380 12708 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:14.573132 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:14.602474 14348 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:14.576089 14347 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:14.608613 14183 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:14.606565 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 113387us
W20260709 13:18:14.610272 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 94575us
I20260709 13:18:14.610272 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:14.623378 12631 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:14.601173 14207 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:14.617851 12783 raft_consensus.cc:1275] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:14.618602 12707 raft_consensus.cc:2410] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:18:14.608981 14026 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.621393 12708 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:14.625416 12632 raft_consensus.cc:1240] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:14.605777 14273 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:14.611688 14361 consensus_queue.cc:1059] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.063s
W20260709 13:18:14.606254 12598 leader_election.cc:336] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.637s (SENT)
I20260709 13:18:14.546514 14264 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:14.632304 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:14.632561 14349 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:14.634258 13383 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 1 LEARNER]: Advancing to term 3
I20260709 13:18:14.622382 14347 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:14.637475 12708 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:14.630682 12598 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:14.642652 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:14.643662 12630 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
W20260709 13:18:14.649838 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 103814us
I20260709 13:18:14.645191 12673 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:14.652077 12672 leader_election.cc:400] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4.
I20260709 13:18:14.652534 12672 leader_election.cc:403] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:18:14.654699 14273 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:14.655231 12708 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:18:14.655651 12708 pending_rounds.cc:85] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 5
I20260709 13:18:14.655893 12708 pending_rounds.cc:107] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:18:14.656147 12708 raft_consensus.cc:2889] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:14.637526 14365 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.658188 14365 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:14.661139 13376 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:14.661654 14336 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:14.662750 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:18:14.662958 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 56728.5us
W20260709 13:18:14.663856 12598 leader_election.cc:336] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.820s (SENT)
I20260709 13:18:14.664069 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:14.664348 12598 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:14.665640 13380 raft_consensus.cc:1240] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:14.629468 14026 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:14.667014 14207 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:14.637969 14304 consensus_queue.cc:1059] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:18:14.668377 14304 consensus_queue.cc:1243] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:18:14.668438 14026 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:14.668951 14304 raft_consensus.cc:3055] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:14.669407 14304 raft_consensus.cc:740] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:14.670414 14304 consensus_queue.cc:260] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:14.671283 12700 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:14.671834 14304 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:14.629258 14278 consensus_queue.cc:1048] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
W20260709 13:18:14.680910 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.680604 12747 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:14.690691 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:14.690734 13349 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:14.691651 12708 raft_consensus.cc:2410] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:18:14.710621 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 65566us
I20260709 13:18:14.714125 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:14.626575 14183 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:14.711907 12673 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:14.704442 12672 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:14.729346 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.730963 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:14.731594 12630 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:14.676878 14110 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:14.742702 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:14.689993 14292 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:18:14.755014 12804 debug-util.cc:398] Leaking SignalData structure 0x7b08001ea940 after lost signal to thread 12733
W20260709 13:18:14.755738 12804 debug-util.cc:398] Leaking SignalData structure 0x7b08001ea900 after lost signal to thread 12734
I20260709 13:18:14.755877 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:14.756803 12709 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
W20260709 13:18:14.757390 12804 debug-util.cc:398] Leaking SignalData structure 0x7b080035b8a0 after lost signal to thread 12807
W20260709 13:18:14.757902 12804 debug-util.cc:398] Leaking SignalData structure 0x7b080035bac0 after lost signal to thread 12808
I20260709 13:18:14.758270 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:14.761304 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:14.762079 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.693717 14126 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4.
I20260709 13:18:14.763983 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:14.711040 14361 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:14.724705 14360 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:14.693329 14273 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:18:14.770006 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.742292 14219 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:14.773607 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:14.739902 14264 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:14.690291 14278 consensus_queue.cc:1059] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.082s
I20260709 13:18:14.723203 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:14.783593 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 54177.2us
I20260709 13:18:14.748826 14365 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:14.767441 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:14.727240 14285 consensus_queue.cc:1059] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:18:14.724045 14289 consensus_queue.cc:1059] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:18:14.735507 14349 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
W20260709 13:18:14.782837 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:14.811808 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.736968 14166 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:18:14.794831 12672 leader_election.cc:400] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3.
I20260709 13:18:14.766623 14352 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:18:14.783996 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:14.816964 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:14.814030 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:14.814416 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.815191 12672 leader_election.cc:403] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:18:14.732383 14362 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:18:14.785283 14219 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:14.808495 12708 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
W20260709 13:18:14.833277 13348 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.099s	user 0.038s	sys 0.050s
W20260709 13:18:14.833948 13348 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.100s	user 0.040s	sys 0.050s
W20260709 13:18:14.834309 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 1.10058e+06us
I20260709 13:18:14.841679 14322 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.115s
I20260709 13:18:14.842868 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:14.843727 12705 raft_consensus.cc:2410] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:18:14.843578 14183 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:14.844008 14289 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:14.843602 14363 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:14.844745 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:14.793594 14292 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.827857 12672 leader_election.cc:400] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3.
I20260709 13:18:14.828336 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:14.849879 12672 leader_election.cc:403] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:18:14.844527 14289 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:14.845410 12705 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:14.855654 12598 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:14.828056 14366 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:14.859694 12630 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:14.859911 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:14.845844 14219 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:14.846210 14273 raft_consensus.cc:3055] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:14.813146 14365 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.819335 14166 consensus_queue.cc:1243] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer 639ab2db265f4747bb7afb93e833f530 log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:18:14.818712 14285 consensus_queue.cc:1243] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:18:14.903337 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:14.861318 13384 raft_consensus.cc:1240] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:14.907076 12598 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:14.918237 12629 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
W20260709 13:18:14.869977 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 142568us
I20260709 13:18:14.910184 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:14.910676 14343 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:14.911567 14375 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3.
I20260709 13:18:14.912771 13383 raft_consensus.cc:1275] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 3 LEARNER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:14.934669 12782 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:14.900043 14366 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:14.922363 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:14.845044 14363 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:14.904775 14364 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3.
I20260709 13:18:14.936292 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:14.946159 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 118436us
I20260709 13:18:14.907202 14273 raft_consensus.cc:740] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:14.906526 14373 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.947011 12672 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:14.858397 14292 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:14.950829 12747 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:14.953044 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:14.953835 12705 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:14.954656 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:14.931180 14183 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:14.932269 14282 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:14.958947 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:14.921497 14145 consensus_queue.cc:1059] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:18:14.916677 14365 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:14.946475 14366 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:14.977272 13382 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:18:14.955355 14362 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:14.936721 14268 consensus_queue.cc:1059] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:18:14.933239 14285 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:18:14.953043 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:14.957647 14349 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:14.961364 14364 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:14.970978 14361 raft_consensus.cc:697] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:14.934933 14343 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:14.957018 14342 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
W20260709 13:18:15.004622 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 56239.2us
I20260709 13:18:14.982118 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:14.989509 14362 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.000684 13380 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:18:14.956183 14373 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:15.004491 14349 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.012089 14376 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:15.014998 12804 debug-util.cc:398] Leaking SignalData structure 0x7b0800383de0 after lost signal to thread 13410
W20260709 13:18:15.015354 12804 debug-util.cc:398] Leaking SignalData structure 0x7b0800380f80 after lost signal to thread 13411
I20260709 13:18:14.987383 14375 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:18:15.011523 12598 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:15.008204 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:15.025246 12630 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:18:15.011415 14362 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:15.022348 14368 log.cc:927] Time spent T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:18:15.021863 14372 consensus_queue.cc:1048] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 13:18:15.023078 14364 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:15.023260 14361 consensus_queue.cc:237] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.005744 14166 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:15.025733 12630 pending_rounds.cc:85] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 1
I20260709 13:18:15.068219 12630 pending_rounds.cc:107] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:18:15.068676 12630 raft_consensus.cc:2889] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:15.021831 14381 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:15.005901 14352 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:18:15.037007 12598 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:15.037762 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:15.001700 14273 consensus_queue.cc:260] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:15.039098 12633 raft_consensus.cc:1275] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:15.029151 14289 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:15.026008 14349 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:15.013684 14026 consensus_queue.cc:1048] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:18:15.046164 14264 raft_consensus.cc:3055] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:15.047833 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:15.086443 12706 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:15.049422 12705 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
W20260709 13:18:15.073077 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 44162.5us
I20260709 13:18:15.080837 12779 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:15.087893 12598 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:15.048223 14288 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:15.040881 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:15.076825 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:15.032860 12701 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:15.090312 12598 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:15.073829 14343 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:15.077500 14268 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:15.102090 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:15.090391 12700 raft_consensus.cc:2410] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:18:15.075286 14386 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:15.102641 12706 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:15.108299 12598 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:15.025542 14376 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:15.086126 14264 raft_consensus.cc:740] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:15.101231 14288 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:15.121747 14268 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:15.107107 14386 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.122750 14386 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:15.134805 14096 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:15.134939 14386 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.069417 14364 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.138653 14386 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:15.074322 14166 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:15.101400 14342 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:15.114462 14381 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:15.119626 14282 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:15.104404 14207 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 13:18:15.143941 14342 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.151857 14342 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:15.139310 14364 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:15.107645 14336 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:18:15.165134 14363 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:15.166235 12779 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:15.166385 14264 consensus_queue.cc:260] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:15.145205 14292 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:15.145140 14366 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:15.170270 14166 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:15.171406 14366 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:15.146057 14219 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.149878 14381 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:15.172634 14342 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.173933 12709 raft_consensus.cc:2368] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5.
I20260709 13:18:15.174005 14381 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:15.176168 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 67927.3us
I20260709 13:18:15.179219 12598 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:15.181917 14219 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:15.178369 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.165364 12701 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:15.186863 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:15.187750 12701 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:15.188946 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:15.168253 14373 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:15.191387 14288 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:15.180943 14336 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:15.203008 14381 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:15.203787 14381 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.208473 12701 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:15.209792 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:15.210546 12701 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:15.218698 14362 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.153262 14347 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:15.211640 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:15.238169 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 59012.9us
I20260709 13:18:15.239051 12778 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:15.244607 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:15.250641 13382 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:15.233913 13376 raft_consensus.cc:1240] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:15.235044 13381 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:15.251772 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:15.253839 14207 consensus_queue.cc:1059] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:15.254223 14321 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:15.254848 13384 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:15.254735 14321 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.256758 14321 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:15.257195 14299 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.257721 14322 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:15.257791 14299 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.258315 14322 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:15.258520 14336 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:15.259922 14299 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:15.260262 14322 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:15.250510 14373 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:15.261188 12598 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:15.262681 12597 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:15.237926 14096 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:15.265848 14096 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:15.266752 12598 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:15.269753 12597 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:15.266844 14373 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.270694 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:15.269966 14336 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:15.283010 14373 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:15.284248 14377 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:15.263441 12701 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:15.285063 14377 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:15.286857 14359 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.784s	user 0.005s	sys 0.000s
I20260709 13:18:15.287369 14377 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:15.274837 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:15.221531 14362 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:15.223779 14342 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:15.274840 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:15.235438 14347 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:15.276429 13381 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:15.278208 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:15.279039 14282 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:15.254751 14207 raft_consensus.cc:3055] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:15.238711 14365 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:15.229027 14381 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:15.280898 12598 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:15.281452 14026 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:15.275959 12779 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:18:15.289686 14336 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:15.291244 13377 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 3 LEARNER]: Advancing to term 4
I20260709 13:18:15.291848 14268 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.293136 12634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:18:15.293243 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:15.295454 12597 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:18:15.295693 12629 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39590 (request call id 139) took 2315 ms (client timeout 2212 ms). Trace:
I20260709 13:18:15.296449 12707 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:15.296693 14200 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:15.323796 14399 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:18:15.324738 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:15.303313 14207 raft_consensus.cc:740] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:15.326054 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:15.301398 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:15.328516 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:15.304615 14394 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:15.331611 14394 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:15.299705 12780 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:15.334169 14394 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:15.334992 14096 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:15.234262 14349 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.311297 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:15.299746 14373 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:15.309689 12779 pending_rounds.cc:85] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 5
I20260709 13:18:15.326205 14207 consensus_queue.cc:260] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.308736 14026 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.337229 12634 raft_consensus.cc:606] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Received request to transfer leadership to TS 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:15.312583 12598 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:15.280875 14386 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:15.342793 12780 raft_consensus.cc:1275] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:15.342994 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:15.343017 12781 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
W20260709 13:18:15.326567 12629 rpcz_store.cc:269] 0709 13:18:12.980407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.988172 (+  7765us) service_pool.cc:224] Handling call
0709 13:18:15.295477 (+2307305us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54002fe9d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:15.295607 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3631898880}
I20260709 13:18:15.348953 14200 consensus_queue.cc:1059] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:15.350081 14207 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:15.348078 12779 pending_rounds.cc:107] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 3.6
W20260709 13:18:15.361176 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 65793.5us
I20260709 13:18:15.363986 12528 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54814 (request call id 218) took 5549 ms. Trace:
I20260709 13:18:15.336901 13377 raft_consensus.cc:1275] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEARNER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:15.365072 12746 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:15.312885 14381 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:15.358584 14026 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:15.377445 14183 raft_consensus.cc:3055] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:15.360935 14386 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:15.385906 13348 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:18:15.391027 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 78527.3us
I20260709 13:18:15.357605 14336 raft_consensus.cc:697] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:15.360700 14361 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.392781 12598 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:15.388186 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:15.389256 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:15.389894 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:15.366163 14399 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:15.392220 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:15.395902 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:15.345984 14268 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.370034 14342 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:15.366060 12779 raft_consensus.cc:2889] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:15.396607 12704 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:15.374691 14110 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:15.358282 14373 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.403457 12704 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:15.405002 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:15.405048 12747 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:15.405930 12704 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:15.400152 14361 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:15.393525 14200 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.367054 12528 rpcz_store.cc:276] 0709 13:18:09.814921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.815072 (+   151us) service_pool.cc:224] Handling call
0709 13:18:15.363922 (+5548850us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:13.274768 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:13.294671 (+ 19903us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:13.295088 (+   417us) write_op.cc:435] Acquired schema lock
0709 13:18:13.295991 (+   903us) tablet.cc:662] Decoding operations
0709 13:18:13.316453 (+ 20462us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:13.316542 (+    89us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:13.316572 (+    30us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:18:13.316958 (+   386us) tablet.cc:701] Row locks acquired
0709 13:18:13.316980 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:18:13.317128 (+   148us) write_op.cc:270] Start()
0709 13:18:13.317255 (+   127us) write_op.cc:276] Timestamp: P: 1783603093317090 usec, L: 0
0709 13:18:13.317281 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:18:13.381317 (+ 64036us) log.cc:844] Serialized 1680 byte log entry
0709 13:18:14.116426 (+735109us) op_driver.cc:464] REPLICATION: finished
0709 13:18:14.264013 (+147587us) write_op.cc:301] APPLY: starting
0709 13:18:14.266681 (+  2668us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:14.282936 (+ 16255us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:14.283082 (+   146us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:14.289209 (+  6127us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:14.398193 (+108984us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:18:14.398273 (+    80us) write_op.cc:312] APPLY: finished
0709 13:18:14.459650 (+ 61377us) log.cc:844] Serialized 41 byte log entry
0709 13:18:14.471602 (+ 11952us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:14.477527 (+  5925us) write_op.cc:454] Released schema lock
0709 13:18:14.481433 (+  3906us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:14.514919 (+ 33486us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:14.517855 (+  2936us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":53880,"reported_tablets":10,"spinlock_wait_cycles":1280,"child_traces":[["op",{"apply.queue_time_us":120833,"apply.run_cpu_time_us":10539,"apply.run_wall_time_us":239649,"num_ops":4,"prepare.queue_time_us":32798,"prepare.run_cpu_time_us":10485,"prepare.run_wall_time_us":106888,"replication_time_us":798984,"thread_start_us":79615,"threads_started":2}]]}
I20260709 13:18:15.408363 14200 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.409713 14395 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:18:15.386215 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:15.392763 14386 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:15.411532 14377 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:15.412081 14377 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:15.404261 14096 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:15.406762 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:15.414089 12777 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:15.386582 14183 raft_consensus.cc:740] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:15.398447 14342 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.444592 12704 raft_consensus.cc:2393] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:15.444876 14342 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:15.415560 12781 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:15.363875 14362 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:18:15.395993 14360 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:15.449692 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:15.398442 14145 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:15.464075 14145 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:15.464382 14377 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:15.464641 14145 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:15.465512 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:15.465730 14360 raft_consensus.cc:697] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:15.466285 14373 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:15.396288 14399 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.449821 14031 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:15.466816 14360 consensus_queue.cc:237] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:15.467674 14031 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.468080 14031 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:15.386813 14381 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:15.470323 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:15.471448 14381 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:15.471611 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:15.472750 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:15.451242 14219 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:15.472764 14349 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:15.451730 14398 consensus_queue.cc:1059] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.096s
I20260709 13:18:15.474606 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:15.475272 14398 raft_consensus.cc:3055] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:15.475514 12779 raft_consensus.cc:2393] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:15.475666 14398 raft_consensus.cc:740] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:15.451542 14288 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.476516 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:15.476840 14288 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.399837 14365 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.079s
I20260709 13:18:15.479069 14288 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:15.413681 14096 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.452050 14336 consensus_queue.cc:237] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.480284 12775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36478 (request call id 1666) took 1073 ms. Trace:
I20260709 13:18:15.480960 14096 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:15.454443 14396 log.cc:927] Time spent T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:18:15.457134 14394 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:15.460053 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:15.453699 14395 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.462350 14200 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:15.404683 14304 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.467172 14206 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:15.473980 14219 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.481796 12778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36478 (request call id 1673) took 1014 ms. Trace:
I20260709 13:18:15.498755 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:15.480665 12775 rpcz_store.cc:276] 0709 13:18:14.407429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.410595 (+  3166us) service_pool.cc:224] Handling call
0709 13:18:15.480098 (+1069503us) spinlock_profiling.cc:243] Waited 91.9 ms on lock 0x7b54001fd954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:15.480228 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":211403520}
I20260709 13:18:15.506654 12778 rpcz_store.cc:276] 0709 13:18:14.467016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.467165 (+   149us) service_pool.cc:224] Handling call
0709 13:18:15.481633 (+1014468us) spinlock_profiling.cc:243] Waited 764 ms on lock 0x7b540044c0d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:15.481730 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1757372800}
I20260709 13:18:15.488142 14399 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:15.449020 12747 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:15.510654 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 61687.3us
I20260709 13:18:15.510993 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:15.511579 13384 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:18:15.477324 12779 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:15.514878 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:15.515122 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:15.515493 12775 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:18:15.516124 12629 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 4: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:15.511781 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:15.491452 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:15.482554 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:15.508395 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:15.518555 12630 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:15.512768 12672 leader_election.cc:400] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3.
I20260709 13:18:15.450385 12704 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:15.517380 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:15.520522 12672 leader_election.cc:403] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:18:15.522464 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:15.517407 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:15.523471 12704 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 2: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:15.526505 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:15.527719 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 135071us
I20260709 13:18:15.506961 14206 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.528370 12598 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:15.537845 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:15.521420 14200 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:15.514103 14352 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.542472 13349 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:15.526600 14250 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.411s	user 0.007s	sys 0.031s
I20260709 13:18:15.540977 14207 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:18:15.546993 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:15.543087 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:15.552718 12704 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:15.527556 14288 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3.
I20260709 13:18:15.527315 14349 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:15.492707 14395 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:15.556257 12634 raft_consensus.cc:2393] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:15.549528 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:15.534791 14110 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:15.537393 14206 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:15.539366 14322 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:15.508468 14304 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.554600 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:15.546437 14378 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:18:15.553139 14250 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.445s	user 0.008s	sys 0.031s
I20260709 13:18:15.546890 14200 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:15.575907 14200 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:15.547382 14361 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:15.476526 14398 consensus_queue.cc:260] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:15.563218 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:15.552002 14365 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:15.567571 14268 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:15.567878 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:15.568717 14219 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:15.569116 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:15.598253 12706 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:15.558279 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:15.558121 14345 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:18:15.559170 14110 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.574478 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:15.564440 14343 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:15.575171 14304 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:15.576577 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:15.555498 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:15.577908 13349 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:15.578258 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:15.585664 14288 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:18:15.590925 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:15.648756 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 70926.1us
I20260709 13:18:15.562515 14322 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:15.562031 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:15.596650 12783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 1678) took 1076 ms. Trace:
I20260709 13:18:15.650638 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:15.651274 12628 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:15.652398 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:15.652855 13349 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:15.597108 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:18:15.633394 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 234281us
I20260709 13:18:15.566622 14388 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:15.570159 14400 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:15.636927 12709 raft_consensus.cc:2393] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 2: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:15.639304 12702 raft_consensus.cc:2393] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 4: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:15.582677 14378 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:15.646047 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:15.647262 12634 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:15.584867 12704 raft_consensus.cc:2368] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5.
I20260709 13:18:15.652964 12628 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:15.576974 14386 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.552034 14352 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:15.690191 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:15.655193 12780 raft_consensus.cc:2368] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5.
I20260709 13:18:15.650836 12783 rpcz_store.cc:276] 0709 13:18:14.510236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.514232 (+  3996us) service_pool.cc:224] Handling call
0709 13:18:14.615808 (+101576us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.624044 (+  8236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.586772 (+962728us) spinlock_profiling.cc:243] Waited 91.6 ms on lock 0x7b54002cb350. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:15.586933 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":210751744}
I20260709 13:18:15.654557 12672 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:15.690793 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:15.691817 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:15.692350 12709 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:15.656062 12597 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:15.693269 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:15.693856 12702 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:15.633332 12700 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:15.666416 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:15.686520 13349 leader_election.cc:400] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5.
I20260709 13:18:15.661609 12706 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
W20260709 13:18:15.722021 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 64546.3us
I20260709 13:18:15.639592 14110 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:15.668529 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.595598 14365 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:15.636042 14398 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:15.655093 14322 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:15.665799 12632 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:15.695741 12628 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:15.649897 14304 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.742218 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:15.737476 12707 raft_consensus.cc:1240] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:18:15.656337 14363 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:15.744444 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:15.663918 14400 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.689527 14166 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:15.749993 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:15.749405 12628 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:15.739512 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:15.754231 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:15.754462 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:15.724613 14219 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:15.655483 14288 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.769979 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:15.730163 14268 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:15.763019 12704 pending_rounds.cc:85] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 4
I20260709 13:18:15.771145 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:15.771534 12704 pending_rounds.cc:107] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:18:15.771817 12704 raft_consensus.cc:2889] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:15.773041 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:15.773578 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:15.774112 12628 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:15.775995 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:15.763772 12783 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:15.776575 12634 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:15.767901 12632 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
W20260709 13:18:15.770413 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.795665 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:15.798825 12632 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:15.755147 14166 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:15.755877 14400 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:15.755546 14363 raft_consensus.cc:697] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:15.768709 14219 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.746994 14250 consensus_queue.cc:1048] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:18:15.802614 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:15.746764 14304 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:15.819583 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:15.821830 14304 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:15.822198 14363 consensus_queue.cc:237] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:15.816056 14219 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:15.823230 14416 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.833858 14416 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.836243 14416 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:15.836869 14416 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:15.776775 14334 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:15.805639 14166 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:15.836952 14408 consensus_queue.cc:1048] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:15.809314 14299 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:15.837365 14416 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.797502 14026 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.795439 14031 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.838060 12597 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:15.842602 12634 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:15.844776 14395 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:15.850387 14416 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:15.909665 14401 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:15.844208 14334 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.849308 14206 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.848877 14299 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:15.937069 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:15.952797 12774 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:15.961571 14365 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:16.088402 12746 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.014s	sys 0.014s
I20260709 13:18:15.987843 14031 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:15.976897 12625 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: []
W20260709 13:18:16.079623 14096 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.011s	sys 0.015s
W20260709 13:18:16.174865 14096 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.011s	sys 0.026s
I20260709 13:18:15.996431 14400 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:16.123140 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:16.117158 14349 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.020s	sys 0.014s
W20260709 13:18:16.177481 14349 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.022s	sys 0.015s
I20260709 13:18:16.177901 14349 raft_consensus.cc:697] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:16.114616 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:15.957687 14419 log.cc:927] Time spent T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:18:15.985869 14378 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:16.147315 14334 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:16.173373 12746 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.014s	sys 0.022s
I20260709 13:18:15.982556 12703 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:16.182379 12627 raft_consensus.cc:1240] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:18:15.929522 12628 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:15.999050 13349 leader_election.cc:400] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5.
W20260709 13:18:16.168478 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 429009us
I20260709 13:18:16.001036 12672 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:15.936875 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:16.201274 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 200313us
I20260709 13:18:16.203943 12672 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:16.074318 12633 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:18:16.200488 12747 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:16.205617 12746 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:16.209811 12627 raft_consensus.cc:1240] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:18:16.199699 13349 leader_election.cc:403] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:18:16.073110 14288 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:16.170580 14154 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:16.105835 14026 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:15.961622 14408 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.247s
I20260709 13:18:16.170207 14375 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:18:16.201601 12634 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:15.960817 14401 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.957957 14273 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:15.985169 14321 raft_consensus.cc:3055] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:15.989050 14299 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:16.213372 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 214696us
I20260709 13:18:16.003484 14381 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.003261 14126 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:16.247161 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 118522us
I20260709 13:18:16.249215 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:16.249538 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:16.250423 13380 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:16.250814 13381 raft_consensus.cc:2393] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:16.252449 14260 consensus_queue.cc:1059] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:15.984158 14206 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:16.243481 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 332288us
W20260709 13:18:16.225785 14145 consensus_meta.cc:287] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.761s	user 0.012s	sys 0.000s
I20260709 13:18:16.244127 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:16.254951 14145 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:16.256296 12707 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:16.210893 14365 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:18:16.258037 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 52569.1us
I20260709 13:18:15.976472 14352 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:16.133674 14292 consensus_queue.cc:1048] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.104s
I20260709 13:18:16.260735 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:16.238474 14342 consensus_meta.cc:287] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.752s	user 0.009s	sys 0.003s
W20260709 13:18:16.332911 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 68876us
I20260709 13:18:16.261344 12781 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
W20260709 13:18:16.333982 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.225818 14390 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5.
I20260709 13:18:16.275470 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:16.348105 12775 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:16.276865 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:16.350590 12776 raft_consensus.cc:1240] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:18:16.279873 14154 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:16.281100 12708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1011) took 1280 ms. Trace:
I20260709 13:18:16.289114 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:16.293017 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:16.307451 14183 consensus_queue.cc:260] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:16.310567 12774 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1451) took 1035 ms. Trace:
I20260709 13:18:16.364334 12774 rpcz_store.cc:276] 0709 13:18:15.275457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.275828 (+   371us) service_pool.cc:224] Handling call
0709 13:18:15.482227 (+206399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.005104 (+522877us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.309443 (+304339us) spinlock_profiling.cc:243] Waited 470 ms on lock 0x7b54001fd954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:16.310222 (+   779us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54001fd950. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:16.310503 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1555569664}
I20260709 13:18:16.313133 14386 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:16.319896 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:16.322472 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:16.328120 13385 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:16.272584 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:16.178285 14400 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:15.996872 14166 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:16.344240 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 143871us
I20260709 13:18:16.239935 14288 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:16.243273 14401 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:16.360435 12703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 133) took 1640 ms. Trace:
I20260709 13:18:16.076540 14322 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:16.356058 12708 rpcz_store.cc:276] 0709 13:18:15.000824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.377404 (+376580us) service_pool.cc:224] Handling call
0709 13:18:15.377705 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.450065 (+ 72360us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:18:15.656666 (+206601us) log.cc:844] Serialized 14 byte log entry
0709 13:18:15.968781 (+312115us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:15.968851 (+    70us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:15.968917 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.280767 (+311850us) spinlock_profiling.cc:243] Waited 72.2 ms on lock 0x7b540006e854. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:16.280824 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.281021 (+   197us) inbound_call.cc:177] Queueing success response
0709 13:18:16.303132 (+ 22111us) spinlock_profiling.cc:243] Waited 1.43 ms on lock 0x7b180030f520. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c53f3 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
Metrics: {"prepare.queue_time_us":364546,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":2335,"raft.queue_time_us":850,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":125,"spinlock_wait_cycles":179345408,"thread_start_us":477170,"threads_started":2,"wal-append.queue_time_us":487644}
I20260709 13:18:16.371376 14441 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:16.354882 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:18:16.209236 14349 consensus_queue.cc:237] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:16.247478 14321 raft_consensus.cc:740] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:16.235172 14110 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:16.333489 14365 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:16.412340 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:16.347719 14145 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:16.351147 14395 consensus_queue.cc:1048] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 13:18:16.426532 13382 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:16.347219 14126 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:16.363673 14154 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:16.405030 12703 rpcz_store.cc:276] 0709 13:18:14.704029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.237033 (+533004us) service_pool.cc:224] Handling call
0709 13:18:15.237351 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.196759 (+959408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.343671 (+146912us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b54002495d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:16.343936 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1712871808}
W20260709 13:18:16.378319 14412 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.044s	sys 0.048s
I20260709 13:18:16.353003 14299 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:16.372037 14429 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:16.397892 14376 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:16.331926 14381 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:16.444909 14299 raft_consensus.cc:697] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:16.446163 12747 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:16.352612 14342 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:16.450052 14427 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.450562 14427 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:16.451061 14342 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:16.399446 14288 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:16.451851 14288 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:16.452797 14427 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:16.453310 14373 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:16.453680 14429 raft_consensus.cc:697] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:16.401037 13377 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 1.
I20260709 13:18:16.446003 14299 consensus_queue.cc:237] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:16.449262 14381 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:16.442080 14362 consensus_queue.cc:1059] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
W20260709 13:18:16.459522 14431 log.cc:927] Time spent T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.300s	user 0.003s	sys 0.000s
I20260709 13:18:16.460187 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:16.460737 14421 log.cc:927] Time spent T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.302s	user 0.000s	sys 0.001s
W20260709 13:18:16.441774 14412 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.044s	sys 0.048s
I20260709 13:18:16.443394 14390 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:16.477134 12701 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:16.481722 12775 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:16.424139 14365 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:16.485433 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:16.506146 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:16.506384 12747 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:16.465930 13348 leader_election.cc:400] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5.
W20260709 13:18:16.440621 14219 consensus_meta.cc:287] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.608s	user 0.005s	sys 0.005s
I20260709 13:18:16.506978 12775 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:16.500356 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:16.507845 12702 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:16.439905 14183 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:16.464926 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:16.509078 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:16.511533 12702 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:16.508260 13348 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:16.513075 12747 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:16.501833 14416 consensus_meta.cc:287] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.542s	user 0.005s	sys 0.005s
I20260709 13:18:16.515156 14376 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:16.515205 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:16.496862 14360 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.517014 12706 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:16.462507 14288 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:16.521440 13348 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:16.518042 12747 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:16.508414 14427 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:16.541684 14410 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.015s	sys 0.054s
W20260709 13:18:16.542114 14410 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.015s	sys 0.054s
I20260709 13:18:16.519318 14334 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:18:16.535291 14200 consensus_meta.cc:287] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.959s	user 0.007s	sys 0.005s
I20260709 13:18:16.536657 14360 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:16.537369 14352 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:16.539872 14321 consensus_queue.cc:260] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:16.547632 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:16.538763 12747 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:16.521452 14096 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:16.659152 12706 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:16.537305 14362 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:16.553171 14429 consensus_queue.cc:237] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:16.611660 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 73629.7us
I20260709 13:18:16.620321 12780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 1690) took 1208 ms. Trace:
I20260709 13:18:16.685142 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:16.685806 12780 rpcz_store.cc:276] 0709 13:18:15.412335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.785207 (+372872us) service_pool.cc:224] Handling call
0709 13:18:15.812161 (+ 26954us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:15.812397 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:15.812414 (+    17us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:15.812819 (+   405us) log.cc:844] Serialized 26 byte log entry
0709 13:18:16.240184 (+427365us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:16.375503 (+135319us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.408326 (+ 32823us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.408407 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:16.620008 (+211601us) raft_consensus.cc:1685] finished
0709 13:18:16.620041 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.620237 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75028,"prepare.queue_time_us":134954,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":8756,"raft.queue_time_us":57350,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":22855,"thread_start_us":390981,"threads_started":1,"wal-append.queue_time_us":787270}
W20260709 13:18:16.621644 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:16.622151 14282 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.000s	sys 0.001s
I20260709 13:18:16.630934 12783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 1683) took 1240 ms. Trace:
I20260709 13:18:16.516172 14219 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:16.549305 14288 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:16.632844 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:16.696079 12783 rpcz_store.cc:276] 0709 13:18:15.390058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.782802 (+392744us) service_pool.cc:224] Handling call
0709 13:18:15.783202 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:15.783652 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:15.783692 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:15.784243 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 13:18:16.168957 (+384714us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:16.169743 (+   786us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.169945 (+   202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.170021 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:16.458975 (+288954us) spinlock_profiling.cc:243] Waited 3.4 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:18:16.630513 (+171538us) raft_consensus.cc:1685] finished
0709 13:18:16.630636 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.630851 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":211,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":54243,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":26968,"spinlock_wait_cycles":7822080,"thread_start_us":336714,"threads_started":1,"wal-append.queue_time_us":340490}
W20260709 13:18:16.678476 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 430126us
I20260709 13:18:16.519071 14416 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:16.705535 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:16.706825 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:16.700868 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:16.692348 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 153592us
I20260709 13:18:16.631379 14408 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.511078 14390 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:16.703948 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:16.713953 12634 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:16.714454 12778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 139) took 1183 ms. Trace:
I20260709 13:18:16.553445 14334 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:16.623890 14426 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:16.695994 14282 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.000s	sys 0.002s
I20260709 13:18:16.689644 14352 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:16.709939 12747 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:16.714998 12778 rpcz_store.cc:276] 0709 13:18:15.530541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.995435 (+464894us) service_pool.cc:224] Handling call
0709 13:18:15.995773 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:15.996233 (+   460us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:15.996526 (+   293us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.469447 (+472921us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:16.477321 (+  7874us) log.cc:844] Serialized 26 byte log entry
0709 13:18:16.477471 (+   150us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:16.477512 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.477635 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:16.714210 (+236575us) raft_consensus.cc:1685] finished
0709 13:18:16.714231 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.714383 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7382,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":54,"raft.queue_time_us":295,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":200,"thread_start_us":472432,"threads_started":1,"wal-append.queue_time_us":537936}
I20260709 13:18:16.711329 12673 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:16.634140 14394 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:16.655246 14096 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:16.696938 14031 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:16.763626 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:16.721971 12777 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:16.722265 14334 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:16.722747 14432 log.cc:927] Time spent T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:18:16.578717 14200 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:16.704433 14264 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:16.760471 12781 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 4. Current term is 5. Ops: []
W20260709 13:18:16.762871 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 53115.8us
I20260709 13:18:16.770380 14288 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:16.719063 14399 consensus_meta.cc:287] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 1.193s	user 0.013s	sys 0.000s
W20260709 13:18:16.776367 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 65124us
I20260709 13:18:16.632617 14401 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:16.719545 14322 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:16.774044 12779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 1692) took 1293 ms. Trace:
I20260709 13:18:16.787252 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:16.774842 14096 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:16.794883 14288 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:16.795713 14399 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:16.796063 14448 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:16.796675 12704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 135) took 1297 ms. Trace:
I20260709 13:18:16.796886 12634 raft_consensus.cc:1240] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:16.776959 14362 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:16.803205 12747 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:16.815500 12704 rpcz_store.cc:276] 0709 13:18:15.498819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.743951 (+245132us) service_pool.cc:224] Handling call
0709 13:18:15.762802 (+ 18851us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:15.772713 (+  9911us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:15.772742 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:15.773244 (+   502us) log.cc:844] Serialized 52 byte log entry
0709 13:18:16.261247 (+488003us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:16.261308 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.261410 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:16.796445 (+535035us) raft_consensus.cc:1685] finished
0709 13:18:16.796466 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.796608 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2284,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":6123,"thread_start_us":487670,"threads_started":1,"wal-append.queue_time_us":766724}
I20260709 13:18:16.713053 14166 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:16.805011 12780 raft_consensus.cc:1240] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 4. Current term is 5. Ops: []
W20260709 13:18:16.793249 14431 log.cc:927] Time spent T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:18:16.798733 14441 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.799831 14126 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:16.803830 14381 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:16.946029 12673 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.65:43603 timed out after 1.530s (SENT) [suppressed 20 similar messages]
I20260709 13:18:16.946945 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:16.723706 14408 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:17.002135 12630 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:16.775818 14352 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:16.981594 14347 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.014s	sys 0.019s
I20260709 13:18:16.777859 14394 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:16.815178 14322 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:18:17.086061 12672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.828s	user 0.018s	sys 0.019s
W20260709 13:18:17.088703 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.095819 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:17.096745 12672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.018s	sys 0.019s
I20260709 13:18:16.765439 14426 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:17.097780 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 853699us
W20260709 13:18:16.989650 14374 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.009s	sys 0.023s
I20260709 13:18:17.105397 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:17.095216 14347 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.015s	sys 0.020s
I20260709 13:18:17.111758 14352 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:17.095700 14322 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:16.983932 14377 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:17.113746 14428 consensus_queue.cc:1059] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.119s
W20260709 13:18:17.128396 14445 log.cc:927] Time spent T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.343s	user 0.001s	sys 0.000s
I20260709 13:18:17.128909 14422 consensus_queue.cc:1059] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:18:17.129964 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:17.138104 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:17.139648 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:17.123016 14448 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:17.150532 14292 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:17.093115 12627 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:17.163089 14288 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:17.165109 14390 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:17.171619 14442 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:17.175494 14401 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:17.177832 14360 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:17.210906 12628 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.012s	user 0.024s	sys 0.019s
W20260709 13:18:17.211370 12628 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.012s	user 0.024s	sys 0.019s
I20260709 13:18:17.179615 14394 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:17.091917 14362 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:18:17.118762 14374 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.836s	user 0.009s	sys 0.026s
I20260709 13:18:17.188364 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:17.193941 12706 raft_consensus.cc:2376] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3. Re-sending same reply.
I20260709 13:18:17.007275 14342 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:17.195617 12707 raft_consensus.cc:2410] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:18:16.981590 14375 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.007879 14278 consensus_queue.cc:1059] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.177s
I20260709 13:18:17.169006 14322 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:17.205476 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:17.268747 12704 raft_consensus.cc:2368] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 4. Current term is 5.
W20260709 13:18:17.191552 14183 consensus_meta.cc:287] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.683s	user 0.003s	sys 0.007s
W20260709 13:18:17.229296 12703 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.771s	user 0.023s	sys 0.046s
W20260709 13:18:17.269956 12703 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.023s	sys 0.046s
I20260709 13:18:17.210449 14200 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:17.232211 12782 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.010s	sys 0.022s
I20260709 13:18:17.230015 12672 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:17.008694 14441 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:17.312038 14441 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:17.105913 14381 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:17.314603 14441 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:17.262018 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:17.315193 14464 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:17.272580 14342 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:17.200337 12700 raft_consensus.cc:2376] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2. Re-sending same reply.
I20260709 13:18:17.316213 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:17.316666 12701 consensus_meta.cc:287] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.806s	user 0.007s	sys 0.003s
I20260709 13:18:17.316962 12707 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:17.268749 12776 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.017s	sys 0.028s
W20260709 13:18:17.317739 12776 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.889s	user 0.017s	sys 0.028s
I20260709 13:18:17.318390 14399 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:17.318542 12776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1472) took 1056 ms. Trace:
W20260709 13:18:17.318701 14273 consensus_meta.cc:287] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 1.071s	user 0.006s	sys 0.006s
I20260709 13:18:17.319461 14219 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:17.319001 12776 rpcz_store.cc:276] 0709 13:18:16.262361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.375684 (+113323us) service_pool.cc:224] Handling call
0709 13:18:16.376074 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.376325 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:18:16.428015 (+ 51690us) log.cc:844] Serialized 14 byte log entry
0709 13:18:17.318196 (+890181us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:17.318253 (+    57us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:17.318295 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.318370 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.318485 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29375,"prepare.queue_time_us":79016,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":82,"thread_start_us":889810,"threads_started":1}
I20260709 13:18:17.320748 14317 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:18:17.322990 14365 consensus_meta.cc:287] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.807s	user 0.009s	sys 0.001s
I20260709 13:18:17.323356 14317 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:17.323405 14365 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:17.324026 14317 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:17.324381 12707 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
W20260709 13:18:17.278743 12709 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.137s	user 0.003s	sys 0.033s
W20260709 13:18:17.239117 13381 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.947s	user 0.037s	sys 0.045s
I20260709 13:18:17.231691 14288 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:17.297418 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 68459.5us
W20260709 13:18:17.329717 13381 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.038s	user 0.038s	sys 0.046s
I20260709 13:18:17.314725 14381 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:17.315999 14464 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.331295 14464 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:17.330987 14441 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:17.332214 14264 consensus_meta.cc:287] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.543s	user 0.008s	sys 0.000s
I20260709 13:18:17.317761 14342 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:17.333729 14441 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:17.334458 12672 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:17.271525 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:17.309088 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:17.308943 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:17.314704 14426 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:17.297124 14362 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:17.317166 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:17.306896 14386 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.802s	user 0.022s	sys 0.092s
W20260709 13:18:17.334754 14207 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.161s	user 0.025s	sys 0.037s
W20260709 13:18:17.337606 12702 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.002s	sys 0.018s
W20260709 13:18:17.298291 12782 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.010s	sys 0.022s
W20260709 13:18:17.261217 12708 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.803s	user 0.012s	sys 0.023s
W20260709 13:18:17.366788 12708 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.909s	user 0.012s	sys 0.023s
W20260709 13:18:17.305167 12597 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.960s	user 0.014s	sys 0.033s
W20260709 13:18:17.367446 12597 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.023s	user 0.015s	sys 0.034s
W20260709 13:18:17.367884 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 1.02327e+06us
I20260709 13:18:17.346181 12705 raft_consensus.cc:1275] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:18:17.347110 12703 raft_consensus.cc:2393] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 3: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:17.414888 12705 pending_rounds.cc:85] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 2
I20260709 13:18:17.415139 12705 pending_rounds.cc:107] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:18:17.415372 12705 raft_consensus.cc:2889] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:17.415604 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:17.416932 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:17.417589 12703 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:17.325397 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:17.354187 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:17.419741 12774 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.974s	user 0.014s	sys 0.035s
W20260709 13:18:17.273203 14398 consensus_meta.cc:287] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 1.527s	user 0.014s	sys 0.000s
I20260709 13:18:17.317335 12701 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:18:17.356935 12747 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.019s	sys 0.065s
I20260709 13:18:16.800799 12779 rpcz_store.cc:276] 0709 13:18:15.480801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.989523 (+508722us) service_pool.cc:224] Handling call
0709 13:18:16.079424 (+ 89901us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:16.106283 (+ 26859us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:16.108528 (+  2245us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:16.147579 (+ 39051us) log.cc:844] Serialized 26 byte log entry
0709 13:18:16.423585 (+276006us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:16.423662 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.463001 (+ 39339us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:16.773663 (+310662us) raft_consensus.cc:1685] finished
0709 13:18:16.773707 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.773950 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4347,"raft.queue_time_us":6800,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":5390,"thread_start_us":243668,"threads_started":1,"wal-append.queue_time_us":610731}
I20260709 13:18:17.356204 14126 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:17.328774 14096 consensus_meta.cc:287] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.509s	user 0.005s	sys 0.001s
I20260709 13:18:17.263723 14416 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:17.360170 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:17.208606 14292 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:17.325505 14427 consensus_meta.cc:287] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.781s	user 0.005s	sys 0.001s
I20260709 13:18:17.346585 14342 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:17.446811 12775 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.011s	sys 0.006s
W20260709 13:18:17.374158 12702 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.830s	user 0.003s	sys 0.018s
W20260709 13:18:17.334738 12709 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.193s	user 0.004s	sys 0.034s
W20260709 13:18:17.447602 12775 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.011s	sys 0.006s
I20260709 13:18:17.204834 14448 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:17.420089 12774 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.974s	user 0.014s	sys 0.035s
I20260709 13:18:17.419742 12707 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 4: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:17.458449 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:17.460858 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:17.428391 12630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 967) took 1867 ms. Trace:
W20260709 13:18:17.363981 14386 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.856s	user 0.022s	sys 0.092s
W20260709 13:18:17.364514 14207 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.219s	user 0.025s	sys 0.037s
W20260709 13:18:17.334607 14412 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.029s	sys 0.085s
I20260709 13:18:17.334201 14288 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:17.429349 12633 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
W20260709 13:18:17.429852 12747 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.020s	sys 0.065s
W20260709 13:18:17.430018 13348 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.015s	sys 0.015s
I20260709 13:18:17.439805 14126 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:17.270421 14375 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:17.454111 14427 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:17.346073 12626 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:17.448207 12709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1014) took 2065 ms. Trace:
I20260709 13:18:17.524186 12702 raft_consensus.cc:2376] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3. Re-sending same reply.
I20260709 13:18:17.448573 12775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1476) took 1185 ms. Trace:
I20260709 13:18:17.412436 14381 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:17.520260 14412 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.020s	user 0.030s	sys 0.088s
I20260709 13:18:17.540530 12704 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:17.536551 12709 rpcz_store.cc:276] 0709 13:18:15.382764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.723943 (+341179us) service_pool.cc:224] Handling call
0709 13:18:15.724363 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.724566 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:18:16.071384 (+346818us) log.cc:844] Serialized 14 byte log entry
0709 13:18:17.447851 (+1376467us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:17.447907 (+    56us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:17.447946 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.448023 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.448148 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":109762,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":31874,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":150,"spinlock_wait_cycles":14592,"thread_start_us":1630511,"threads_started":2}
I20260709 13:18:17.542228 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:17.543146 12709 raft_consensus.cc:2368] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 4.
I20260709 13:18:17.412919 12597 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:17.417922 14365 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:17.418381 14395 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.419406 14183 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:17.474882 12631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 150) took 2647 ms. Trace:
I20260709 13:18:17.493430 13381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 359) took 1206 ms. Trace:
I20260709 13:18:17.477545 12706 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:17.519500 12633 pending_rounds.cc:85] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 5
W20260709 13:18:17.488570 13384 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.163s	user 0.029s	sys 0.058s
I20260709 13:18:17.547099 12631 rpcz_store.cc:276] 0709 13:18:14.815850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.816664 (+   814us) service_pool.cc:224] Handling call
0709 13:18:15.378631 (+561967us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:15.401460 (+ 22829us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:15.401492 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:15.407571 (+  6079us) log.cc:844] Serialized 26 byte log entry
0709 13:18:15.963387 (+555816us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b180024b920. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf520d1 00007f4c6d36da0f
0709 13:18:16.168108 (+204721us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:16.168168 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.168287 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:16.446934 (+278647us) raft_consensus.cc:1685] finished
0709 13:18:16.446952 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.463496 (+1016544us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b5400049cd0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:17.463662 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22416,"raft.queue_time_us":28135,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":38693,"spinlock_wait_cycles":1271044352,"thread_start_us":204508,"threads_started":1,"wal-append.queue_time_us":852884,"wal-append.run_cpu_time_us":2067,"wal-append.run_wall_time_us":1111835}
W20260709 13:18:17.553783 13384 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.228s	user 0.029s	sys 0.060s
I20260709 13:18:17.553335 12633 pending_rounds.cc:107] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:18:17.554957 12633 raft_consensus.cc:2889] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:17.505084 12746 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.011s	sys 0.012s
I20260709 13:18:17.237048 14442 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:17.520272 13348 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.826s	user 0.015s	sys 0.025s
I20260709 13:18:17.557396 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:18:17.557683 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 863771us
I20260709 13:18:17.535200 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:17.523180 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:17.468842 14363 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.343s	user 0.029s	sys 0.076s
W20260709 13:18:17.533587 12632 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.009s	sys 0.019s
W20260709 13:18:17.519855 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 698704us
I20260709 13:18:17.444047 14096 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:17.565793 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:17.570319 13382 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
W20260709 13:18:17.538486 12629 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.828s	user 0.013s	sys 0.019s
W20260709 13:18:17.571859 12629 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.013s	sys 0.019s
I20260709 13:18:17.538053 12775 rpcz_store.cc:276] 0709 13:18:16.263521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.634033 (+370512us) service_pool.cc:224] Handling call
0709 13:18:16.671946 (+ 37913us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.672242 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:18:16.706162 (+ 33920us) log.cc:844] Serialized 14 byte log entry
0709 13:18:17.448256 (+742094us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:17.448305 (+    49us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:17.448350 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.448421 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.448525 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":35443,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":4693,"raft.queue_time_us":93,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":12176,"thread_start_us":741532,"threads_started":1}
W20260709 13:18:17.474355 13351 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.774s	user 0.017s	sys 0.018s
I20260709 13:18:17.519680 12630 rpcz_store.cc:276] 0709 13:18:15.550860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.551000 (+   140us) service_pool.cc:224] Handling call
0709 13:18:15.551238 (+   238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.089461 (+1538223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.418402 (+328941us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b540027ae54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:17.418583 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3336240512}
I20260709 13:18:17.538229 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:17.474884 12701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1033) took 1013 ms. Trace:
W20260709 13:18:17.545855 12674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.018s	sys 0.018s
W20260709 13:18:17.522505 14349 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.940s	user 0.011s	sys 0.025s
I20260709 13:18:17.555526 14398 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:18:17.549823 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 137022us
I20260709 13:18:17.579708 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:17.548753 14334 consensus_meta.cc:287] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.757s	user 0.007s	sys 0.010s
W20260709 13:18:17.555888 12746 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.013s	sys 0.013s
I20260709 13:18:17.596194 12631 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:17.564038 14390 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.567094 12747 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:17.566316 12632 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.010s	sys 0.019s
I20260709 13:18:17.581897 12780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1478) took 1317 ms. Trace:
W20260709 13:18:17.446408 14410 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.898s	user 0.002s	sys 0.016s
W20260709 13:18:17.587190 12674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.021s	sys 0.020s
W20260709 13:18:17.585341 13351 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.885s	user 0.018s	sys 0.023s
I20260709 13:18:17.587399 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:17.590579 12597 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:17.547569 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:17.623240 12709 raft_consensus.cc:2376] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3. Re-sending same reply.
W20260709 13:18:17.566288 14363 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.441s	user 0.030s	sys 0.076s
I20260709 13:18:17.592130 12702 raft_consensus.cc:2368] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 4.
I20260709 13:18:17.489509 12703 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
W20260709 13:18:17.580446 14282 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.011s	sys 0.018s
I20260709 13:18:17.598932 14426 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.599968 14288 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:17.600731 14416 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.585402 12701 rpcz_store.cc:276] 0709 13:18:16.461554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.476587 (+ 15033us) service_pool.cc:224] Handling call
0709 13:18:16.476984 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.440682 (+963698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.467616 (+ 26934us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:17.586328 14349 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.004s	user 0.012s	sys 0.025s
I20260709 13:18:17.547091 14381 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:17.612111 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 45264.9us
I20260709 13:18:17.612686 12780 rpcz_store.cc:276] 0709 13:18:16.264033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.725166 (+461133us) service_pool.cc:224] Handling call
0709 13:18:16.785182 (+ 60016us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.822194 (+ 37012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.581638 (+759444us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b5400089154. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:17.581838 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45111040}
I20260709 13:18:17.616246 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:17.616914 12627 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:17.620965 12704 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:17.554777 14183 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:17.571257 14352 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:17.591992 14398 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.685842 14207 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:17.614297 14410 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.072s	user 0.003s	sys 0.016s
I20260709 13:18:17.555676 14395 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:17.552891 13381 rpcz_store.cc:276] 0709 13:18:16.287052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.289904 (+  2852us) service_pool.cc:224] Handling call
0709 13:18:16.290649 (+   745us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:16.291175 (+   526us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:16.291210 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:16.291777 (+   567us) log.cc:844] Serialized 26 byte log entry
0709 13:18:17.330582 (+1038805us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:17.330634 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.330718 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:17.493095 (+162377us) raft_consensus.cc:1685] finished
0709 13:18:17.493138 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.493346 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1207,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":118,"thread_start_us":1038195,"threads_started":1,"wal-append.queue_time_us":1187493}
I20260709 13:18:17.622198 12630 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:17.612238 14478 raft_consensus.cc:3055] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:17.574389 12700 raft_consensus.cc:2376] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3. Re-sending same reply.
I20260709 13:18:17.683429 14322 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:17.612989 14334 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:17.632205 14282 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.838s	user 0.012s	sys 0.019s
I20260709 13:18:17.622118 14278 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:17.692976 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:17.656127 14400 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:17.613440 14386 consensus_queue.cc:1059] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.084s
I20260709 13:18:17.696394 12706 raft_consensus.cc:2410] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:17.462973 14342 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:17.662142 14399 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.700202 12597 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:17.703140 12774 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1473) took 1440 ms. Trace:
I20260709 13:18:17.843891 12774 rpcz_store.cc:276] 0709 13:18:16.262666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.400496 (+137830us) service_pool.cc:224] Handling call
0709 13:18:16.420026 (+ 19530us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.420422 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:18:17.455830 (+1035408us) log.cc:844] Serialized 14 byte log entry
0709 13:18:17.702833 (+247003us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:17.702887 (+    54us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:17.702928 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.702991 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.703089 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68470,"prepare.queue_time_us":39122,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":11451,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":5544,"thread_start_us":1244806,"threads_started":2}
I20260709 13:18:17.679150 14362 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:17.691638 14299 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.028s	sys 0.027s
I20260709 13:18:17.665030 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:17.626979 14390 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:17.685709 14460 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:17.792747 12708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1029) took 1338 ms. Trace:
W20260709 13:18:17.766996 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 52387.3us
I20260709 13:18:17.764727 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:17.691263 14455 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:17.679334 14426 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:17.699506 14373 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.151s	user 0.017s	sys 0.051s
I20260709 13:18:17.773839 13348 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:17.798527 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:17.688906 14416 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:17.805356 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:17.710366 14398 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:17.910796 14469 log.cc:927] Time spent T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.224s	user 0.000s	sys 0.000s
I20260709 13:18:17.911459 14110 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:17.910928 14472 consensus_queue.cc:1059] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
W20260709 13:18:17.764201 14376 consensus_meta.cc:287] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 1.249s	user 0.011s	sys 0.001s
I20260709 13:18:17.913823 14376 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:17.915475 14479 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.916290 12630 raft_consensus.cc:2393] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 2: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:17.916101 14479 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:17.917227 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:17.918372 14479 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:17.918517 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:17.919688 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:17.832906 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:17.789692 14349 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:17.809864 14292 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:17.874404 12673 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.012s	sys 0.020s
W20260709 13:18:17.941817 12673 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.013s	sys 0.021s
W20260709 13:18:17.942129 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 996077us
I20260709 13:18:17.793956 14374 raft_consensus.cc:3055] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:17.875298 12700 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:18:17.799554 14478 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:17.940582 13384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 301) took 1619 ms. Trace:
I20260709 13:18:17.712632 14412 consensus_queue.cc:1059] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.105s
I20260709 13:18:17.808403 14278 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.884280 13382 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:17.710069 14352 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:18.058292 14352 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:17.717334 14207 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:18.060339 14207 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:17.893064 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:18.061187 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 168173us
I20260709 13:18:18.061563 14110 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:18.062124 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:18.062325 14110 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:18.062780 14110 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:18.063009 12747 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:18.063657 13382 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:17.892488 12708 rpcz_store.cc:276] 0709 13:18:16.454595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.455631 (+  1036us) service_pool.cc:224] Handling call
0709 13:18:16.455876 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.456062 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:18:17.412207 (+956145us) log.cc:844] Serialized 14 byte log entry
0709 13:18:17.792312 (+380105us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:17.792381 (+    69us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:17.792438 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.792520 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.792674 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29204,"prepare.queue_time_us":5711,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":24,"raft.queue_time_us":32324,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":29637,"thread_start_us":1262126,"threads_started":2}
I20260709 13:18:18.064657 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:18.065440 12708 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:17.894860 14273 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:18.066610 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:18.067401 12708 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:17.910579 14426 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:17.915702 14373 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.375s	user 0.017s	sys 0.053s
W20260709 13:18:17.921211 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 147512us
I20260709 13:18:18.069191 13348 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:17.923074 14416 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:17.713389 14347 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.023s	sys 0.009s
W20260709 13:18:17.874595 14299 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.029s	sys 0.040s
I20260709 13:18:17.713240 14096 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:17.928803 12630 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:17.886224 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:18.094008 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:17.791623 14026 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:18:17.940093 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 239944us
I20260709 13:18:18.001545 14412 consensus_queue.cc:1243] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:18:17.940945 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:18.097317 12627 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:17.947360 13384 rpcz_store.cc:276] 0709 13:18:16.321038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.323803 (+  2765us) service_pool.cc:224] Handling call
0709 13:18:16.324281 (+   478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:16.324854 (+   573us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:16.324899 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:16.325578 (+   679us) log.cc:844] Serialized 26 byte log entry
0709 13:18:17.554369 (+1228791us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:17.563543 (+  9174us) log.cc:844] Serialized 14 byte log entry
0709 13:18:17.564337 (+   794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.564422 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:17.940348 (+375926us) raft_consensus.cc:1685] finished
0709 13:18:17.940368 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.940521 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":530,"prepare.queue_time_us":9325,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":29247,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":29225,"thread_start_us":1228319,"threads_started":1,"wal-append.queue_time_us":1487182}
I20260709 13:18:17.893371 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:17.959455 12673 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:18.117336 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 157958us
I20260709 13:18:18.118042 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:18.118109 12673 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:17.876192 14399 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:17.941959 14465 log.cc:927] Time spent T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.395s	user 0.001s	sys 0.001s
I20260709 13:18:17.949074 14410 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:17.940614 14394 raft_consensus.cc:3055] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:18.127424 14394 raft_consensus.cc:740] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:18.127434 14288 consensus_queue.cc:1059] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:18.128482 14394 consensus_queue.cc:260] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:18.132782 14408 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:18.124619 14373 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:17.959820 14336 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:17.913125 14472 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:17.922186 14395 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:18.066363 14273 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:18.142181 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:18.074358 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:18.114939 12701 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:17.945428 14374 raft_consensus.cc:740] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:17.957908 14145 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.085100 12708 raft_consensus.cc:1240] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:17.987010 14278 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:18.036775 14363 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:18.170384 14363 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:18.170872 14363 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:18.171376 12782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1474) took 1908 ms. Trace:
I20260709 13:18:18.172204 12708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1378) took 1019 ms. Trace:
I20260709 13:18:18.172709 12708 rpcz_store.cc:276] 0709 13:18:17.153126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.084622 (+931496us) service_pool.cc:224] Handling call
0709 13:18:18.084927 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:18.171965 (+ 87038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.172152 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:18.090834 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:17.930853 14455 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:18.177963 12630 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:18.092530 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:18.180575 12627 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:17.946494 14349 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:17.946725 14442 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:18.134507 14381 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:18.084390 14480 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:18.070585 14360 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:18.086781 14347 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.890s	user 0.023s	sys 0.010s
I20260709 13:18:18.010262 14479 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:17.949671 14400 raft_consensus.cc:697] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:18.083145 14352 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:17.901909 14460 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:18.082695 14429 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:18.123670 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:18.122174 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:18.127534 14441 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:17.950335 14375 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:18.126924 12673 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:18.127487 14477 log.cc:927] Time spent T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 13:18:18.153620 12632 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 2.
I20260709 13:18:18.240029 12632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56838 (request call id 995) took 1241 ms. Trace:
I20260709 13:18:18.240381 12632 rpcz_store.cc:276] 0709 13:18:16.999066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.014103 (+1015037us) service_pool.cc:224] Handling call
0709 13:18:18.239957 (+225854us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:18.154587 12597 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:18:18.241617 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 87123.4us
I20260709 13:18:18.190601 14480 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:18.114362 14292 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:18.143452 14336 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:18.168900 14496 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:18.169622 14273 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:18.169783 14473 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:18.235545 14321 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:18.077144 14489 raft_consensus.cc:3055] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:18.186681 12630 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:18.241814 14373 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:18.146104 14394 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:18.134159 14386 consensus_queue.cc:1059] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
W20260709 13:18:18.195231 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 68563.1us
I20260709 13:18:18.180049 14455 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:18.236286 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 109864us
I20260709 13:18:18.293998 12673 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:18.233455 12781 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 1.
I20260709 13:18:18.189033 14322 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:18.193544 14334 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:18.079536 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 183772us
I20260709 13:18:18.124011 14026 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:18.125589 14478 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:18.251237 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:18.289776 14154 consensus_meta.cc:287] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 1.825s	user 0.010s	sys 0.007s
I20260709 13:18:18.171782 12782 rpcz_store.cc:276] 0709 13:18:16.262966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.483800 (+220834us) service_pool.cc:224] Handling call
0709 13:18:16.484235 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:16.484629 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:16.484657 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:16.485102 (+   445us) log.cc:844] Serialized 26 byte log entry
0709 13:18:17.366076 (+880974us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:17.366139 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.366241 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:18.171155 (+804914us) raft_consensus.cc:1685] finished
0709 13:18:18.171179 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:18.171317 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":82,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":113,"thread_start_us":880216,"threads_started":1,"wal-append.queue_time_us":979847}
I20260709 13:18:18.237821 14342 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:18.235620 14441 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:18.309780 14441 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:18.310031 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:18.310827 12705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1355) took 3160 ms. Trace:
I20260709 13:18:18.311503 12634 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:18.311380 12705 rpcz_store.cc:276] 0709 13:18:15.150333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.378155 (+227822us) service_pool.cc:224] Handling call
0709 13:18:15.378439 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.415791 (+2037352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.310584 (+894793us) spinlock_profiling.cc:243] Waited 1.95 s on lock 0x7b540014bed4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:18.310749 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4494309120}
I20260709 13:18:18.313283 12705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1384) took 1152 ms. Trace:
I20260709 13:18:18.258993 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:18.189903 14145 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:18.243908 14408 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:18.201524 14360 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:18.246932 14480 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:18.244280 14429 raft_consensus.cc:697] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:18.231323 14479 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:18.256317 14487 log.cc:927] Time spent T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 13:18:18.297343 13349 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:18.301334 12673 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:18.331955 12673 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:18.332053 13349 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:18.257908 14489 raft_consensus.cc:740] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:18.302965 14207 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:18.305792 14412 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:18.308713 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:18.309312 14482 log.cc:927] Time spent T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 13:18:18.139361 14183 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:18.308182 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:18.265619 14390 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:18.252506 14292 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:18.309209 14026 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:18.316020 12630 raft_consensus.cc:2393] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:18.315647 12705 rpcz_store.cc:276] 0709 13:18:17.160511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.312541 (+1152030us) service_pool.cc:224] Handling call
0709 13:18:18.312810 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:18.312993 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:18.313018 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:18.313058 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:18.313094 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.313145 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:18.313234 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:18.321265 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.328035 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:18.308750 14497 log.cc:927] Time spent T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
W20260709 13:18:18.316780 14485 log.cc:927] Time spent T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
W20260709 13:18:18.360663 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 72624us
I20260709 13:18:18.329077 14408 raft_consensus.cc:697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:18.304844 14399 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:18.333498 12673 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:18.305563 14299 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:18.363359 14299 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:18.363950 13348 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:18.323385 14478 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:18.260839 14473 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:18.367939 14473 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:18.340916 14484 log.cc:927] Time spent T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
I20260709 13:18:18.341277 14429 consensus_queue.cc:237] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:18.343529 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1374) took 1199 ms. Trace:
I20260709 13:18:18.344873 12628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 981) took 2701 ms. Trace:
I20260709 13:18:18.293613 14400 consensus_queue.cc:237] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:18.339176 14288 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:18.354333 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:18.355180 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:18.356034 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:18.245942 14374 consensus_queue.cc:260] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:18.324918 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:18.315164 14154 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:18.370908 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:18.380859 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:18.369971 12702 rpcz_store.cc:276] 0709 13:18:17.144422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.665890 (+521468us) service_pool.cc:224] Handling call
0709 13:18:17.666253 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:17.666697 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.666730 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:17.687886 (+ 21156us) log.cc:844] Serialized 26 byte log entry
0709 13:18:17.986189 (+298303us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:17.986234 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.986311 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:18.343242 (+356931us) raft_consensus.cc:1685] finished
0709 13:18:18.343265 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:18.343459 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3175,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":19259,"thread_start_us":219488,"threads_started":1,"wal-append.queue_time_us":539635}
W20260709 13:18:18.357489 14495 log.cc:927] Time spent T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:18:18.338498 14375 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:18.374056 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:18:18.378986 14493 log.cc:927] Time spent T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:18:18.378871 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:18.394305 12777 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:18.379441 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:18.395421 12781 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:18.370563 12628 rpcz_store.cc:276] 0709 13:18:15.643615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.198318 (+554703us) service_pool.cc:224] Handling call
0709 13:18:16.198687 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:17.229004 (+1030317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.229041 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:17.353318 (+124277us) log.cc:844] Serialized 26 byte log entry
0709 13:18:17.713660 (+360342us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:17.958001 (+244341us) log.cc:844] Serialized 14 byte log entry
0709 13:18:17.958877 (+   876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.958974 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:18.343026 (+384052us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b18002dd900. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c53f3 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:18:18.344594 (+  1568us) raft_consensus.cc:1685] finished
0709 13:18:18.344688 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:18.344817 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13886,"prepare.queue_time_us":476630,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":225,"raft.queue_time_us":203272,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":16700,"spinlock_wait_cycles":100167296,"thread_start_us":1436170,"threads_started":3,"wal-append.queue_time_us":560942}
I20260709 13:18:18.384862 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:18.396903 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:18.397527 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:18.379535 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:18.403038 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 71042.9us
I20260709 13:18:18.392652 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:18.405026 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:18.405833 13382 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 1.
I20260709 13:18:18.407493 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:18.408039 13382 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:18.409729 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:18.410323 12781 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:18.411219 12630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56832 (request call id 1751) took 1134 ms. Trace:
I20260709 13:18:18.411543 12630 rpcz_store.cc:276] 0709 13:18:17.276380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.392399 (+1116019us) service_pool.cc:224] Handling call
0709 13:18:18.411164 (+ 18765us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:18.411972 12628 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:18.412536 12628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 996) took 1413 ms. Trace:
I20260709 13:18:18.412694 12630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 1753) took 1135 ms. Trace:
I20260709 13:18:18.412889 12628 rpcz_store.cc:276] 0709 13:18:16.999300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.411388 (+1412088us) service_pool.cc:224] Handling call
0709 13:18:18.411668 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:18.412318 (+   650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.412487 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:18.370357 14390 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:18.413209 12630 rpcz_store.cc:276] 0709 13:18:17.277078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.412056 (+1134978us) service_pool.cc:224] Handling call
0709 13:18:18.412272 (+   216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:18.412421 (+   149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:18.412476 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:18.412512 (+    36us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:18.412539 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.412589 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:18.412656 (+    67us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:18.380071 14347 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:18.402593 12701 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:18.416070 12628 raft_consensus.cc:1240] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:18:18.348732 14207 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:18.426707 12705 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:18.427492 12628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 1757) took 1092 ms. Trace:
I20260709 13:18:18.427518 12701 raft_consensus.cc:1240] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:18:18.432169 12628 rpcz_store.cc:276] 0709 13:18:17.335464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.414166 (+1078702us) service_pool.cc:224] Handling call
0709 13:18:18.414526 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:18.427251 (+ 12725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.427443 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:18.417793 12779 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:18.409288 14473 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:18.433748 12701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 144) took 1238 ms. Trace:
I20260709 13:18:18.433826 14473 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:18.434322 14473 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:18.434140 12701 rpcz_store.cc:276] 0709 13:18:17.197412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.426846 (+1229434us) service_pool.cc:224] Handling call
0709 13:18:18.427259 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:18.432683 (+  5424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.433693 (+  1010us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:18.430590 13382 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:18.410415 14374 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:18.392912 14375 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:18.388491 14349 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:18.437680 12775 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:18.450898 12672 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:18.441921 14498 log.cc:927] Time spent T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:18:18.441778 14390 raft_consensus.cc:697] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:18.442775 14367 raft_consensus.cc:3055] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:18.452417 12777 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:18.348775 14412 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:18.405545 13380 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:18.419193 14299 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:18.468930 12701 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:18.445106 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:18.421432 14347 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:18.430962 14288 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:18.465124 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:18.434151 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:18.435796 14373 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:18.485152 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:18.489686 12700 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
W20260709 13:18:18.494272 14501 log.cc:927] Time spent T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:18:18.494855 12597 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:18.455825 14375 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:18.466377 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:18.476611 14471 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.210s	user 0.001s	sys 0.001s
W20260709 13:18:18.470537 14398 consensus_meta.cc:287] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.557s	user 0.010s	sys 0.001s
I20260709 13:18:18.483352 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:18.488557 14508 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.494668 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:18.497331 12708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1380) took 1338 ms. Trace:
I20260709 13:18:18.505313 12708 rpcz_store.cc:276] 0709 13:18:17.159238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.174077 (+1014839us) service_pool.cc:224] Handling call
0709 13:18:18.174433 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:18.174751 (+   318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:18.174783 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:18.175284 (+   501us) log.cc:844] Serialized 26 byte log entry
0709 13:18:18.289800 (+114516us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:18.289853 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.289967 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:18.497124 (+207157us) raft_consensus.cc:1685] finished
0709 13:18:18.497142 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:18.497277 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6421,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":7006,"thread_start_us":114118,"threads_started":1,"wal-append.queue_time_us":176652}
I20260709 13:18:18.507416 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:18.507810 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:18.508008 12708 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:18.508809 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:18.458530 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 125087us
I20260709 13:18:18.471575 14367 raft_consensus.cc:740] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:18.502879 12703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1373) took 1359 ms. Trace:
I20260709 13:18:18.480280 14347 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:18.479815 14299 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.464668 14460 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:18.499289 14317 consensus_meta.cc:287] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 1.175s	user 0.011s	sys 0.003s
W20260709 13:18:18.513664 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.480019 13377 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:18.512298 12703 rpcz_store.cc:276] 0709 13:18:17.143969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.634304 (+490335us) service_pool.cc:224] Handling call
0709 13:18:17.634716 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:17.694560 (+ 59844us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.694609 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:17.695125 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 13:18:17.984564 (+289439us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:17.984625 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.984723 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:18.502660 (+517937us) raft_consensus.cc:1685] finished
0709 13:18:18.502680 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:18.502821 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":102,"spinlock_wait_cycles":9094272,"thread_start_us":194500,"threads_started":1,"wal-append.queue_time_us":488020}
W20260709 13:18:18.497618 14502 log.cc:927] Time spent T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:18:18.510716 12673 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:18.516607 12701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 146) took 1310 ms. Trace:
I20260709 13:18:18.536181 12701 rpcz_store.cc:276] 0709 13:18:17.206063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.435368 (+1229305us) service_pool.cc:224] Handling call
0709 13:18:18.468794 (+ 33426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:18.516160 (+ 47366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:18.516207 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:18.516264 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:18.516312 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.516396 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:18.516537 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":91,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":3448}
I20260709 13:18:18.497591 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:18.522894 14507 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:18.540169 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:18.543334 12628 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:18.460022 14408 consensus_queue.cc:237] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:18.542631 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.545498 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:18.546202 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 93865.1us
I20260709 13:18:18.547430 12628 raft_consensus.cc:2393] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 2: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:18.514015 14401 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:18.548568 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:18.552830 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:18.553460 12628 raft_consensus.cc:2376] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4. Re-sending same reply.
I20260709 13:18:18.553957 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 143) took 1361 ms. Trace:
I20260709 13:18:18.554504 12702 rpcz_store.cc:276] 0709 13:18:17.194955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.381983 (+1187028us) service_pool.cc:224] Handling call
0709 13:18:18.382367 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:18.474450 (+ 92083us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:18.474501 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:18.482077 (+  7576us) log.cc:844] Serialized 26 byte log entry
0709 13:18:18.512612 (+ 30535us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:18.512659 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.512744 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:18.553664 (+ 40920us) raft_consensus.cc:1685] finished
0709 13:18:18.553696 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:18.553885 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":79729,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":145,"thread_start_us":117464,"threads_started":2,"wal-append.queue_time_us":69942}
I20260709 13:18:18.521229 14479 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:18.547365 14464 consensus_meta.cc:287] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 1.212s	user 0.007s	sys 0.006s
I20260709 13:18:18.507081 14398 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:18.546319 14288 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:18.563050 14145 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:18.566226 14110 consensus_meta.cc:287] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.503s	user 0.008s	sys 0.003s
I20260709 13:18:18.570573 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:18.550621 14401 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:18.574033 12628 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:18.543362 14317 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:18.568629 14292 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:18.584551 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:18.579341 14110 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:18.586819 14110 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:18.567283 14464 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:18.588698 12709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1371) took 1445 ms. Trace:
I20260709 13:18:18.589812 14464 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:18.589087 12709 rpcz_store.cc:276] 0709 13:18:17.143047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.634006 (+490959us) service_pool.cc:224] Handling call
0709 13:18:17.805246 (+171240us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:17.805709 (+   463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.805749 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:17.842716 (+ 36967us) log.cc:844] Serialized 26 byte log entry
0709 13:18:18.084074 (+241358us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:18.084110 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.084170 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:18.519348 (+435178us) raft_consensus.cc:1685] finished
0709 13:18:18.519367 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:18.588464 (+ 69097us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b54002481d0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:18.588637 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":610,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":25129,"spinlock_wait_cycles":392762112,"thread_start_us":166152,"threads_started":1,"wal-append.queue_time_us":264200}
I20260709 13:18:18.557992 14507 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:18.590905 14507 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:18.592278 12632 raft_consensus.cc:2393] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:18.576246 14401 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:18.541448 14299 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:18.542357 14363 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:18.610391 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:18.577740 12779 raft_consensus.cc:2410] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:18:18.569236 14288 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:18:18.634455 12706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1376) took 1489 ms (client timeout 1587 ms). Trace:
I20260709 13:18:18.635237 12707 raft_consensus.cc:1240] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:18.635798 14361 raft_consensus.cc:3055] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:18.544077 14390 consensus_queue.cc:237] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:18.540840 14508 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:18.700830 12598 connection.cc:256] RPC call timeout handler was delayed by -1.4609844970000267s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1587s for a response
I20260709 13:18:18.665912 12777 raft_consensus.cc:2393] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:18:18.709182 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 134821us
W20260709 13:18:18.709879 12706 rpcz_store.cc:269] 0709 13:18:17.145083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.235437 (+ 90354us) service_pool.cc:224] Handling call
0709 13:18:18.634241 (+1398804us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54002490d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:18.634394 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":324764032}
I20260709 13:18:18.637223 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:18.703233 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:18.712423 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:18.637744 12634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56832 (request call id 1755) took 1360 ms. Trace:
I20260709 13:18:18.713598 12672 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
W20260709 13:18:18.700644 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 107426us
I20260709 13:18:18.584591 14317 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:18.722774 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:18.713805 12634 rpcz_store.cc:276] 0709 13:18:17.277753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.712783 (+435030us) service_pool.cc:224] Handling call
0709 13:18:18.637533 (+924750us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b54000df554. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:18.637680 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":892036992}
I20260709 13:18:18.721436 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:18.716157 12776 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:18.725164 12628 raft_consensus.cc:2410] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:18:18.722018 13349 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:18.722417 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:18.697994 14509 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.179s	user 0.000s	sys 0.002s
I20260709 13:18:18.608852 14464 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:18.733532 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:18.733911 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:18.724875 12709 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:18.735010 12775 raft_consensus.cc:2410] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:18.735419 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:18.735365 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:18.736140 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:18.736887 12628 raft_consensus.cc:2393] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:18.737607 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:18.737673 13349 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:18.738268 12628 raft_consensus.cc:2393] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:18.739051 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:18.637696 14378 raft_consensus.cc:3055] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:18.743242 14378 raft_consensus.cc:740] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:18.745679 14378 consensus_queue.cc:260] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:18.755635 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:18.768946 14378 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:18.769112 12775 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 5: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:18.773218 12778 raft_consensus.cc:1240] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:18.739032 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:18.667532 14207 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:18.714464 14288 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:18.761538 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:18.722255 14292 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:18.764170 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:18.786820 12709 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:18.576997 14367 consensus_queue.cc:260] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:18.762275 12633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 163) took 2064 ms. Trace:
I20260709 13:18:18.762233 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:18.754407 14026 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.774753 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:18.806690 12780 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:18.775131 12628 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:18.808506 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:18.708619 14361 raft_consensus.cc:740] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:18.700969 14154 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:18.696355 14375 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:18.781486 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:18.814569 12701 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:18.789161 14292 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:18.733327 14145 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:18.726866 14479 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.788079 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:18.764616 12626 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39590 (request call id 157) took 3266 ms (client timeout 2080 ms). Trace:
I20260709 13:18:18.761942 14273 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:18.776069 14464 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:18.809115 14412 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:18.813457 14367 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:18.792968 12633 rpcz_store.cc:276] 0709 13:18:16.697491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.697635 (+   144us) service_pool.cc:224] Handling call
0709 13:18:16.718897 (+ 21262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.555436 (+836539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.762050 (+1206614us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b54002ea9d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:18.762214 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1633058048}
I20260709 13:18:18.697093 14480 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:18.764778 14352 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:18.809438 12628 raft_consensus.cc:2393] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:18.697798 14489 consensus_queue.cc:260] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:18.846517 14489 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:18.762284 14441 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:18.851933 14441 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:18.819059 12709 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:18.775806 14496 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:18.852952 14496 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:18.853667 14496 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:18.818977 12626 rpcz_store.cc:269] 0709 13:18:15.498310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.498527 (+   217us) service_pool.cc:224] Handling call
0709 13:18:18.764425 (+3265898us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b54000dd754. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:18.764538 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4229590784}
W20260709 13:18:18.832175 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 1.23484e+06us
I20260709 13:18:18.856009 12746 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:18.835307 14361 consensus_queue.cc:260] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:18.817375 14292 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:18.824587 14273 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:18.810514 14026 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:18.836473 13379 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51762 (request call id 409) took 3298 ms (client timeout 1530 ms). Trace:
I20260709 13:18:18.836122 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:18.877242 12633 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:18.853550 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:18.768150 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:18.860611 12780 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:18.881274 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:18.881897 12774 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:18.882551 12780 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:18.864125 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:18.887136 12709 raft_consensus.cc:2393] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 5: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:18.823289 14145 raft_consensus.cc:697] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:18.889741 12783 raft_consensus.cc:2393] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
W20260709 13:18:18.910143 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 197734us
W20260709 13:18:18.940680 12598 leader_election.cc:336] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.518s (SENT)
W20260709 13:18:18.942651 12598 connection.cc:256] RPC call timeout handler was delayed by -1.3016844969999966s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1764s for a response
I20260709 13:18:18.942749 12778 raft_consensus.cc:1240] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 4. Current term is 5. Ops: []
W20260709 13:18:18.944401 12598 connection.cc:256] RPC call timeout handler was delayed by -1.1378844969999591s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1946s for a response
W20260709 13:18:18.945995 12598 connection.cc:256] RPC call timeout handler was delayed by -1.0892844970001079s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2s for a response
W20260709 13:18:18.947014 12598 connection.cc:256] RPC call timeout handler was delayed by -1.0640844970000671s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2028s for a response
I20260709 13:18:18.952875 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:18.955554 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:18.958021 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:18.966302 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:18:18.969892 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.971100 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:18.822135 14479 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:18.875981 14273 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:18.886958 14398 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:18.875662 14110 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:18.994621 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:19.002557 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:19.005417 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 215266us
I20260709 13:18:19.006521 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:19.007926 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:18.941392 14361 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:19.011890 12628 raft_consensus.cc:2393] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 4: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:18.864923 14412 consensus_queue.cc:1059] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.110s
I20260709 13:18:18.969388 14299 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:18.889626 14386 consensus_queue.cc:1059] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.112s
I20260709 13:18:19.021700 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:19.032430 12628 raft_consensus.cc:2393] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 4: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:18:19.032968 14278 consensus_meta.cc:287] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.839s	user 0.009s	sys 0.001s
I20260709 13:18:19.033224 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:19.033887 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:19.033982 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:19.034269 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:19.036193 12628 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:19.036798 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:19.037039 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:19.038029 14145 consensus_queue.cc:237] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:19.039095 14480 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:19.031013 12598 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:19.042594 12598 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:19.043314 14479 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:19.043665 12598 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:19.044162 14460 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:19.043788 14401 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.045214 12634 raft_consensus.cc:2393] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 5: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:19.045783 13383 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:19.046479 14400 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:19.046911 14154 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:19.038036 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:19.047407 14154 raft_consensus.cc:697] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:18:19.048731 14374 consensus_meta.cc:287] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.592s	user 0.010s	sys 0.000s
I20260709 13:18:18.999233 14110 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:18.887012 13379 rpcz_store.cc:269] 0709 13:18:15.537887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.540325 (+  2438us) service_pool.cc:224] Handling call
0709 13:18:18.836193 (+3295868us) spinlock_profiling.cc:243] Waited 2.01 s on lock 0x7b5400690854. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:18.836345 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4616764672}
I20260709 13:18:19.050019 14110 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:19.049172 14154 consensus_queue.cc:237] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:18.945421 14507 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.053006 14507 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:19.053701 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:19.054342 12628 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:19.054940 14464 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:19.055444 12628 raft_consensus.cc:1240] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:18:19.048038 12598 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:19.045904 14363 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:19.061517 12628 raft_consensus.cc:1275] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:19.024006 12701 raft_consensus.cc:2410] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:19.057534 12598 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:19.037309 14473 consensus_meta.cc:287] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.602s	user 0.004s	sys 0.007s
I20260709 13:18:19.023126 12774 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:19.063063 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:19.063138 13349 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:19.063862 12626 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:19.057435 12633 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:18:19.064504 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:19.068207 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:19.069465 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:19.071480 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:19.085740 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:19.061630 14347 consensus_meta.cc:287] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.535s	user 0.009s	sys 0.001s
I20260709 13:18:19.089890 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:19.092000 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:19.094987 12778 raft_consensus.cc:2393] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 4: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:19.095947 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:19.021750 14390 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.065716 14398 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:19.069382 14400 raft_consensus.cc:3055] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:19.098497 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:19.073220 14508 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:19.073477 14489 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:19.033403 14278 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.094568 14523 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:19.107895 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:19.109690 13379 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:18:19.073319 14374 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:18:19.069749 14473 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.099124 14400 raft_consensus.cc:740] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:19.120193 12782 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:19.105368 14489 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:19.094480 14347 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.126595 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:19.132117 12706 raft_consensus.cc:1240] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:19.117930 14401 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:19.103755 14390 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.124840 14489 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:19.134081 14375 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:19.153273 14400 consensus_queue.cc:260] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:19.159090 13383 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:19.159567 14400 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:19.152565 14026 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:19.161455 13379 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:19.164876 14292 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.165745 14523 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:19.166138 14530 log.cc:927] Time spent T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:18:19.166217 14523 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:19.167656 14292 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:19.171906 14278 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:19.174700 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:19.169785 14528 log.cc:927] Time spent T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:18:19.176934 13377 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:19.191918 14390 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:19.198202 14473 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:19.203939 14496 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.151957 14448 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:19.218832 14444 log.cc:927] Time spent T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 2.287s	user 0.006s	sys 0.001s
I20260709 13:18:19.219501 14273 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.218858 14347 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:19.175557 14288 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:19.227480 12704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1385) took 2066 ms (client timeout 1707 ms). Trace:
W20260709 13:18:19.220999 14394 consensus_meta.cc:287] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.932s	user 0.011s	sys 0.000s
W20260709 13:18:19.227885 12704 rpcz_store.cc:269] 0709 13:18:17.160870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.269507 (+108637us) service_pool.cc:224] Handling call
0709 13:18:19.227295 (+1957788us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54001d8654. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:19.227412 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":277027200}
W20260709 13:18:19.223799 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:19.232162 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:19.234035 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:19.235594 14523 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.333545 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:19.245514 12672 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:19.360882 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:19.364778 12702 raft_consensus.cc:2393] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 4: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:19.368629 12703 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:19.299237 14489 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.332499 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:19.369529 12700 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:19.371852 12627 raft_consensus.cc:1240] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:19.379642 12703 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:19.380746 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:19.381862 12703 raft_consensus.cc:2393] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 4: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:19.382524 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:19.383679 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:19.382649 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:19.384871 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:19.385262 12703 raft_consensus.cc:2368] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3.
I20260709 13:18:19.385768 12704 raft_consensus.cc:2368] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4.
I20260709 13:18:19.386085 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:19.386590 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:19.386648 12703 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:19.387347 12704 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:19.388624 12704 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:19.390087 12704 raft_consensus.cc:1275] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:19.393198 12703 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:19.398984 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:19.399631 12779 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 4
W20260709 13:18:19.403306 14524 log.cc:927] Time spent T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.403s	user 0.001s	sys 0.001s
I20260709 13:18:19.408744 12779 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:19.409957 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:19.412714 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:19.413507 12779 raft_consensus.cc:2393] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:19.414379 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:19.415112 12779 raft_consensus.cc:2393] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:19.415860 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:19.416353 12779 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:19.423391 12779 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:19.533795 14395 raft_consensus.cc:3055] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:19.534408 14395 raft_consensus.cc:740] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:19.535249 14395 consensus_queue.cc:260] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:19.537120 14395 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:19.292037 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:19.551683 13382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51762 (request call id 419) took 1107 ms. Trace:
I20260709 13:18:19.552023 13382 rpcz_store.cc:276] 0709 13:18:18.444606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.444734 (+   128us) service_pool.cc:224] Handling call
0709 13:18:19.291804 (+847070us) spinlock_profiling.cc:243] Waited 985 us on lock 0x7b5800095198. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c723780a0 00007f4c72377984 00007f4c6b8f26d4 00007f4c6b8f2662
0709 13:18:19.551592 (+259788us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2264832}
W20260709 13:18:19.564702 14519 log.cc:927] Time spent T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.564s	user 0.000s	sys 0.003s
I20260709 13:18:19.250828 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:19.593261 12633 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:18:19.596547 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:19.597136 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:19.597447 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:19.599720 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:19.608618 12529 catalog_manager.cc:5697] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 1 to 2, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 2 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:19.611651 12529 catalog_manager.cc:5697] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:19.612737 12529 catalog_manager.cc:5697] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 2 to 3, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:19.615226 12529 catalog_manager.cc:5697] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:19.620275 14367 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:19.620934 14367 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.623323 14367 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:19.630306 14489 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:19.639286 12775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 162) took 1236 ms. Trace:
I20260709 13:18:19.641398 12705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1386) took 2480 ms. Trace:
I20260709 13:18:19.646394 12705 rpcz_store.cc:276] 0709 13:18:17.161158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.350660 (+1189502us) service_pool.cc:224] Handling call
0709 13:18:18.426539 (+ 75879us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:18.432067 (+  5528us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:19.637252 (+1205185us) spinlock_profiling.cc:243] Waited 75.5 ms on lock 0x7b54001090d0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:19.641314 (+  4062us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":173668736}
I20260709 13:18:19.645109 14360 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:19.639792 12775 rpcz_store.cc:276] 0709 13:18:18.404185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.769798 (+365613us) service_pool.cc:224] Handling call
0709 13:18:18.770100 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:18.770436 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:18.770470 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:18.770952 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 13:18:18.818344 (+ 47392us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:18.818385 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.818448 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:19.402717 (+584269us) spinlock_profiling.cc:243] Waited 9.59 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:18:19.638957 (+236240us) raft_consensus.cc:1685] finished
0709 13:18:19.638980 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:19.639217 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2429,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":103,"spinlock_wait_cycles":22053632,"thread_start_us":47033,"threads_started":1,"wal-append.queue_time_us":94055}
I20260709 13:18:19.649700 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:19.651991 12632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56832 (request call id 1765) took 1707 ms. Trace:
I20260709 13:18:19.652392 12632 rpcz_store.cc:276] 0709 13:18:17.944995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.373816 (+428821us) service_pool.cc:224] Handling call
0709 13:18:19.651752 (+1277936us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b54003d8954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:19.651893 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":466772352}
I20260709 13:18:19.666817 14360 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:19.667495 14360 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.669844 14360 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:19.678077 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:19.678956 12633 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:19.680213 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:19.681288 12633 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:19.681993 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:19.682849 12627 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:19.683516 14491 log.cc:927] Time spent T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 1.369s	user 0.007s	sys 0.000s
I20260709 13:18:19.677017 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:19.686856 12632 raft_consensus.cc:2393] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:19.687793 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:19.688773 12632 raft_consensus.cc:2376] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3. Re-sending same reply.
I20260709 13:18:19.690528 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:19.691875 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:19.690816 14336 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:19.694288 12634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56832 (request call id 1778) took 1062 ms. Trace:
I20260709 13:18:19.694667 12634 rpcz_store.cc:276] 0709 13:18:18.631676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.724445 (+ 92769us) service_pool.cc:224] Handling call
0709 13:18:19.694110 (+969665us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b540004b5d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:19.694234 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":576983936}
I20260709 13:18:19.695848 12776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54346 (request call id 154) took 1578 ms. Trace:
I20260709 13:18:19.695995 12627 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:19.696419 12776 rpcz_store.cc:276] 0709 13:18:18.117672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.117792 (+   120us) service_pool.cc:224] Handling call
0709 13:18:19.695674 (+1577882us) spinlock_profiling.cc:243] Waited 597 ms on lock 0x7b540008a554. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:19.695788 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1373874432}
W20260709 13:18:19.711935 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 654418us
I20260709 13:18:19.714057 12598 leader_election.cc:400] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 4.
I20260709 13:18:19.715032 12598 leader_election.cc:400] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 4.
I20260709 13:18:19.715219 12598 leader_election.cc:403] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:18:19.718623 14394 raft_consensus.cc:1064] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:18:19.719030 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:19.719609 12777 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:19.721864 14394 consensus_queue.cc:237] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 2, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.735558 14317 raft_consensus.cc:3055] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:19.736194 14317 raft_consensus.cc:740] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:19.737874 14317 consensus_queue.cc:260] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.739902 14317 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:19.740993 14376 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.741819 14376 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.744376 14376 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:19.255030 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:19.749603 13380 raft_consensus.cc:2393] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 5: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
W20260709 13:18:19.753839 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 508405us
I20260709 13:18:19.758577 14448 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:19.759238 14448 raft_consensus.cc:697] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:19.760213 14448 consensus_queue.cc:237] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.763126 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:19.763859 12706 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
I20260709 13:18:19.764976 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:19.765635 12705 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:19.769991 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:19.770934 12700 raft_consensus.cc:2393] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 4: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:19.774624 14273 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:19.778034 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:19.301375 14288 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:19.245118 14110 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.785519 14523 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:19.802476 14110 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:19.804417 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:19.810014 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:19.810889 13382 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 3: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:19.806025 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:19.812142 13376 raft_consensus.cc:2368] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4.
I20260709 13:18:19.812363 12705 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
I20260709 13:18:19.800807 14347 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:19.813139 12783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36478 (request call id 1701) took 1459 ms (client timeout 1533 ms). Trace:
I20260709 13:18:19.813493 14347 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.815454 12628 raft_consensus.cc:1240] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:18:19.816313 14347 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:19.801311 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:19.817759 12781 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:19.818893 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:19.819990 12777 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:19.821175 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:19.822888 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:19.813539 12783 rpcz_store.cc:269] 0709 13:18:18.353297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.354973 (+  1676us) service_pool.cc:224] Handling call
0709 13:18:19.812952 (+1457979us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b540040e554. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:19.813085 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1180178176}
I20260709 13:18:19.825649 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:19.825928 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:18:19.834007 14513 log.cc:927] Time spent T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 1.101s	user 0.005s	sys 0.000s
I20260709 13:18:19.827958 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:19.835242 13381 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51762 (request call id 417) took 1416 ms (client timeout 1841 ms). Trace:
W20260709 13:18:19.835664 13381 rpcz_store.cc:269] 0709 13:18:18.418783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.421553 (+  2770us) service_pool.cc:224] Handling call
0709 13:18:19.827792 (+1406239us) spinlock_profiling.cc:243] Waited 6.51 ms on lock 0x7b5800095198. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c723780a0 00007f4c72377984 00007f4c6b8f26d4 00007f4c6b8f2662
0709 13:18:19.835123 (+  7331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14972672}
I20260709 13:18:19.828418 13376 raft_consensus.cc:1240] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:19.837111 12780 raft_consensus.cc:1240] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:18:19.839138 12625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 1760) took 1895 ms. Trace:
I20260709 13:18:19.839604 12625 rpcz_store.cc:276] 0709 13:18:17.943415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.435509 (+492094us) service_pool.cc:224] Handling call
0709 13:18:18.435828 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:18.451267 (+ 15439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:18.451322 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:18.457097 (+  5775us) log.cc:844] Serialized 26 byte log entry
0709 13:18:18.715603 (+258506us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:18.715646 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.715711 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:19.833330 (+1117619us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:18:19.838861 (+  5531us) raft_consensus.cc:1685] finished
0709 13:18:19.838883 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:19.839081 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7677,"raft.queue_time_us":18410,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":8860,"spinlock_wait_cycles":7079808,"thread_start_us":238515,"threads_started":1,"wal-append.queue_time_us":263774}
I20260709 13:18:19.850497 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:19.851203 14455 raft_consensus.cc:3055] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:19.851603 14455 raft_consensus.cc:740] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:19.852375 14455 consensus_queue.cc:260] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:19.853682 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:19.854997 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:19.858942 12705 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:18:19.861663 14518 log.cc:927] Time spent T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 1.129s	user 0.002s	sys 0.003s
I20260709 13:18:19.863812 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:19.864609 12706 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:19.851213 12700 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:19.865948 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:19.853360 14455 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:19.866648 12705 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:19.867110 14349 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:19.870268 14349 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.235913 14496 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:19.873236 14349 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:19.875418 14400 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:19.876173 14400 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 4.
W20260709 13:18:19.878222 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 489376us
W20260709 13:18:19.878415 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:19.880187 12597 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:19.881239 12597 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:19.884040 12597 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:19.883848 14336 consensus_queue.cc:1059] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:19.867676 14110 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:19.830715 13379 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:19.881618 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:19.882318 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:19.869768 13377 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:19.882455 13385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 362) took 2255 ms. Trace:
W20260709 13:18:19.885479 12708 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.850s	user 0.000s	sys 0.014s
W20260709 13:18:19.894356 12708 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.859s	user 0.001s	sys 0.014s
I20260709 13:18:19.896710 12628 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
I20260709 13:18:19.887172 14183 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:19.900031 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:19.885357 12597 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:19.885193 14336 consensus_queue.cc:1059] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:19.905108 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:19.906313 12705 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 3: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:19.901348 14401 raft_consensus.cc:3055] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:19.907382 13380 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:19.926980 12597 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:19.886529 14207 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:19.907982 14464 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.908052 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:19.928131 12708 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:19.895350 13385 rpcz_store.cc:276] 0709 13:18:17.627518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.627757 (+   239us) service_pool.cc:224] Handling call
0709 13:18:17.677216 (+ 49459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:17.677665 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.677706 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:17.692486 (+ 14780us) log.cc:844] Serialized 26 byte log entry
0709 13:18:17.953208 (+260722us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:17.953258 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.953321 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:19.682379 (+1729058us) spinlock_profiling.cc:243] Waited 6.74 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:18:19.882130 (+199751us) raft_consensus.cc:1685] finished
0709 13:18:19.882150 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:19.882377 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2315,"raft.queue_time_us":9527,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":6341,"spinlock_wait_cycles":15491968,"thread_start_us":245608,"threads_started":1,"wal-append.queue_time_us":542061}
I20260709 13:18:19.909765 14183 raft_consensus.cc:697] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:18:19.910234 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 192382us
I20260709 13:18:19.901706 14110 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.913229 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1393) took 1183 ms. Trace:
I20260709 13:18:19.913393 12704 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:19.932744 12702 rpcz_store.cc:276] 0709 13:18:18.729406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.761248 (+ 31842us) service_pool.cc:224] Handling call
0709 13:18:19.912964 (+1151716us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b540006e854. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:19.913155 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1329637248}
W20260709 13:18:19.933139 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 128827us
I20260709 13:18:19.914506 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:19.909361 12700 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:19.923519 13384 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:19.913975 14292 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.885814 14400 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.925060 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:19.926589 12746 leader_election.cc:400] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4.
W20260709 13:18:19.934741 13384 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51762 (request call id 413) took 1531 ms (client timeout 1610 ms). Trace:
I20260709 13:18:19.938346 12701 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:19.937302 12746 leader_election.cc:403] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:19.941320 12746 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:19.945050 12708 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
W20260709 13:18:19.937601 13384 rpcz_store.cc:269] 0709 13:18:18.403809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.403941 (+   132us) service_pool.cc:224] Handling call
0709 13:18:19.825785 (+1421844us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b5800095198. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c723780a0 00007f4c72377984 00007f4c6b8f26d4 00007f4c6b8f2662
0709 13:18:19.934683 (+108898us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33360640}
I20260709 13:18:19.927093 14401 raft_consensus.cc:740] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:18:19.951632 12747 connection.cc:256] RPC call timeout handler was delayed by -1.5121266619999005s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22010000000000002s for a response
I20260709 13:18:19.952845 12598 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:19.953289 14455 raft_consensus.cc:3055] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:19.953765 14375 consensus_queue.cc:1059] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:18:19.948150 14527 log.cc:927] Time spent T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.913s	user 0.003s	sys 0.002s
I20260709 13:18:19.948320 12672 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:19.935587 14441 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:19.935859 14508 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.937882 14126 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:19.946929 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:19.939252 14395 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.972164 14395 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:19.974617 14395 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:19.975337 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:19.976748 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:19.947331 14292 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.977929 12706 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:18:19.248313 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:19.978930 12782 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:19.979854 14292 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:19.951781 14207 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:19.961737 14398 consensus_queue.cc:1059] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:18:19.982055 14110 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:19.983042 14126 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.957783 14408 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:19.955557 14317 consensus_queue.cc:1059] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:18:19.986202 14408 raft_consensus.cc:3055] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:19.986455 14317 raft_consensus.cc:3055] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:19.986650 14408 raft_consensus.cc:740] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:19.986909 14317 raft_consensus.cc:740] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:19.987591 14408 consensus_queue.cc:260] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:19.987905 14317 consensus_queue.cc:260] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.954496 14183 consensus_queue.cc:237] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:19.946425 14378 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4.
I20260709 13:18:19.961433 12598 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:19.991910 12598 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:19.949879 14386 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:19.998584 14126 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:19.999032 14436 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:20.000380 14207 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:19.965615 14336 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:20.004127 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:20.005038 12634 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
I20260709 13:18:19.989373 14317 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:20.006239 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:20.007777 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:20.008569 12632 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:19.963680 14508 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:20.014643 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:20.015635 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:19.948339 14400 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:19.966288 14441 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:20.018870 14441 raft_consensus.cc:697] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:20.019161 14400 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:20.020166 14441 consensus_queue.cc:237] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:20.022121 12628 raft_consensus.cc:2393] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:19.967872 14401 consensus_queue.cc:260] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:20.026592 14401 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:20.027866 12673 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:19.954380 14422 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:19.957177 14455 raft_consensus.cc:740] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:19.969554 12780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1485) took 1090 ms. Trace:
I20260709 13:18:20.035477 12780 rpcz_store.cc:276] 0709 13:18:18.879487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.883584 (+  4097us) service_pool.cc:224] Handling call
0709 13:18:18.884500 (+   916us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:19.838430 (+953930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:19.969294 (+130864us) spinlock_profiling.cc:243] Waited 952 ms on lock 0x7b54002cc254. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:19.969495 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2190044032}
W20260709 13:18:20.037175 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 117846us
I20260709 13:18:20.031415 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:20.038856 12628 raft_consensus.cc:2410] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:18:19.971616 12631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56838 (request call id 997) took 2972 ms (client timeout 1961 ms). Trace:
I20260709 13:18:19.994930 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:20.039871 12631 rpcz_store.cc:269] 0709 13:18:16.999524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.559556 (+560032us) service_pool.cc:224] Handling call
0709 13:18:19.971413 (+2411857us) spinlock_profiling.cc:243] Waited 7.15 ms on lock 0x7b54001595d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:19.971545 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16451328}
I20260709 13:18:20.040910 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:19.997929 12598 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:19.998294 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:19.936724 14464 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:19.964948 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:20.053165 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 88282.4us
I20260709 13:18:20.055238 12672 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:19.999136 14375 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:20.072554 14464 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:20.063334 14373 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:20.052716 14410 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:20.074239 14410 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.074643 14410 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:20.062892 14508 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:20.075670 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:20.076583 13377 raft_consensus.cc:2393] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 3: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:19.955017 14349 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.098s
I20260709 13:18:20.080263 14455 consensus_queue.cc:260] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:20.081619 14455 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:20.037528 14479 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:18:20.050115 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 52273.7us
I20260709 13:18:20.089150 14145 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:20.091922 12598 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:20.092826 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 47539.8us
I20260709 13:18:20.087510 12672 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:20.056640 12673 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:20.063036 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:18:20.095856 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 142357us
I20260709 13:18:20.063016 12632 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:20.034999 14422 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:20.093528 12598 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:20.098834 12598 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:19.969960 12747 connection.cc:256] RPC call timeout handler was delayed by -1.1764266619998125s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2574s for a response
I20260709 13:18:20.099912 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:20.100989 12780 raft_consensus.cc:2393] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 3: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:20.100833 12598 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:20.030956 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:20.101873 12782 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:18:20.102934 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:20.103686 13382 raft_consensus.cc:2393] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 5: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
W20260709 13:18:20.104033 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 47484.1us
I20260709 13:18:20.098678 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:20.105547 14523 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:20.106050 14523 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.106405 14523 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:20.110143 12781 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:20.110332 12597 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:20.110666 13385 raft_consensus.cc:1275] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEARNER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:18:20.111135 14394 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.111616 14394 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:20.112440 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:20.113022 14436 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:20.113629 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:20.114382 13379 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:18:20.105477 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:20.115499 12777 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:19.256327 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:20.114387 14394 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:20.116629 13378 raft_consensus.cc:2393] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:20.117421 14394 consensus_queue.cc:1048] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:20.117728 14207 consensus_queue.cc:1059] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:20.118508 14207 consensus_queue.cc:1059] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:20.104795 12673 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:20.121737 14394 raft_consensus.cc:3055] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:20.130671 12672 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:20.136545 12672 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:20.089735 14373 raft_consensus.cc:697] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:20.158943 12707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1040) took 2447 ms. Trace:
I20260709 13:18:20.159543 12707 rpcz_store.cc:276] 0709 13:18:17.711187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.443869 (+732682us) service_pool.cc:224] Handling call
0709 13:18:18.635045 (+191176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:18.709611 (+ 74566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.158685 (+1449074us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b54002490d0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:20.158870 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":439061760}
I20260709 13:18:20.165467 13384 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 1. Current term is 2. Ops: []
W20260709 13:18:20.104095 14460 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.059s	user 0.046s	sys 0.095s
I20260709 13:18:20.103972 14422 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:20.135414 14394 raft_consensus.cc:740] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:20.085906 14375 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:20.157683 13376 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:20.175041 14394 consensus_queue.cc:260] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:20.095757 14145 raft_consensus.cc:697] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:20.103983 14401 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:20.135664 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:20.136755 14336 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:18:20.192233 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 56636.6us
I20260709 13:18:20.133685 14268 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:18:20.176276 14460 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.132s	user 0.046s	sys 0.095s
I20260709 13:18:20.135664 14126 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:20.136998 14478 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:20.135521 14479 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:18:20.233341 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 117651us
I20260709 13:18:20.162101 14395 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:20.206652 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:20.238283 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:20.212990 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.235800 12597 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:20.131559 14400 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:20.171413 14448 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:20.130430 14347 consensus_queue.cc:1059] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 13:18:20.156730 14367 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 13:18:20.240572 12597 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:20.220135 14375 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:20.220597 14378 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:20.248814 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:20.228092 14460 raft_consensus.cc:697] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:20.231493 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:20.228186 14268 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:18:20.239420 14545 log.cc:927] Time spent T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:18:20.252720 12782 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:20.243594 14401 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:20.247759 14126 consensus_queue.cc:1059] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
W20260709 13:18:20.253985 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 117506us
I20260709 13:18:20.250444 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:20.256734 12629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 160) took 3567 ms. Trace:
I20260709 13:18:20.242563 14508 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:18:20.261999 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.241799 14448 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:20.250552 14336 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:20.252573 14506 raft_consensus.cc:3055] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:20.236663 14523 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:20.287104 13349 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:20.237654 14478 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:20.256469 14399 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:20.252323 14400 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:18:20.278097 14394 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:20.260175 14378 raft_consensus.cc:697] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:20.287005 14448 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:20.272900 14268 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:20.289608 14336 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.282536 14110 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:20.345305 13378 raft_consensus.cc:2393] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:20.273518 14026 consensus_queue.cc:1059] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 13:18:20.294409 14508 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:20.293591 14561 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:20.362921 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 195767us
I20260709 13:18:20.293200 14506 raft_consensus.cc:740] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:20.367331 12778 raft_consensus.cc:1275] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:18:20.367395 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 136782us
I20260709 13:18:20.379436 12747 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:20.272219 14373 consensus_queue.cc:237] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:20.358012 13376 raft_consensus.cc:1240] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:20.380594 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:20.382668 14367 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:20.383070 14367 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:20.383440 14367 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:20.383602 12747 leader_election.cc:400] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3.
I20260709 13:18:20.383965 12747 leader_election.cc:403] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:18:20.381433 12673 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:20.369831 14336 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:20.384783 12747 leader_election.cc:400] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4.
I20260709 13:18:20.388645 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:20.373059 14360 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:20.388605 12673 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:20.393883 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.313675 14145 consensus_queue.cc:237] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:20.383795 12730 debug-util.cc:398] Leaking SignalData structure 0x7b08000a3900 after lost signal to thread 11520
I20260709 13:18:20.379778 14508 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:20.288108 12629 rpcz_store.cc:276] 0709 13:18:16.689004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.691195 (+  2191us) service_pool.cc:224] Handling call
0709 13:18:16.691516 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.691734 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:16.709995 (+ 18261us) log.cc:844] Serialized 14 byte log entry
0709 13:18:19.953285 (+3243290us) spinlock_profiling.cc:243] Waited 8.36 ms on lock 0x7b180024b800. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c2455 00007f4c70daa3c7 00007f4c6d3bc9a7 00007f4c6d3cc0bf 00007f4c6d3cc038
0709 13:18:20.100584 (+147299us) log.cc:844] Serialized 14 byte log entry
0709 13:18:20.256252 (+155668us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:20.256308 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:20.256355 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.256556 (+   201us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:20.256675 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9047,"prepare.queue_time_us":606914,"prepare.run_cpu_time_us":158,"prepare.run_wall_time_us":23751,"raft.queue_time_us":8980,"raft.run_cpu_time_us":546,"raft.run_wall_time_us":33874,"spinlock_wait_cycles":19233152,"thread_start_us":1164043,"threads_started":3,"wal-append.queue_time_us":1544249,"wal-append.run_cpu_time_us":1378,"wal-append.run_wall_time_us":1045901}
I20260709 13:18:20.389542 14361 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3.
I20260709 13:18:20.415483 14378 consensus_queue.cc:237] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:20.415927 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:20.407603 14410 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:20.375175 14110 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:20.357076 14399 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:20.415809 14361 consensus_queue.cc:1048] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:20.402572 14506 consensus_queue.cc:260] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:20.469682 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.418860 12783 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:20.482470 13349 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:20.389438 14349 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:18:20.491570 12672 leader_election.cc:336] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Timed out: RequestConsensusVote RPC to 127.11.63.68:36021 timed out after 1.530s (SENT)
I20260709 13:18:20.492100 12672 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:20.448624 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.492816 14479 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:20.493194 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:20.493661 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:20.453393 12673 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:20.415541 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:20.494976 12673 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:20.495132 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:20.495433 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:20.496075 12673 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:20.496549 12706 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:18:20.497143 14292 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:20.494990 12708 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:18:20.497891 14292 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:20.498603 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:20.499056 14375 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:20.499792 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:20.500253 12705 raft_consensus.cc:1275] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:18:20.500761 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:20.501075 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:20.501619 12708 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:20.501704 12703 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:20.502125 12709 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:20.502640 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:20.503050 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:20.503228 12703 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
I20260709 13:18:20.503633 12709 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
I20260709 13:18:20.504535 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:20.505108 12701 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:20.505205 14292 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:20.497087 12673 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:20.496493 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:20.506266 14292 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:20.506925 14292 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:20.507141 12673 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:20.507404 14292 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:20.508061 14479 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:20.508466 14479 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.508803 14479 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:20.508966 12706 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:20.501276 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:20.511116 12705 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:20.419795 14026 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.511749 14026 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:20.419417 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.504555 12707 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:20.513787 14564 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:20.514545 14564 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.514948 14564 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:20.500173 12672 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:20.518414 14375 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:20.521111 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:20.521377 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:20.523094 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:20.523744 12774 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:20.524093 12629 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:18:20.526856 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:20.536404 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:20.538552 12780 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:20.540570 12625 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:20.542060 14207 consensus_queue.cc:1059] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:20.543354 14394 raft_consensus.cc:3055] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:20.543795 14394 raft_consensus.cc:740] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:20.544806 14394 consensus_queue.cc:260] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:20.545907 12774 raft_consensus.cc:1240] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:20.547082 12673 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:20.547924 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:20.548545 14401 consensus_queue.cc:1059] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:20.549964 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:20.396916 14562 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:20.554972 14562 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.555325 14562 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:20.556193 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:20.556977 12631 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 3: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:20.557124 12630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 170) took 2666 ms. Trace:
I20260709 13:18:20.557514 12630 rpcz_store.cc:276] 0709 13:18:17.890398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.418292 (+527894us) service_pool.cc:224] Handling call
0709 13:18:19.973688 (+1555396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:19.985853 (+ 12165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:19.985903 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:19.985964 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:19.986015 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.132346 (+146331us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b54001595d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:20.132406 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:20.556957 (+424551us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b54001595d0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:20.557068 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3602896512}
I20260709 13:18:20.558173 14479 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:20.560909 14479 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:20.561933 14464 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:20.563313 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:20.550355 12672 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:20.550516 12777 raft_consensus.cc:1240] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:20.551653 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:20.565809 12780 raft_consensus.cc:2393] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:20.567026 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:20.563910 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:20.398375 14560 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:20.553040 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:20.568231 12774 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:20.552306 12708 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:20.555459 14288 raft_consensus.cc:3055] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:20.490279 14507 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:20.617763 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:20.620837 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 73831.7us
I20260709 13:18:20.622491 12673 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:20.623492 14479 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:20.623984 14479 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.624385 14479 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:20.624358 12673 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:20.625944 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 59729.3us
I20260709 13:18:20.626840 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:20.631371 12673 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:20.386874 14460 consensus_queue.cc:237] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:20.384392 14561 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:20.640087 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.441820 14422 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:20.597661 12730 debug-util.cc:398] Leaking SignalData structure 0x7b0800112c60 after lost signal to thread 12566
I20260709 13:18:20.595765 14400 raft_consensus.cc:3055] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:18:20.737802 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:20.652915 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:20.728911 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.596143 14464 raft_consensus.cc:2764] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 13:18:20.614985 14288 raft_consensus.cc:740] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:20.755832 13384 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:20.646203 14547 consensus_queue.cc:1048] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.113s
W20260709 13:18:20.762749 14567 log.cc:927] Time spent T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:18:20.763015 14478 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:20.763836 14478 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.764431 14478 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:20.766185 14441 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:20.764940 14422 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:20.767525 14561 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:20.740243 14183 raft_consensus.cc:3055] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:20.768002 14422 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.768311 14183 raft_consensus.cc:740] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:20.768697 14422 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:20.769531 14183 consensus_queue.cc:260] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:20.770047 14288 consensus_queue.cc:260] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:20.770767 14422 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:20.771111 14288 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:20.771214 14183 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:20.774893 14410 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:20.780148 14336 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:20.780742 14395 consensus_meta.cc:287] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.537s	user 0.009s	sys 0.001s
I20260709 13:18:20.794672 14395 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:20.767518 14441 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:20.769340 14561 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:20.796775 14561 raft_consensus.cc:697] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 5 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:18:20.792637 14568 log.cc:927] Time spent T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.347s	user 0.000s	sys 0.000s
I20260709 13:18:20.791115 14110 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:20.797739 14523 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:20.817437 12629 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
W20260709 13:18:20.822436 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 431657us
I20260709 13:18:20.823892 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:20.618778 14401 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.129s
I20260709 13:18:20.800653 14561 consensus_queue.cc:237] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:20.738152 14400 raft_consensus.cc:740] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:20.629890 14507 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:20.871515 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.741676 14566 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:18:20.927330 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 425221us
W20260709 13:18:20.930902 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.886266 14110 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:20.887267 14399 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:20.880695 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 56905.9us
I20260709 13:18:20.977360 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:20.995946 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:20.997361 12747 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:20.896164 13383 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.010s	sys 0.012s
I20260709 13:18:20.953372 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:21.011185 12747 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:21.012919 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 59675.1us
W20260709 13:18:21.002619 13351 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.012s	sys 0.018s
I20260709 13:18:21.017921 12700 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:21.020339 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:21.027439 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 400646us
W20260709 13:18:21.002461 12730 debug-util.cc:398] Leaking SignalData structure 0x7b080025b900 after lost signal to thread 12658
W20260709 13:18:21.029738 13351 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.013s	sys 0.019s
I20260709 13:18:20.994426 14349 raft_consensus.cc:697] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:18:21.017249 13383 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.010s	sys 0.013s
W20260709 13:18:21.024600 14455 consensus_meta.cc:287] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.942s	user 0.012s	sys 0.000s
I20260709 13:18:20.871942 14564 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.009503 14566 raft_consensus.cc:697] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:21.007166 14110 raft_consensus.cc:697] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:18:21.039039 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:21.062340 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:21.062762 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.034030 14400 consensus_queue.cc:260] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.047063 14571 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:18:21.037050 14560 raft_consensus.cc:697] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:18:21.070171 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:21.071599 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:21.081703 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:21.086082 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 454758us
I20260709 13:18:21.088268 12673 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:21.090071 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.091463 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:21.048520 14378 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:21.096892 14145 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:21.099151 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:21.097303 12673 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:21.099850 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:21.100102 14145 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:21.096623 12672 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:21.100327 12634 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:21.101886 12673 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:21.100489 12629 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:21.107265 12673 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:21.107426 12704 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:21.109623 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:21.109992 12708 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:21.105978 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:21.100582 14145 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:21.116532 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:21.119936 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:21.273242 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:21.275264 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.145241 12598 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:21.281011 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 135721us
I20260709 13:18:21.283419 12598 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:21.273339 12630 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:18:21.287506 12674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.007s	sys 0.024s
W20260709 13:18:21.102253 12730 debug-util.cc:398] Leaking SignalData structure 0x7b080016e180 after lost signal to thread 12733
I20260709 13:18:21.145473 12707 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:21.172230 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:21.289408 12730 debug-util.cc:398] Leaking SignalData structure 0x7b08002acb80 after lost signal to thread 12734
W20260709 13:18:21.292181 12730 debug-util.cc:398] Leaking SignalData structure 0x7b0800398160 after lost signal to thread 12807
W20260709 13:18:21.288348 12674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.007s	sys 0.025s
W20260709 13:18:21.294756 12730 debug-util.cc:398] Leaking SignalData structure 0x7b0800374b80 after lost signal to thread 13410
W20260709 13:18:21.295111 12730 debug-util.cc:398] Leaking SignalData structure 0x7b0800374a60 after lost signal to thread 13411
W20260709 13:18:21.145490 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 219865us
I20260709 13:18:21.176779 14110 consensus_queue.cc:237] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:21.178416 14268 consensus_meta.cc:287] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.804s	user 0.013s	sys 0.002s
W20260709 13:18:21.197484 14292 consensus_meta.cc:287] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.689s	user 0.007s	sys 0.000s
I20260709 13:18:21.409384 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:21.408471 14292 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:21.412007 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 129288us
I20260709 13:18:21.412969 14292 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:21.184443 14349 consensus_queue.cc:237] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.430132 14268 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:21.204044 14367 consensus_meta.cc:287] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.818s	user 0.007s	sys 0.002s
I20260709 13:18:21.434504 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:21.434333 14367 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:21.098299 14378 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:21.436995 14410 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:21.438164 14378 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:21.170742 14398 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.442510 14398 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.121579 14400 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:21.101181 14441 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:18:21.178869 14448 consensus_meta.cc:287] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.810s	user 0.011s	sys 0.000s
I20260709 13:18:21.451512 14448 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.171464 14478 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:21.201525 14507 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:21.036273 14562 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.206561 14564 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:21.186650 14566 consensus_queue.cc:237] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:21.187731 14571 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:21.457469 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:21.458029 14571 raft_consensus.cc:697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:21.458662 13379 raft_consensus.cc:2376] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3. Re-sending same reply.
W20260709 13:18:21.459676 13379 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51762 (request call id 426) took 1301 ms (client timeout 1610 ms). Trace:
W20260709 13:18:21.238086 14575 log.cc:927] Time spent T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
W20260709 13:18:21.236728 14576 log.cc:927] Time spent T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 13:18:21.239091 14578 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
W20260709 13:18:21.227129 14580 log.cc:927] Time spent T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:18:21.471019 12597 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:21.471580 13383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 384) took 1218 ms. Trace:
I20260709 13:18:21.181595 14582 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:21.171411 14583 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:21.473410 14582 raft_consensus.cc:697] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:21.473932 14583 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:21.474721 14583 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:21.199522 14584 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.483937 14584 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:21.487152 14360 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:21.488045 14360 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:21.207690 14587 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:21.196347 14586 consensus_queue.cc:1059] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:21.229141 14588 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:21.491528 14588 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:21.475121 14582 consensus_queue.cc:237] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:21.492125 14588 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:21.234496 14590 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:21.491492 14586 raft_consensus.cc:3055] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:21.222368 14592 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:21.503250 14586 raft_consensus.cc:740] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:21.185657 14594 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:18:21.221055 14375 consensus_meta.cc:287] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.702s	user 0.013s	sys 0.009s
W20260709 13:18:21.221432 14479 consensus_meta.cc:287] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.596s	user 0.006s	sys 0.006s
W20260709 13:18:21.356346 14288 consensus_meta.cc:287] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.585s	user 0.009s	sys 0.004s
I20260709 13:18:21.349768 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:21.357918 14591 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:21.368137 14608 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:21.369359 14602 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:21.372838 14593 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:21.373901 14399 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:21.374471 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 356421us
W20260709 13:18:21.381836 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.400478 14547 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:21.401279 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 359837us
I20260709 13:18:21.369802 14558 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.183164 14336 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:21.520861 14448 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:21.521250 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:21.425542 14292 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:18:21.523720 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.427189 12782 raft_consensus.cc:1240] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:18:21.194969 14573 log.cc:927] Time spent T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
W20260709 13:18:21.460352 13379 rpcz_store.cc:269] 0709 13:18:20.157898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.159543 (+  1645us) service_pool.cc:224] Handling call
0709 13:18:21.457009 (+1297466us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b5800095198. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c723780a0 00007f4c72377984 00007f4c6b8f26d4 00007f4c6b8f2662
0709 13:18:21.459552 (+  2543us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5502336}
I20260709 13:18:21.523712 14268 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:21.461931 14571 consensus_queue.cc:237] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.527624 14367 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:21.527300 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:21.530289 14561 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:21.478297 12707 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
W20260709 13:18:21.502851 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.183056 14560 consensus_queue.cc:237] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.507475 14375 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:21.508355 14479 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.510013 12630 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:21.472260 13383 rpcz_store.cc:276] 0709 13:18:20.253309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.253777 (+   468us) service_pool.cc:224] Handling call
0709 13:18:20.360893 (+107116us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:20.382277 (+ 21384us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:18:20.382314 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:20.383487 (+  1173us) log.cc:844] Serialized 556 byte log entry
0709 13:18:21.048567 (+665080us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:21.064237 (+ 15670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:21.069092 (+  4855us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:21.470900 (+401808us) raft_consensus.cc:1685] finished
0709 13:18:21.470959 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:21.471291 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":654063,"threads_started":1,"wal-append.queue_time_us":659195}
I20260709 13:18:21.518002 14593 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:21.525780 14441 raft_consensus.cc:697] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:18:21.541133 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.541399 14399 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:21.547966 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 345045us
W20260709 13:18:21.559731 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 73779.5us
I20260709 13:18:21.547561 14398 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:21.560933 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:21.541781 14602 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:21.567831 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:21.568327 14612 consensus_queue.cc:1059] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:18:21.568620 12598 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:21.573777 14593 consensus_queue.cc:1048] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:21.544091 14561 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:21.566893 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:21.574249 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:21.579605 12625 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:21.541465 14608 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:21.560465 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:21.588267 14608 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:21.589350 12629 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:21.591634 14479 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:21.593532 12782 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:18:21.594892 12782 raft_consensus.cc:606] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Received request to transfer leadership to TS 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:21.595484 12782 raft_consensus.cc:614] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:21.595707 14399 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:21.595935 14599 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:21.596380 14183 consensus_meta.cc:287] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.824s	user 0.014s	sys 0.002s
I20260709 13:18:21.510977 14586 consensus_queue.cc:260] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.597802 12774 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:21.552490 14292 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:21.562316 14558 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.602308 14590 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.563977 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:21.603032 14590 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.608732 14590 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:21.609702 14611 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.574244 14612 raft_consensus.cc:3055] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:21.589078 12527 catalog_manager.cc:5697] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 1 to 3, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 3 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:21.617851 12630 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:21.610409 14611 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.619128 14183 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:21.574755 14547 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.619580 14183 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:21.622731 14183 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:21.605283 14367 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.615692 13349 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:21.623214 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 59458.2us
I20260709 13:18:21.596554 14599 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.616258 14612 raft_consensus.cc:740] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:21.623749 14399 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:21.631294 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 62917.7us
I20260709 13:18:21.631047 14367 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.628417 12629 raft_consensus.cc:1275] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:21.617638 13377 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:21.635339 14145 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:21.638152 14145 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:21.666152 14547 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:21.668396 14349 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:21.623893 14398 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:18:21.663602 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 94037.1us
I20260709 13:18:21.599330 14441 consensus_queue.cc:237] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.675251 12625 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:18:21.676148 12625 raft_consensus.cc:606] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Received request to transfer leadership to TS 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:21.676677 12625 raft_consensus.cc:614] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:21.680765 14558 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:21.681902 14586 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:21.675379 14594 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.681599 14558 consensus_queue.cc:1059] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:21.688791 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:21.685642 14145 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:21.686709 14110 consensus_queue.cc:1059] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:21.690049 14599 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:21.689766 14398 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:21.684579 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:21.693583 14268 consensus_queue.cc:1059] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:21.694555 14613 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:21.683964 14399 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.701845 12630 raft_consensus.cc:2393] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:21.674475 14349 raft_consensus.cc:697] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:21.722198 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:21.731428 14561 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:21.732064 14561 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.732587 14561 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:21.733721 14594 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:21.739406 12673 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:21.739845 14602 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:21.700718 14588 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:21.742422 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 45947.7us
I20260709 13:18:21.742370 14611 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:21.743944 14410 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:21.744942 14410 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:21.745908 14399 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:21.748517 14564 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:21.749138 14564 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:21.749706 14564 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:21.689139 14583 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.750628 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:21.754046 14584 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:21.756886 14349 consensus_queue.cc:237] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.752889 14602 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:21.761904 14602 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:21.762352 14602 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:21.747850 14410 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:21.769771 12629 raft_consensus.cc:2393] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:21.761096 14183 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.757951 14367 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:21.728612 14547 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:18:21.778847 12597 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:21.727706 14426 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 13:18:21.779865 14478 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:21.786170 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:21.786512 14565 log.cc:927] Time spent T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 1.277s	user 0.008s	sys 0.008s
I20260709 13:18:21.788566 14378 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:21.791389 14508 consensus_meta.cc:287] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 1.226s	user 0.012s	sys 0.008s
I20260709 13:18:21.792168 14508 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.792989 14455 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:21.793718 14558 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.794387 14558 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.796607 14558 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:21.799741 12774 raft_consensus.cc:1275] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:21.801537 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:21.809544 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:21.809870 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:21.810485 12625 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:21.810889 12626 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:21.811774 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:21.812706 12632 raft_consensus.cc:2410] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:18:21.814132 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:21.814241 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:21.814855 12629 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:21.815721 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:21.815910 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:21.816589 12780 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:21.816713 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:21.817736 12777 raft_consensus.cc:1275] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:18:21.819145 14429 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.819777 14429 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:21.821121 14183 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:21.834273 12780 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:21.802507 14408 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:21.835893 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:21.836767 12630 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:21.836025 14410 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:21.838173 12747 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:21.838388 14410 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:21.838817 14410 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:21.756573 14612 consensus_queue.cc:260] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:21.842047 14416 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:21.842511 14612 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:21.834249 12673 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:21.845355 12779 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:21.775329 14590 consensus_queue.cc:1048] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 13:18:21.851061 14441 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.851760 14441 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.852453 14506 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:21.855356 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 85494us
I20260709 13:18:21.847260 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:21.857908 12630 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:21.858932 12673 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:21.846033 14594 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:21.852787 14564 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.854344 14145 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:21.855120 14561 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:21.890450 14378 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:21.857620 14183 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.891233 14608 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:18:21.894554 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 138552us
I20260709 13:18:21.895870 14183 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:21.896345 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:21.898154 13349 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:21.898476 14183 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:21.899394 14618 log.cc:927] Time spent T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:18:21.899698 14552 raft_consensus.cc:2955] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Committing config change with OpId 4.4: config changed from index 2 to 4, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:21.869748 14145 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:21.864275 14560 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.144s
I20260709 13:18:21.902515 14378 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:21.857858 14590 consensus_queue.cc:1243] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:18:21.908229 14591 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:21.908793 14588 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:21.909643 14508 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:21.909754 14489 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:21.909909 14460 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:21.910418 14441 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:21.891808 14608 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:21.913178 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:21.864101 14398 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:21.912257 12673 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:21.887174 14154 raft_consensus.cc:3055] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:21.929915 14496 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:21.930428 14496 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:21.929747 14460 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.931047 14496 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:21.932720 14460 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:21.867831 14479 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.933918 14479 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:21.935108 14583 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:21.936245 14479 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:21.922991 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:21.937081 12780 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:21.923236 14489 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:21.937924 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:21.938174 14489 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:21.924649 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:21.925659 14561 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:21.929814 14154 raft_consensus.cc:740] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:18:21.913209 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.913395 14145 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:21.940716 14608 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:21.859819 14455 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:21.941773 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:18:21.944779 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:21.945210 14619 log.cc:927] Time spent T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
W20260709 13:18:21.925439 14617 log.cc:927] Time spent T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
W20260709 13:18:21.948341 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 104944us
I20260709 13:18:21.951047 12779 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:18:21.952926 14395 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.000s	sys 0.009s
I20260709 13:18:21.953430 14590 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:21.946266 14154 consensus_queue.cc:260] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:21.961150 14154 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:21.962365 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:21.963376 12632 raft_consensus.cc:2393] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:21.949690 14429 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:21.966238 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:21.966989 12630 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:18:21.956784 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:21.967864 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.959954 12628 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:21.951615 12672 leader_election.cc:336] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Timed out: RequestConsensusVote RPC to 127.11.63.68:36021 timed out after 1.610s (SENT)
I20260709 13:18:21.952966 14505 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:18:21.956916 14395 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.000s	sys 0.010s
I20260709 13:18:21.972473 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:21.972638 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:21.987279 12629 pending_rounds.cc:85] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 1
I20260709 13:18:21.992017 12628 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:21.993454 14429 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:21.994450 14479 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:18:21.994951 14400 consensus_meta.cc:287] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.541s	user 0.014s	sys 0.000s
I20260709 13:18:21.995990 12634 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:21.997044 14292 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:21.997824 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 53671.2us
I20260709 13:18:21.998245 12633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56832 (request call id 1786) took 2322 ms. Trace:
I20260709 13:18:22.006601 12633 rpcz_store.cc:276] 0709 13:18:19.675860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.689933 (+ 14073us) service_pool.cc:224] Handling call
0709 13:18:21.997915 (+2307982us) spinlock_profiling.cc:243] Waited 1.94 s on lock 0x7b54001590d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:21.998064 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4454974336}
I20260709 13:18:21.998826 12631 raft_consensus.cc:3055] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:21.999779 14505 raft_consensus.cc:697] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:22.008033 12631 raft_consensus.cc:740] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:22.008893 12631 consensus_queue.cc:260] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:22.009876 12631 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:22.010258 14026 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:22.012883 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:22.000712 14564 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:22.013285 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:21.972649 14590 raft_consensus.cc:697] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:22.013926 12700 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:22.014575 14590 consensus_queue.cc:237] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:21.993902 14360 consensus_meta.cc:287] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.505s	user 0.005s	sys 0.010s
I20260709 13:18:22.017982 14360 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:22.018723 14410 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:22.020761 14562 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:22.000586 14394 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:22.005756 14611 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.021086 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:22.022120 14611 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:22.005722 14183 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.008767 14505 consensus_queue.cc:237] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:22.026021 14564 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.026728 14564 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:22.020592 14400 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:22.024225 14611 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:22.004374 12598 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:21.970547 14496 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:21.998682 14455 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:22.032390 14564 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:22.032912 14026 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:22.039237 14496 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:22.033298 14336 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:22.040122 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 70867.6us
I20260709 13:18:22.040580 14336 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:22.041064 12747 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:22.041296 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:22.039865 14026 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:21.977725 14602 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:22.042065 12774 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:22.044759 12778 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:22.044374 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:22.021131 14360 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:22.050034 12700 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:22.044373 12673 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:22.029533 12629 pending_rounds.cc:107] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260709 13:18:22.052529 12629 raft_consensus.cc:2889] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:22.043911 12747 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:22.056104 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.060801 12628 raft_consensus.cc:2393] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:22.043411 14026 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:22.065421 12625 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:18:22.065544 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:22.066223 12625 raft_consensus.cc:606] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Received request to transfer leadership to TS 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:22.061967 14564 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:22.074383 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:22.078280 14378 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:22.078454 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:22.078897 14378 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:22.079329 14378 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:22.082995 13377 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:22.084669 12780 raft_consensus.cc:1240] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:18:22.043205 14336 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:22.091288 12779 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:22.094566 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:22.098510 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:22.112814 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:22.112807 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:22.114953 12625 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 6: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:22.117344 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:22.118101 12700 raft_consensus.cc:2393] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 3: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:22.123317 12627 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:22.097277 14560 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:22.165472 14560 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:22.165941 14560 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:22.166447 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:22.167330 12700 raft_consensus.cc:2410] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:18:22.169153 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:22.169549 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:22.170120 12702 raft_consensus.cc:2410] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:22.170311 12705 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:22.171483 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:22.172362 12706 raft_consensus.cc:2393] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 5: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
W20260709 13:18:22.172466 14622 log.cc:927] Time spent T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
I20260709 13:18:22.171443 12700 raft_consensus.cc:1240] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 4. Ops: []
W20260709 13:18:22.175417 13348 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.744s (SENT) [suppressed 27 similar messages]
W20260709 13:18:22.176429 12627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56838 (request call id 1029) took 1625 ms (client timeout 1760 ms). Trace:
W20260709 13:18:22.176788 12627 rpcz_store.cc:269] 0709 13:18:20.550583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.552784 (+  2201us) service_pool.cc:224] Handling call
0709 13:18:22.176207 (+1623423us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b54002cdb54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:22.176358 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3509948160}
I20260709 13:18:22.069136 14183 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:22.179509 14183 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:22.181080 12527 catalog_manager.cc:5697] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 3 to 4, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:22.129729 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:22.130729 13382 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:22.070818 14400 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:22.138079 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:22.139578 14410 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:22.094187 14608 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:22.143977 13383 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:22.144101 12628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d46747dede234934bea0f0c324106c28"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:18:22.146209 14145 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:22.147897 14489 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:22.148862 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 105011us
I20260709 13:18:22.149916 14602 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:22.154269 12708 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:22.156950 14268 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:22.161724 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 117491us
I20260709 13:18:22.164981 14506 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.127040 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:22.181080 14591 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:22.196393 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.197397 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:22.199990 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:22.200793 12783 raft_consensus.cc:1240] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:22.200847 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:22.214578 12632 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:22.225344 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:22.234807 12628 raft_consensus.cc:606] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Received request to transfer leadership to TS 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:22.243919 12628 raft_consensus.cc:614] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:22.244226 12778 raft_consensus.cc:2410] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:18:22.238746 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:22.243222 12782 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:22.254916 12707 raft_consensus.cc:2393] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 5: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:22.239210 14183 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.262075 12774 raft_consensus.cc:2410] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:22.262413 13384 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:22.261482 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:22.269793 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:22.270501 12782 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 2.
I20260709 13:18:22.276995 13385 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:22.278947 12779 raft_consensus.cc:2393] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 4: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:22.281544 12628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:18:22.282199 12628 raft_consensus.cc:606] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Received request to transfer leadership to TS 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:22.282521 12628 raft_consensus.cc:614] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:22.282903 14608 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:22.254079 14268 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:22.285845 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:22.285725 14429 consensus_queue.cc:1048] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:22.286372 12673 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:22.286585 12706 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:22.288082 14566 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:22.288607 14566 raft_consensus.cc:697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:22.290061 12782 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:22.291586 14602 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:22.292220 14602 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:22.295459 14400 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:22.295830 14602 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:22.296130 12628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:18:22.299567 14586 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:22.301525 12747 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:22.301748 14183 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:22.302469 14566 consensus_queue.cc:237] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:22.307453 14145 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:22.307952 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:22.308369 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:22.308674 12776 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:22.309181 12777 raft_consensus.cc:2393] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 3: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:22.310174 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:22.309451 12783 raft_consensus.cc:2955] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index 2 to 4, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:22.310829 12774 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:22.311864 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:22.312750 12779 raft_consensus.cc:2410] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:22.313387 14317 raft_consensus.cc:3055] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:22.313803 14317 raft_consensus.cc:740] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:22.313971 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:22.314561 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.314888 12778 raft_consensus.cc:2410] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:22.314600 14317 consensus_queue.cc:260] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:22.316675 14317 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:22.318785 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:22.266486 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:22.319654 14591 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:22.260028 14460 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:22.320664 12628 raft_consensus.cc:606] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Received request to transfer leadership to TS 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:22.321096 12628 raft_consensus.cc:614] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:22.320921 14460 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:22.324249 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:22.324551 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:22.325448 12702 raft_consensus.cc:2410] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:22.329491 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:18:22.335528 14591 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:22.337412 12529 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54800 (request call id 228) took 10342 ms. Trace:
I20260709 13:18:22.337541 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:22.338510 12703 raft_consensus.cc:2410] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:22.338768 14378 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:22.315828 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:22.341099 12780 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:22.342289 12527 catalog_manager.cc:5697] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 1 to 3, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 3 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:18:22.276029 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.337818 12529 rpcz_store.cc:276] 0709 13:18:11.994955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.995238 (+   283us) service_pool.cc:224] Handling call
0709 13:18:12.019097 (+ 23859us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b5400002210. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28e0b2 00007f4c73207a89 00007f4c7319fda6 00007f4c73171088 00007f4c7326f37e 00007f4c717302a4 00007f4c71730232 00007f4c71730161 00007f4c717300dc
0709 13:18:12.307438 (+288341us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b540011eed4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31c090 00007f4c6d3baf3a 00007f4c731711cc 00007f4c7326f37e 00007f4c717302a4 00007f4c71730232 00007f4c71730161 00007f4c717300dc 00007f4c7172f392 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:22.337191 (+10029753us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:19.629058 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:19.629635 (+   577us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:19.629672 (+    37us) write_op.cc:435] Acquired schema lock
0709 13:18:19.629691 (+    19us) tablet.cc:662] Decoding operations
0709 13:18:19.632699 (+  3008us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:19.632785 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:19.632816 (+    31us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:18:19.633250 (+   434us) tablet.cc:701] Row locks acquired
0709 13:18:19.633273 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:18:19.633433 (+   160us) write_op.cc:270] Start()
0709 13:18:19.633551 (+   118us) write_op.cc:276] Timestamp: P: 1783603099633384 usec, L: 0
0709 13:18:19.633591 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 13:18:19.634280 (+   689us) log.cc:844] Serialized 1741 byte log entry
0709 13:18:19.693392 (+ 59112us) op_driver.cc:464] REPLICATION: finished
0709 13:18:19.705455 (+ 12063us) write_op.cc:301] APPLY: starting
0709 13:18:19.705616 (+   161us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:19.705794 (+   178us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:19.705963 (+   169us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:19.706803 (+   840us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:19.707011 (+   208us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:18:19.707069 (+    58us) write_op.cc:312] APPLY: finished
0709 13:18:19.707370 (+   301us) log.cc:844] Serialized 41 byte log entry
0709 13:18:19.707552 (+   182us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:19.707670 (+   118us) write_op.cc:454] Released schema lock
0709 13:18:19.707716 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:19.709740 (+  2024us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:19.709773 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":11290,"reported_tablets":13,"spinlock_wait_cycles":650270976,"child_traces":[["op",{"apply.queue_time_us":11586,"apply.run_cpu_time_us":2646,"apply.run_wall_time_us":4216,"num_ops":4,"prepare.queue_time_us":7246,"prepare.run_cpu_time_us":9987,"prepare.run_wall_time_us":13178,"raft.queue_time_us":48801,"raft.run_cpu_time_us":10078,"raft.run_wall_time_us":11960,"replication_time_us":59663,"spinlock_wait_cycles":489088,"thread_start_us":71138,"threads_started":4,"wal-append.queue_time_us":7885,"wal-append.run_cpu_time_us":39492,"wal-append.run_wall_time_us":1098781}]]}
I20260709 13:18:22.346299 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:22.346980 12706 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:22.346334 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:22.348441 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:22.344206 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:22.346299 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:22.353454 12708 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:22.317344 14489 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:22.362772 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:22.370069 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:22.253882 14506 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:22.373944 12780 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:22.383850 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:22.382956 12702 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:22.341012 14378 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:22.305306 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:22.393769 12628 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 13:18:22.396720 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.398483 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:22.397579 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:22.400072 12780 raft_consensus.cc:2410] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:18:22.395268 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 47694.9us
I20260709 13:18:22.407545 12672 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:22.376884 14608 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.409101 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:22.417243 12703 raft_consensus.cc:2410] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:22.417668 12627 raft_consensus.cc:1275] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:22.418596 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:22.419337 14631 log.cc:927] Time spent T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:18:22.420897 12632 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:22.421697 13378 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:22.422508 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:22.438848 12631 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:22.426156 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:22.426364 12597 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:22.426903 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:22.426998 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:22.441221 12782 raft_consensus.cc:2393] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 5: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:22.413889 13384 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:22.437119 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:22.437659 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 118984us
I20260709 13:18:22.440502 12629 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:22.447850 12672 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:22.429608 14361 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:22.455585 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:22.459582 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.459184 12673 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:22.459233 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:22.468807 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:22.474987 12632 raft_consensus.cc:1240] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:22.445504 14183 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:22.447388 14611 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.448945 14395 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:22.430677 14590 consensus_queue.cc:1048] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 13:18:22.488117 12672 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:22.468106 14441 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:22.467371 14394 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:22.505836 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:22.482046 14154 consensus_queue.cc:1059] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:18:22.504302 14183 consensus_queue.cc:1059] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:18:22.499511 14611 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:22.500814 14145 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:22.510723 14145 raft_consensus.cc:697] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:22.511888 14145 consensus_queue.cc:237] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:22.509876 14588 consensus_queue.cc:1059] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:22.471599 14400 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.466209 14608 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:22.525029 14336 raft_consensus.cc:3055] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:22.513407 14590 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:22.471167 14591 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
W20260709 13:18:22.526003 14552 consensus_meta.cc:287] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.551s	user 0.007s	sys 0.004s
I20260709 13:18:22.470625 14317 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:18:22.504182 14394 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:22.504544 14623 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:18:22.543883 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 80572.5us
I20260709 13:18:22.481915 14489 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:18:22.476290 14292 consensus_queue.cc:1048] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:18:22.478250 14604 consensus_queue.cc:1048] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
W20260709 13:18:22.550848 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 47993.6us
I20260709 13:18:22.551637 12707 raft_consensus.cc:2410] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:18:22.526330 14629 log.cc:927] Time spent T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.212s	user 0.000s	sys 0.002s
I20260709 13:18:22.515691 14441 raft_consensus.cc:697] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:22.537740 14336 raft_consensus.cc:740] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:22.534369 12631 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:22.550890 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:22.526034 14154 raft_consensus.cc:3055] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:22.548960 14508 raft_consensus.cc:3055] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:22.553318 14590 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:22.570508 12628 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:22.535493 14400 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:22.575552 12597 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:22.562423 14394 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:22.598062 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:22.602674 12627 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:22.571576 14154 raft_consensus.cc:740] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:22.573958 14623 raft_consensus.cc:697] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:22.581496 14590 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:22.611066 13385 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:22.574905 14508 raft_consensus.cc:740] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:22.608628 12597 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:18:22.606987 14026 consensus_meta.cc:287] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.541s	user 0.011s	sys 0.000s
I20260709 13:18:22.625475 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:22.630892 14608 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:22.638082 14336 consensus_queue.cc:260] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:22.646561 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 68791.6us
I20260709 13:18:22.622150 14479 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.611944 14506 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:22.615978 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:22.657101 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 48555.9us
I20260709 13:18:22.667866 12708 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:22.607708 14410 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:22.604085 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:22.678711 12597 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:18:22.681584 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.665899 14591 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:22.641680 14026 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:22.665642 14292 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:22.669798 14378 consensus_queue.cc:1059] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:18:22.625582 14508 consensus_queue.cc:260] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:22.680174 13376 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:22.667723 14441 consensus_queue.cc:237] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:22.677693 14410 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:22.732182 12629 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
W20260709 13:18:22.733353 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 233005us
W20260709 13:18:22.735024 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 247022us
I20260709 13:18:22.685984 12597 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:18:22.739809 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 53906.6us
W20260709 13:18:22.741875 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 74135.5us
I20260709 13:18:22.739388 12672 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:22.711994 14560 consensus_meta.cc:287] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.544s	user 0.011s	sys 0.000s
I20260709 13:18:22.683215 14408 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:22.756512 12629 raft_consensus.cc:1275] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 2. (index mismatch)
I20260709 13:18:22.687275 14591 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:22.764189 12626 raft_consensus.cc:2393] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:22.764354 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:22.681938 14479 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:22.766278 12632 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:22.765504 12747 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:22.684335 13377 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:22.766867 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:22.769625 12633 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:22.751305 14183 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:22.743754 14410 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:22.676769 14398 consensus_queue.cc:1059] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 13:18:22.770653 14400 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:22.772019 12747 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:22.772552 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.775403 13349 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:22.777035 14608 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:22.779415 14508 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:22.782526 14460 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:22.784271 14586 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:22.762074 14395 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:22.698061 14154 consensus_queue.cc:260] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:22.767643 14591 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:22.756346 14408 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:22.800233 13384 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:22.765708 14604 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:18:22.756834 14560 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:22.763031 13382 raft_consensus.cc:2955] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index 2 to 4, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:22.792446 14360 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:22.786542 14183 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:22.726969 14623 consensus_queue.cc:237] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:22.795298 14317 raft_consensus.cc:3055] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:22.813743 14378 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:18:22.829067 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 57142.7us
I20260709 13:18:22.816012 14408 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:22.820677 14183 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:22.819562 14360 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:22.792693 14292 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:22.840485 12625 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:22.799770 14608 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:22.842978 12747 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:22.829097 14317 raft_consensus.cc:740] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:22.818735 14505 consensus_queue.cc:1059] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:18:22.840344 14154 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:22.881835 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.841324 14360 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:22.887440 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 118638us
I20260709 13:18:22.852867 14564 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:22.896274 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:22.900453 12633 raft_consensus.cc:1240] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:22.901544 12779 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:22.884567 14586 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:22.906944 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:22.907648 12774 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:22.909087 13384 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:22.904491 12625 pending_rounds.cc:85] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 1
I20260709 13:18:22.904727 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:22.909912 12625 pending_rounds.cc:107] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:18:22.910262 12625 raft_consensus.cc:2889] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:22.912667 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:22.908604 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:22.922899 14317 consensus_queue.cc:260] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:22.924458 14317 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:22.926031 12783 raft_consensus.cc:2410] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:18:22.928365 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 176738us
I20260709 13:18:22.929616 12782 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:22.907539 14026 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:22.927690 14611 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:22.966050 14571 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.034s	sys 0.037s
I20260709 13:18:22.991016 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:22.999677 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:23.004302 14647 log.cc:927] Time spent T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
W20260709 13:18:22.994155 14571 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.034s	sys 0.038s
W20260709 13:18:23.012065 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 102570us
W20260709 13:18:23.012872 13348 leader_election.cc:336] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.533s (SENT)
W20260709 13:18:23.015266 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:23.018996 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.020110 13348 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:22.999104 14562 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.021740 14611 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:23.036936 14643 log.cc:927] Time spent T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
I20260709 13:18:23.036976 14611 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:23.039691 14611 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:23.042202 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 114454us
I20260709 13:18:23.043969 12598 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:23.016126 14479 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:23.018822 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:18:23.055141 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 53403.8us
I20260709 13:18:23.058018 14352 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:23.061642 14292 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:23.079900 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.080834 12778 raft_consensus.cc:2393] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 5: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
W20260709 13:18:23.081156 14651 log.cc:927] Time spent T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.120s	user 0.000s	sys 0.002s
I20260709 13:18:23.082310 14268 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:23.082731 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.084017 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.084646 12779 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:23.085214 14336 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:23.087934 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:23.088869 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.090564 12782 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:23.091004 12774 raft_consensus.cc:2393] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 4: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:23.092764 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.093899 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:23.094340 12778 raft_consensus.cc:2410] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:18:23.100803 12779 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
I20260709 13:18:23.054198 14608 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:23.103456 12775 raft_consensus.cc:1240] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:23.106387 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.111955 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:23.112969 12625 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:23.114454 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:23.115265 12778 raft_consensus.cc:2393] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 5: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:23.117174 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:23.118920 13348 leader_election.cc:336] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.630s (SENT)
I20260709 13:18:23.121555 12778 raft_consensus.cc:1240] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:23.119652 13348 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:23.129294 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:23.132210 12775 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
W20260709 13:18:23.140793 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 129883us
I20260709 13:18:23.140913 12783 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 4: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:23.047722 14562 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:23.147651 12777 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:23.151178 14562 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:23.151352 12704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1423) took 1034 ms. Trace:
I20260709 13:18:23.151908 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.152356 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.152995 12778 raft_consensus.cc:2393] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 5: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:23.154412 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.155293 12779 raft_consensus.cc:2376] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5. Re-sending same reply.
I20260709 13:18:23.156424 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:18:23.139886 14394 consensus_meta.cc:287] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.529s	user 0.002s	sys 0.009s
I20260709 13:18:23.157464 12775 raft_consensus.cc:2393] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 4: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:23.158263 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:23.160673 12783 raft_consensus.cc:2410] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:18:23.161244 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:23.157909 14352 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:18:23.167024 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.145884 14588 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:23.193701 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 74886.6us
I20260709 13:18:23.145306 14591 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:23.151760 12704 rpcz_store.cc:276] 0709 13:18:22.116491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.165797 (+ 49306us) service_pool.cc:224] Handling call
0709 13:18:22.166326 (+   529us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:22.198473 (+ 32147us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:22.374998 (+176525us) log.cc:844] Serialized 14 byte log entry
0709 13:18:22.737880 (+362882us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:23.127654 (+389774us) spinlock_profiling.cc:243] Waited 4.84 ms on lock 0x7b2c00011ed8. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6bcaba11 00007f4c6bcbb372 00007f4c6d3eda01 00007f4c6d3b243a 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2
0709 13:18:23.148645 (+ 20991us) log.cc:844] Serialized 52 byte log entry
0709 13:18:23.148983 (+   338us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:23.149077 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.149195 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:23.150943 (+  1748us) raft_consensus.cc:1685] finished
0709 13:18:23.151036 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.151177 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30758,"prepare.queue_time_us":76795,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":413,"raft.queue_time_us":32849,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":14805,"spinlock_wait_cycles":11135744,"thread_start_us":479004,"threads_started":2,"wal-append.queue_time_us":400420}
I20260709 13:18:23.201437 12628 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:23.203419 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:23.158113 14410 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:23.207651 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:23.160451 14394 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:23.209376 12783 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:18:23.215485 12627 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:18:23.226466 12775 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:23.227695 14394 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:23.229677 14479 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.230276 14479 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:23.231854 12630 raft_consensus.cc:1240] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:23.232753 14591 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:23.234247 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.234719 12672 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:23.235816 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.236459 12779 raft_consensus.cc:2376] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5. Re-sending same reply.
I20260709 13:18:23.237732 14508 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:23.238948 14410 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:23.184602 14588 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:23.239298 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.240187 12779 raft_consensus.cc:2376] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3. Re-sending same reply.
W20260709 13:18:23.242086 14590 consensus_meta.cc:287] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.596s	user 0.012s	sys 0.000s
I20260709 13:18:23.242449 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:23.242512 14590 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:23.242118 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:23.242540 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.245482 12779 raft_consensus.cc:2393] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 5: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:23.245785 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:23.246367 12625 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:23.238972 12672 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:23.248256 14562 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:23.251260 12707 raft_consensus.cc:1240] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:18:23.201122 14145 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.251927 14145 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:23.202971 14608 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.252863 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:23.253671 12704 raft_consensus.cc:2393] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 3: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:23.254408 14562 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:23.254355 14608 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:23.256613 12672 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:23.262919 14645 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:18:23.263553 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 206864us
I20260709 13:18:23.263831 14608 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:23.264151 12672 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:23.264477 14552 consensus_queue.cc:1059] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:23.264973 14608 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:23.264178 12597 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:23.265357 14552 raft_consensus.cc:3055] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:23.265765 14552 raft_consensus.cc:740] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:23.266752 12597 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:23.266551 14552 consensus_queue.cc:260] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:23.267625 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:23.267722 14552 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:23.268600 14268 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:23.270004 14145 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:23.270315 14408 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:23.272266 12632 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:23.272359 14608 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:23.273644 14400 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:23.273437 14183 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:23.274163 14400 raft_consensus.cc:697] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:23.275601 14183 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:23.276275 12628 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:23.276885 12597 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:23.277851 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:23.278930 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:23.259333 12625 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
W20260709 13:18:23.261369 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 217491us
I20260709 13:18:23.281431 12775 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:23.261864 14292 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:23.261787 14398 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:23.245383 14590 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:23.283599 12707 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:23.282171 12598 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:23.285426 12709 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:23.284943 14154 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:23.293597 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:23.293414 14154 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.293852 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:23.294193 12705 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:23.294430 14154 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:23.294800 12706 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 6: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:23.296489 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:23.296867 12702 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:18:23.294138 14505 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
W20260709 13:18:23.297964 13348 leader_election.cc:336] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.789s (SENT)
I20260709 13:18:23.298256 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:23.298892 12702 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:23.280792 14410 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:23.299829 14410 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.300202 14410 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:23.301352 12707 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:18:23.302817 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:23.303779 12707 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:23.306531 14154 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:23.308990 14154 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:23.310398 14360 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:23.311654 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:23.311829 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:23.312461 13382 raft_consensus.cc:2393] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:23.312585 14360 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:23.312342 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:23.314474 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:23.315281 12709 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:23.316840 12597 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:23.321818 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:23.322511 12598 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:23.322898 14657 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:23.323770 12598 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:23.324865 12598 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:23.275244 14400 consensus_queue.cc:237] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:23.272397 14408 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:23.326377 14336 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:23.326977 14660 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:23.328073 12626 raft_consensus.cc:1240] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:23.328853 12627 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:23.339277 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:23.333303 12598 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:23.334892 12705 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:23.343067 12598 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:23.344527 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:23.345428 12704 raft_consensus.cc:2393] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 4: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:23.346802 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:23.352756 13382 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:23.352577 12598 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:23.350020 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:23.353893 12709 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:23.355368 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:23.355962 12705 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:23.356863 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:23.357429 12705 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:23.359519 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:23.315040 14268 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:23.363106 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:23.332829 14398 raft_consensus.cc:697] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:23.371091 13383 raft_consensus.cc:2393] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 5: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:23.347543 14026 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:23.351017 14654 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:23.376132 12705 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:23.358696 14394 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:23.378304 12625 raft_consensus.cc:1240] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:23.377640 12704 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:23.362041 14360 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:23.365906 14183 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:23.371050 14336 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:23.387660 14410 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:23.385717 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:23.371280 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:18:23.399757 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.400841 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:23.402158 13349 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:23.409490 12627 raft_consensus.cc:2393] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 5: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:23.411737 12709 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
I20260709 13:18:23.418078 13349 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:23.407301 14360 consensus_queue.cc:1059] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:18:23.368230 14408 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:18:23.382261 14394 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.427537 14183 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:23.428072 14183 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:23.428157 14564 consensus_queue.cc:1059] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:23.429445 14317 raft_consensus.cc:3055] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:23.425160 14352 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:18:23.430104 14317 raft_consensus.cc:740] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:23.431022 14317 consensus_queue.cc:260] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:23.432008 14398 consensus_queue.cc:237] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:23.432763 14317 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:23.423740 14506 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:23.434011 14506 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.434378 14506 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:23.437834 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 85344.3us
I20260709 13:18:23.438972 12598 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:23.440503 12598 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:23.442471 14654 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:23.424583 14505 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:18:23.442970 14644 log.cc:927] Time spent T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.450s	user 0.003s	sys 0.000s
I20260709 13:18:23.423857 14394 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:23.447412 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.448665 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.448828 12598 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:23.450675 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.452286 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.452596 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.453090 12776 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:23.461972 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.463284 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.455130 12746 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:23.475554 12775 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:23.476945 12774 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:23.476817 12598 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:23.478415 12598 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:23.479419 14336 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:23.479984 14336 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:23.480445 14336 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:23.491185 14588 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:23.493760 14360 consensus_queue.cc:1059] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:23.494352 14560 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:23.494741 14360 consensus_queue.cc:1059] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:23.495649 14360 consensus_queue.cc:1059] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:23.478816 14154 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:18:23.499294 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 44280.2us
I20260709 13:18:23.501384 12598 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:23.502722 12597 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:23.513989 14352 raft_consensus.cc:3055] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:23.514858 14479 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:23.516204 14352 raft_consensus.cc:740] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:23.512349 14613 raft_consensus.cc:3055] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:23.512452 14571 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:23.518565 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:23.498996 14268 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:23.515266 12775 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:23.532426 12776 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:23.448710 14489 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 13:18:23.536651 12597 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:23.517295 14352 consensus_queue.cc:260] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:23.541450 14352 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:23.541551 14655 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.542368 14655 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:23.544766 14655 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:23.547643 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 46378us
W20260709 13:18:23.549798 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.545046 14394 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:23.517488 14671 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:23.524461 14654 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:23.521790 14410 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:23.556529 12597 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:23.533126 14613 raft_consensus.cc:740] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:23.570977 14654 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:23.571729 14654 raft_consensus.cc:697] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:23.572105 14613 consensus_queue.cc:260] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:23.572736 14654 consensus_queue.cc:237] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:23.535251 14571 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.573952 14571 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:23.563431 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.560096 12598 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:23.571076 12597 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:23.547585 14154 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:23.515156 14183 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:23.579304 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:23.579542 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 87264.3us
I20260709 13:18:23.557704 14591 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:23.585147 12598 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:23.585960 13348 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:23.573472 14613 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:23.561185 14667 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:23.601058 12598 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:23.579612 14352 raft_consensus.cc:3055] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:23.580371 14400 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:23.603536 14352 raft_consensus.cc:740] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:23.584520 14268 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:23.576458 14408 consensus_queue.cc:1059] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:18:23.608862 14671 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:23.588182 14317 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:23.609673 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 190254us
I20260709 13:18:23.616935 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:23.603778 14154 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:23.609308 14352 consensus_queue.cc:260] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:23.619251 14154 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.619699 14154 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:23.619768 14352 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:23.622610 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:23.623283 12700 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:23.624590 14506 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:23.617059 14400 raft_consensus.cc:697] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:23.626660 14400 consensus_queue.cc:237] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:23.627343 14678 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:18:23.630601 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.628747 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:23.602254 14591 raft_consensus.cc:697] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:23.647294 12700 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
W20260709 13:18:23.671278 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.670365 13349 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:23.673775 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.673792 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:23.675735 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:23.675789 12705 raft_consensus.cc:2393] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 5: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:23.617679 14680 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:18:23.674427 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 88547.1us
I20260709 13:18:23.679839 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:23.679724 14678 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:23.681131 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:23.681694 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.682036 12780 raft_consensus.cc:2410] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:23.683300 12776 raft_consensus.cc:1240] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:23.639108 14336 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:23.685636 12776 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:23.686120 14336 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:23.687347 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.690120 14678 raft_consensus.cc:697] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:23.691170 14678 consensus_queue.cc:237] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:23.694643 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:23.695351 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.697669 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:23.698189 12707 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:18:23.698289 12783 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:23.699297 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.699437 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:23.700496 12709 raft_consensus.cc:2410] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:18:23.700474 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.713627 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:23.716662 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:23.698203 14677 log.cc:927] Time spent T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260709 13:18:23.701099 14675 log.cc:927] Time spent T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:18:23.718346 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:23.720150 12707 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:23.725812 12777 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:23.725832 13348 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:23.722872 12775 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:23.639586 14317 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:23.724126 12709 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:23.727634 12780 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:23.728070 12702 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:23.724769 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:23.695290 14394 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:23.733170 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:23.733764 12708 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:23.724570 14679 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:18:23.738654 14591 consensus_queue.cc:237] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:23.749308 12708 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:23.752051 12707 raft_consensus.cc:1240] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 13:18:23.758157 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:23.758268 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:23.758541 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:23.759353 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:23.759541 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:23.760293 12705 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:23.760485 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:23.770396 14506 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:23.771252 14317 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:23.772368 12775 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:23.716567 14680 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:23.774206 14154 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:23.775034 14680 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:23.776445 14154 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:23.758196 12709 raft_consensus.cc:1240] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3. Ops: []
W20260709 13:18:23.781322 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.782449 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:23.783573 12709 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:23.724104 14336 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:23.759619 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:23.772280 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:23.786398 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:23.790196 12707 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:23.794986 12783 raft_consensus.cc:2393] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:23.795182 12705 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:23.794927 12708 raft_consensus.cc:2393] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 5: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
W20260709 13:18:23.797546 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 90757.5us
I20260709 13:18:23.804515 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:23.805526 12702 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:23.807536 12709 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:18:23.802692 14336 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:23.813061 14336 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:23.810369 12702 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:23.836091 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:23.814697 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:23.836922 12634 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:23.818430 13348 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:23.838300 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:23.839416 13348 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:23.840382 14506 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:23.841223 14506 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:23.794378 14154 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.796983 14613 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:23.843099 14154 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:23.769084 14571 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:23.844303 12627 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:23.845717 12779 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:18:23.830925 12706 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:23.832695 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:23.847728 12775 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:23.833465 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.848935 12774 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:23.834107 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:23.834950 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:23.850028 12777 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:23.834846 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:23.852636 12625 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:23.835234 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:23.850210 12633 raft_consensus.cc:2393] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:23.853633 14571 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:23.853892 12630 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:23.835427 12782 raft_consensus.cc:3055] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:23.854640 12782 raft_consensus.cc:740] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:23.855449 12782 consensus_queue.cc:260] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:23.856379 12782 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:23.858623 12634 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:23.845466 14154 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:23.842931 14506 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:18:23.874281 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.879355 12775 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:23.867678 12673 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:23.893721 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.897208 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.899783 12747 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:23.901221 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:23.901912 12781 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:23.904845 12706 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:23.894726 12673 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:23.905972 12775 raft_consensus.cc:1240] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:23.878894 14460 raft_consensus.cc:3055] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:23.907276 12673 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:23.912056 14560 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:23.912753 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:23.919171 12673 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:23.893821 14571 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:23.902814 14602 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:23.907886 14506 raft_consensus.cc:697] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:18:23.939523 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 54361.9us
I20260709 13:18:23.918135 14523 raft_consensus.cc:3055] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:23.896164 14154 consensus_queue.cc:1059] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:18:23.948432 12747 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:23.912317 14689 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:23.949362 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 85194.6us
I20260709 13:18:23.918409 14560 raft_consensus.cc:697] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:23.933509 14588 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:23.913316 14460 raft_consensus.cc:740] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:18:23.955087 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.938385 14590 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:23.917999 14479 consensus_queue.cc:1048] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
W20260709 13:18:24.010428 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.948024 14523 raft_consensus.cc:740] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:23.953285 14588 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.938083 14571 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:23.994408 14590 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.023954 14560 consensus_queue.cc:237] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:24.024883 14571 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:24.012748 13348 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:23.947727 14602 raft_consensus.cc:697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:18:24.013609 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 65276us
I20260709 13:18:23.997282 14680 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:24.039327 14680 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:24.039930 14680 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:24.021819 14588 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:24.025995 14590 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:24.023976 14460 consensus_queue.cc:260] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:24.051402 14460 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:24.037917 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:24.054052 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 126777us
I20260709 13:18:24.054666 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:24.055238 12774 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:24.066268 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:24.067189 12783 raft_consensus.cc:2393] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:24.063866 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:24.054260 14154 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:24.076077 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 63392.3us
I20260709 13:18:24.078504 12781 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:24.080215 14523 consensus_queue.cc:260] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:24.060060 14613 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:24.085651 12597 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:24.088642 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:24.092424 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.064174 14571 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:24.073909 14698 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.106130 12747 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:24.107925 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:24.109545 12776 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:24.110175 13379 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:18:24.111109 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 49750.2us
I20260709 13:18:24.092157 14682 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:24.093344 14523 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:24.113274 12673 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:24.093140 14657 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:18:24.116230 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.113534 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:24.116537 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:24.074798 14562 consensus_queue.cc:1059] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:18:24.098544 14571 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.117449 13383 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:24.084438 14154 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:24.100594 14692 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:24.061625 14506 consensus_queue.cc:237] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:24.116091 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:24.111852 14698 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:24.110551 14317 consensus_queue.cc:1059] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:18:24.116644 14682 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.160185 14692 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:24.163360 12747 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:24.150481 14571 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:24.156255 14688 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:24.108695 14687 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
W20260709 13:18:24.194797 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.150409 14654 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 13:18:24.179503 14154 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:24.194082 14682 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:24.117539 14602 consensus_queue.cc:237] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:24.171730 14564 consensus_queue.cc:1059] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:18:24.220486 14564 consensus_queue.cc:1243] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 639ab2db265f4747bb7afb93e833f530 log is divergent from this leader: its last log entry 5.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:18:24.221122 14564 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:24.221633 14564 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.222107 14564 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:24.173942 14588 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:24.231383 14684 consensus_queue.cc:1059] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.074s
I20260709 13:18:24.232832 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:24.233680 12627 raft_consensus.cc:2393] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 4: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:24.234050 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:24.237155 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:24.234611 12634 raft_consensus.cc:2376] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3. Re-sending same reply.
W20260709 13:18:24.242123 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 78863us
W20260709 13:18:24.247660 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 135772us
I20260709 13:18:24.233217 14562 raft_consensus.cc:3055] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:24.248777 12747 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:24.232831 14689 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:24.250540 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:24.232077 14698 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:24.255409 14562 raft_consensus.cc:740] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:24.257359 12634 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:24.258553 12703 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:18:24.263231 12673 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:24.264758 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:24.235737 14590 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:24.250041 14692 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:24.279230 14681 log.cc:927] Time spent T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.515s	user 0.007s	sys 0.000s
I20260709 13:18:24.270397 14689 raft_consensus.cc:3055] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:24.286032 14689 raft_consensus.cc:740] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:24.287282 14708 raft_consensus.cc:3055] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:24.287147 14689 consensus_queue.cc:260] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:24.288480 14689 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:24.291198 14590 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:24.287707 14708 raft_consensus.cc:740] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:18:24.292140 13349 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.014s	sys 0.020s
W20260709 13:18:24.292487 13349 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.014s	sys 0.020s
I20260709 13:18:24.292619 14708 consensus_queue.cc:260] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:24.294171 14708 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:24.296633 14183 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:24.298210 14588 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:24.300925 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 600208us
I20260709 13:18:24.261749 14704 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:24.302824 14704 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.303167 14704 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:24.258549 14552 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:18:24.305722 14686 log.cc:927] Time spent T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.533s	user 0.009s	sys 0.012s
I20260709 13:18:24.305825 14552 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:24.308038 14552 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:24.315361 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:24.316143 12626 raft_consensus.cc:2393] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 4: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:24.316927 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:24.317513 12634 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:24.321033 12633 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:24.022250 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:24.323478 14571 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:24.327201 14682 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:24.335106 14680 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:24.337311 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:24.337986 12634 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:24.339529 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:24.340081 12626 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:24.341094 12782 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:24.341500 12634 raft_consensus.cc:1240] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 13:18:24.341723 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 78600.2us
I20260709 13:18:24.342201 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:24.342523 12673 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:24.342849 12777 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:24.343166 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:24.345613 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:24.346216 12774 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:24.342849 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:24.346431 14613 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:24.347124 12625 raft_consensus.cc:2410] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:24.347951 14590 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:24.348500 14590 raft_consensus.cc:697] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:24.349478 14590 consensus_queue.cc:237] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:24.350302 14292 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:24.350737 14292 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:24.351392 14292 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:24.351469 14588 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:24.353621 14588 consensus_queue.cc:1059] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:24.354738 14701 raft_consensus.cc:3055] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:24.355309 14701 raft_consensus.cc:740] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:24.356187 14701 consensus_queue.cc:260] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:24.360201 12777 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:24.362538 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:24.364549 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:24.367573 14708 consensus_queue.cc:1059] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:24.372016 14708 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:24.368129 12781 raft_consensus.cc:1275] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 2. (term mismatch)
I20260709 13:18:24.370662 14571 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:24.369221 14654 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:24.402925 12778 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:18:24.372079 14479 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:24.373270 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.405098 12776 raft_consensus.cc:1275] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:24.368063 12627 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:24.377801 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:24.379599 12781 pending_rounds.cc:85] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 1
I20260709 13:18:24.382596 12672 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:24.409971 12781 pending_rounds.cc:107] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:18:24.410409 12781 raft_consensus.cc:2889] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:24.410848 12781 pending_rounds.cc:107] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:18:24.411154 12781 raft_consensus.cc:2889] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:24.384330 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:24.302420 13349 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:24.418319 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 115982us
I20260709 13:18:24.386077 12779 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:18:24.386935 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:24.421720 12628 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:24.422181 12779 pending_rounds.cc:85] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 1
I20260709 13:18:24.342105 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:18:24.403957 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 155257us
I20260709 13:18:24.422547 12779 pending_rounds.cc:107] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:18:24.424636 12779 raft_consensus.cc:2889] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:24.426080 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:24.426963 13383 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 2: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
W20260709 13:18:24.427080 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 44588.6us
W20260709 13:18:24.428937 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.428617 12672 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:24.430591 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:18:24.431689 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.430007 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:24.332532 14704 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:24.437465 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:24.441617 12625 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:24.378940 14708 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.439683 12672 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:24.442902 12628 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: []
W20260709 13:18:24.444087 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.402542 14698 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:24.456023 14703 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:18:24.454056 12527 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40528 (request call id 134) took 12342 ms (client timeout 15000 ms). Trace:
I20260709 13:18:24.457099 14678 consensus_queue.cc:1059] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:24.460649 14508 consensus_queue.cc:1059] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:18:24.457828 12527 rpcz_store.cc:269] 0709 13:18:12.111219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.144354 (+ 33135us) service_pool.cc:224] Handling call
0709 13:18:12.476077 (+331723us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b540011eed4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31c090 00007f4c6d3baf3a 00007f4c731711cc 00007f4c7326f37e 00007f4c717302a4 00007f4c71730232 00007f4c71730161 00007f4c717300dc 00007f4c7172f392 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:24.453976 (+11977899us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:22.922947 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:22.923637 (+   690us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:22.923684 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:18:22.923703 (+    19us) tablet.cc:662] Decoding operations
0709 13:18:22.924343 (+   640us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:22.924445 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:22.924472 (+    27us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:18:22.924923 (+   451us) tablet.cc:701] Row locks acquired
0709 13:18:22.924993 (+    70us) write_op.cc:260] PREPARE: finished
0709 13:18:22.925181 (+   188us) write_op.cc:270] Start()
0709 13:18:22.925324 (+   143us) write_op.cc:276] Timestamp: P: 1783603102925122 usec, L: 0
0709 13:18:22.925371 (+    47us) op_driver.cc:348] REPLICATION: starting
0709 13:18:23.020361 (+ 94990us) log.cc:844] Serialized 1355 byte log entry
0709 13:18:23.444909 (+424548us) op_driver.cc:464] REPLICATION: finished
0709 13:18:23.604646 (+159737us) write_op.cc:301] APPLY: starting
0709 13:18:23.604770 (+   124us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:23.604932 (+   162us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:23.605071 (+   139us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:23.609539 (+  4468us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:23.628120 (+ 18581us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:18:23.628192 (+    72us) write_op.cc:312] APPLY: finished
0709 13:18:23.628545 (+   353us) log.cc:844] Serialized 35 byte log entry
0709 13:18:23.628742 (+   197us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:23.628845 (+   103us) write_op.cc:454] Released schema lock
0709 13:18:23.628916 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:23.629377 (+   461us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:23.629408 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2141743,"reported_tablets":13,"spinlock_wait_cycles":315710848,"child_traces":[["op",{"apply.queue_time_us":143256,"apply.run_cpu_time_us":2974,"apply.run_wall_time_us":24704,"num_ops":3,"prepare.queue_time_us":367013,"prepare.run_cpu_time_us":53850,"prepare.run_wall_time_us":346872,"raft.queue_time_us":205326,"raft.run_cpu_time_us":24851,"raft.run_wall_time_us":138527,"replication_time_us":516316,"thread_start_us":647972,"threads_started":4,"wal-append.queue_time_us":165770}]]}
I20260709 13:18:24.462486 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:24.464881 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:24.430593 14613 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:24.470237 12747 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:24.472506 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.476300 12672 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:24.435634 14560 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:24.435801 14654 raft_consensus.cc:697] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:24.453655 14708 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:24.437873 14657 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.444659 14689 raft_consensus.cc:3055] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:24.485525 12628 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: []
W20260709 13:18:24.481444 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 137503us
I20260709 13:18:24.488298 12774 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:24.490074 14704 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:24.490520 14613 raft_consensus.cc:697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:24.449730 14679 consensus_queue.cc:1059] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:18:24.431159 14479 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:24.479053 14508 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:24.487473 12747 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:24.467166 14317 consensus_queue.cc:1243] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 639ab2db265f4747bb7afb93e833f530 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:18:24.506295 12626 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:24.437386 14571 consensus_queue.cc:1059] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:18:24.431025 14564 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:24.536651 14564 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:24.508466 12673 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:24.491035 14692 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:24.538194 14692 raft_consensus.cc:697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:24.509897 14711 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:24.539250 14692 consensus_queue.cc:237] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:24.541648 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:24.542230 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:24.542762 12634 raft_consensus.cc:1240] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:24.542941 12632 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 13:18:24.481650 14698 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:24.544507 12633 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: [3.5-5.6]
I20260709 13:18:24.545531 14571 consensus_queue.cc:1243] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 522fd2ba0e9f402691b5ef29af8579cd log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:18:24.545800 14707 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:24.546139 14571 raft_consensus.cc:3055] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:24.546530 14571 raft_consensus.cc:740] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:24.542285 12629 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:24.547317 14571 consensus_queue.cc:260] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:24.548247 14292 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:24.550632 14292 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:24.551026 14571 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:24.558269 12626 raft_consensus.cc:1240] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 13:18:24.510835 14714 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:24.481093 14687 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:18:24.444898 14154 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 13:18:24.568171 14564 raft_consensus.cc:2764] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:18:24.574753 14564 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.575217 14564 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:24.578915 14571 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:24.454751 14562 consensus_queue.cc:260] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:24.505851 14689 raft_consensus.cc:740] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:24.507777 14657 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:24.527158 12625 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:24.513801 14679 raft_consensus.cc:3055] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:24.529696 14479 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:24.577237 14564 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:24.513517 14613 consensus_queue.cc:237] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:24.560743 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 73366.4us
W20260709 13:18:24.602989 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 140574us
I20260709 13:18:24.567301 12628 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:24.559787 12673 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:24.581529 12626 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:24.589504 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:24.595826 12633 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:24.592512 14714 raft_consensus.cc:697] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:24.579598 14571 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:24.530695 14654 consensus_queue.cc:237] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:24.599383 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.546550 14698 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:24.597692 14679 raft_consensus.cc:740] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:18:24.609524 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 49833.6us
I20260709 13:18:24.617970 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:24.618495 12673 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:24.619660 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:24.619941 12709 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:24.612136 14479 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:18:24.619009 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.625255 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:24.627650 12706 raft_consensus.cc:2393] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 4: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:24.615494 12707 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:24.623065 12673 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:24.612913 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 136932us
I20260709 13:18:24.635267 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:24.636837 12747 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:24.637744 14711 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:24.617601 14562 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:24.641335 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:24.645702 12634 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:24.646991 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:24.647771 12634 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:24.664861 12709 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:24.658469 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:24.631148 14479 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.612520 14317 consensus_queue.cc:1059] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:18:24.619457 14292 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:18:24.629920 14560 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:24.657048 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:24.680413 12628 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:24.672467 12633 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:24.650888 14508 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:18:24.646297 14591 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.876s	user 0.003s	sys 0.028s
I20260709 13:18:24.674280 12598 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:24.651165 14590 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:24.614331 14687 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:18:24.629930 14564 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:24.671545 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:24.693879 12629 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:24.694927 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:24.695726 12629 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:24.696928 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:18:24.679060 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:24.674489 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 46142.7us
I20260709 13:18:24.676266 14479 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:24.672747 14460 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:24.709892 12629 raft_consensus.cc:2393] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
W20260709 13:18:24.682372 14591 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.912s	user 0.003s	sys 0.028s
W20260709 13:18:24.717178 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.680133 14704 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:24.691761 14508 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.680140 14560 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.722293 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:24.723002 12628 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:24.659576 14689 consensus_queue.cc:260] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:24.728667 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:24.729815 12630 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:24.728129 14724 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.693022 14698 consensus_queue.cc:1059] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:18:24.727825 12629 raft_consensus.cc:1275] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:24.713714 14714 consensus_queue.cc:237] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:24.720495 14154 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:24.748457 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:24.749325 12777 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 1.
I20260709 13:18:24.727255 14560 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:24.725317 14508 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:24.733377 14683 log.cc:927] Time spent T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.900s	user 0.004s	sys 0.005s
I20260709 13:18:24.753948 14736 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:24.757709 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 95787.9us
W20260709 13:18:24.759541 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 124506us
I20260709 13:18:24.763263 14724 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:24.763746 14729 consensus_queue.cc:1048] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:24.705195 14590 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:24.775298 12673 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:24.788419 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 59813.1us
I20260709 13:18:24.789032 12673 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:24.776731 12672 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:24.765152 14713 log.cc:927] Time spent T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:18:24.733435 14710 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:24.732988 14702 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:24.789413 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:24.712595 14564 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:24.720750 14460 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:24.800719 12673 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:24.803273 12672 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:24.692238 14679 consensus_queue.cc:260] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:24.733779 14720 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.153s
I20260709 13:18:24.804397 14689 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:24.800357 14292 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:18:24.843582 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 84594.8us
I20260709 13:18:24.854858 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:24.808997 14571 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:24.809221 14317 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:24.810552 14698 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:24.840091 14360 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.811997 14702 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:18:24.871289 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.812744 14704 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:24.808320 14154 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 13:18:24.852638 14687 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:24.843425 14657 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:24.859535 14722 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.857949 14292 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:24.894021 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 104715us
W20260709 13:18:24.881628 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 78435.6us
I20260709 13:18:24.874718 14698 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.888809 14708 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:24.874745 14679 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:24.908659 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:24.901728 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 46969.3us
I20260709 13:18:24.797772 14736 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:24.916749 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:24.910353 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 99756.3us
I20260709 13:18:24.920246 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:24.809675 14710 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:24.890487 14360 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:24.891808 14704 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:24.909761 14722 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:24.877549 14732 consensus_queue.cc:1048] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.070s
I20260709 13:18:24.937881 14745 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:24.938869 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:24.943117 12633 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:24.901427 14292 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:24.932323 12634 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
W20260709 13:18:24.912304 14685 log.cc:927] Time spent T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.973s	user 0.009s	sys 0.000s
I20260709 13:18:24.933214 14590 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:24.913051 14698 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:24.949895 12701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 171) took 1588 ms. Trace:
I20260709 13:18:24.917454 14687 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:18:24.975740 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 269456us
I20260709 13:18:24.950330 14734 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:24.978268 14734 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.978657 14734 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:24.979568 14701 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:24.981894 14745 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:24.984623 13384 raft_consensus.cc:1240] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:24.989689 14711 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:24.989943 12634 raft_consensus.cc:1240] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:24.990772 14724 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:24.991364 14722 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:24.951551 14737 log.cc:927] Time spent T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 13:18:24.976038 14735 log.cc:927] Time spent T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:18:24.973676 12701 rpcz_store.cc:276] 0709 13:18:23.274244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.275019 (+   775us) service_pool.cc:224] Handling call
0709 13:18:23.275372 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:23.275735 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:23.275765 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:23.285057 (+  9292us) log.cc:844] Serialized 26 byte log entry
0709 13:18:23.623167 (+338110us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b180043db60. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf520d1 00007f4c6d36da0f
0709 13:18:23.677999 (+ 54832us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:23.678055 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.678156 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:24.593206 (+915050us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:18:24.862762 (+269556us) raft_consensus.cc:1685] finished
0709 13:18:24.862783 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:24.862959 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4638,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":3669,"spinlock_wait_cycles":74659584,"thread_start_us":53934,"threads_started":1,"wal-append.queue_time_us":489141}
I20260709 13:18:24.935949 14721 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:24.994796 14360 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:24.995009 14721 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:24.994804 14715 consensus_queue.cc:1059] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:24.996780 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:24.975720 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:24.941612 14571 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:24.920624 14657 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:25.010159 14698 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:24.935019 14655 raft_consensus.cc:3055] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:25.011006 14657 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:25.011089 14655 raft_consensus.cc:740] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:25.011554 14657 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:24.918988 14720 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:24.997538 14721 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:25.004442 14460 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:25.010499 14734 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:24.999934 14708 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:25.000860 14687 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.998834 14736 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:24.998971 14711 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:25.016060 14687 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:24.977756 14742 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:25.017685 12747 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:25.021420 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:25.022104 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:25.022193 12626 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:25.023008 12634 raft_consensus.cc:2393] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 4: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:25.023052 12703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 175) took 1744 ms. Trace:
I20260709 13:18:25.023553 12703 rpcz_store.cc:276] 0709 13:18:23.278826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.278934 (+   108us) service_pool.cc:224] Handling call
0709 13:18:23.293786 (+ 14852us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:24.260778 (+966992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.022793 (+762015us) spinlock_profiling.cc:243] Waited 963 ms on lock 0x7b540021f2d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:25.022991 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2215215488}
I20260709 13:18:25.024132 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:25.019762 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:25.025511 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:25.026139 12625 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:25.039626 14698 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:25.048295 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.025537 14711 raft_consensus.cc:697] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:25.002074 14317 consensus_queue.cc:1059] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 13:18:25.042104 14588 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:25.066672 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 148837us
I20260709 13:18:25.070389 14721 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.025425 14708 consensus_queue.cc:1059] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
W20260709 13:18:25.057636 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 157290us
I20260709 13:18:25.052119 14745 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:25.068424 14702 raft_consensus.cc:697] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:25.099988 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:25.100767 12634 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:25.100167 12673 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:25.114758 12625 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:25.115761 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:25.116492 12625 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:25.087790 14590 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.064167 14571 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:18:25.101497 14708 consensus_queue.cc:1243] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:18:25.133194 12625 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:25.080022 14588 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:25.113045 14745 raft_consensus.cc:697] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:25.148849 12783 raft_consensus.cc:3055] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:25.188000 12783 raft_consensus.cc:740] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:25.131922 14742 raft_consensus.cc:697] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:25.125173 14704 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:25.101219 14721 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:25.142740 14724 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.129518 14698 consensus_queue.cc:1059] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:18:25.190049 14707 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.216516 12783 consensus_queue.cc:260] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:25.189164 14564 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:25.132843 14590 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:25.264801 12783 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:25.265514 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.230527 14734 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:25.270380 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 170275us
I20260709 13:18:25.233024 14714 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:25.285755 12673 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:25.190173 14711 consensus_queue.cc:237] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:25.303923 12673 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:25.304487 12775 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:25.304842 12673 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:25.277575 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:25.220829 14724 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:25.308774 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 174927us
I20260709 13:18:25.236819 14721 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:25.276582 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:25.267853 14562 consensus_meta.cc:287] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.616s	user 0.011s	sys 0.000s
I20260709 13:18:25.302958 14564 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.224756 14707 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:25.314517 12782 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
W20260709 13:18:25.328631 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.313211 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:25.311793 12776 raft_consensus.cc:2410] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:18:25.327852 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 292630us
I20260709 13:18:25.220328 14702 consensus_queue.cc:237] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:25.362042 12781 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:25.268914 14750 raft_consensus.cc:3055] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:25.278417 14710 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:25.364236 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:25.311853 14479 consensus_meta.cc:287] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.587s	user 0.010s	sys 0.002s
W20260709 13:18:25.370935 14753 log.cc:927] Time spent T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:18:25.371412 14682 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:25.314448 14654 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.372496 12778 raft_consensus.cc:3055] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:25.372885 12778 raft_consensus.cc:740] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:25.372720 14654 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:25.373700 12778 consensus_queue.cc:260] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:25.374652 12778 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:25.375003 14654 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:25.360203 14742 consensus_queue.cc:237] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:25.342427 14623 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:18:25.367921 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.311455 12672 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:25.306308 14292 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:25.388098 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:25.364516 14750 raft_consensus.cc:740] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:25.391083 12747 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:25.394966 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 84800.9us
I20260709 13:18:25.393752 12783 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:18:25.285808 14560 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:25.381577 14479 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:25.406646 14707 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:25.406903 12632 raft_consensus.cc:1240] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:25.383747 14562 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:25.396255 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:25.433518 12777 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:25.398255 12779 raft_consensus.cc:1240] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:25.405282 14720 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:18:25.437026 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 125636us
I20260709 13:18:25.399559 12783 pending_rounds.cc:85] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 2
I20260709 13:18:25.381434 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:25.409324 12775 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:25.397416 14523 raft_consensus.cc:3055] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:25.229640 14745 consensus_queue.cc:237] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:25.405208 14560 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:25.442829 12747 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:25.366006 14736 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:18:25.432324 14762 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:25.380790 14770 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:25.447328 14590 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:25.447942 14590 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:25.448294 14590 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:25.448619 14590 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:25.441676 14523 raft_consensus.cc:740] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:25.449987 12777 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
W20260709 13:18:25.450433 14764 log.cc:927] Time spent T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:18:25.435238 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:25.450314 14523 consensus_queue.cc:260] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:25.451792 14523 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:25.452330 14772 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:25.453321 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:25.453971 12632 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:25.398382 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:25.443042 14623 raft_consensus.cc:697] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:25.455637 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:25.455461 14623 consensus_queue.cc:237] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:25.457008 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:25.457808 12634 raft_consensus.cc:2393] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 4: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:25.458751 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:25.459334 12634 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:25.459797 14657 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:25.461055 12633 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:25.461186 14689 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.462376 14657 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:25.463790 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:25.464545 12626 raft_consensus.cc:2393] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 3: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:25.469115 12701 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:18:25.470525 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:25.471964 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:25.472700 12709 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
W20260709 13:18:25.473872 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 62419.9us
I20260709 13:18:25.477941 14750 consensus_queue.cc:260] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:25.479537 14750 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:25.481798 14562 raft_consensus.cc:697] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:25.481920 14768 consensus_queue.cc:1048] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:25.483414 14562 consensus_queue.cc:237] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:25.394881 14725 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:25.363302 14564 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:25.484722 14725 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:25.486207 12702 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:18:25.440526 12775 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:25.487634 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:25.395769 14771 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:25.488477 12631 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:25.488741 14770 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:25.489259 14770 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:25.489161 14758 consensus_queue.cc:1059] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 13:18:25.490089 14758 raft_consensus.cc:3055] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:25.379292 14317 consensus_queue.cc:1059] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:18:25.418169 14588 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:25.490433 14758 raft_consensus.cc:740] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:25.490712 14317 consensus_queue.cc:1243] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:18:25.491204 14708 raft_consensus.cc:3055] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:25.491772 14708 raft_consensus.cc:740] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:25.462096 14689 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:25.494408 12709 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:25.494519 14689 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:25.461056 14710 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:25.487720 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:25.405297 14479 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:25.508157 14724 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:25.512109 14679 consensus_meta.cc:287] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.597s	user 0.010s	sys 0.000s
I20260709 13:18:25.446887 14762 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:25.513373 14762 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:25.408043 14682 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:25.516350 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:25.399386 14714 raft_consensus.cc:697] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:25.491317 14758 consensus_queue.cc:260] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:25.497051 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:25.410482 12597 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:25.498879 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:25.465343 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:25.535188 12775 raft_consensus.cc:2393] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:25.503087 14292 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:25.503595 14590 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:25.499873 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 57100.6us
I20260709 13:18:25.504597 14725 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:25.505353 14508 consensus_meta.cc:287] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.707s	user 0.007s	sys 0.003s
I20260709 13:18:25.438524 12783 pending_rounds.cc:107] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:18:25.552708 12783 raft_consensus.cc:2889] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:25.523001 12778 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:25.554679 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:25.555421 12778 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:25.530145 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.532522 12634 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:25.535329 12779 raft_consensus.cc:2393] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
W20260709 13:18:25.535230 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 124838us
I20260709 13:18:25.536588 12631 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:25.541734 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:25.443409 13377 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:25.546221 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:25.563789 12709 raft_consensus.cc:2393] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 4: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:25.549280 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:25.564922 12775 raft_consensus.cc:2393] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:25.565886 12628 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:25.566017 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:25.569640 12747 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:25.550971 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:25.520406 12632 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:25.569757 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:25.572914 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.573941 12783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 1789) took 1229 ms. Trace:
I20260709 13:18:25.574460 12783 rpcz_store.cc:276] 0709 13:18:24.340278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.344746 (+  4468us) service_pool.cc:224] Handling call
0709 13:18:24.345047 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:25.148711 (+803664us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54003a5320. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d329dc0 00007f4c6d3b41c0 00007f4c6d3b5231 00007f4c6d3b1d8f 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c
0709 13:18:25.569793 (+421082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.570018 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434466176}
I20260709 13:18:25.540423 14740 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:18:25.579358 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.582392 12631 raft_consensus.cc:1275] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:18:25.586486 12627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56838 (request call id 1037) took 1350 ms (client timeout 1697 ms). Trace:
I20260709 13:18:25.587007 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:25.589190 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:25.589996 12634 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:25.587050 12627 rpcz_store.cc:269] 0709 13:18:24.236618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.236827 (+   209us) service_pool.cc:224] Handling call
0709 13:18:25.586274 (+1349447us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b54002db9d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:25.586413 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":284590464}
I20260709 13:18:25.592896 12627 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:25.586227 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:25.571578 14758 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:25.568744 14775 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.608675 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:25.609728 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:25.609921 12673 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:25.577112 14689 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:25.572543 14317 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:25.623468 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:25.624406 12780 raft_consensus.cc:2393] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:25.613552 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:25.552158 14571 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:18:25.568729 14508 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:25.623430 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:25.625770 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:25.626739 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:25.581669 14740 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:25.625862 12707 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:25.628322 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:25.628523 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:25.628479 12634 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:25.633253 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:25.633283 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 70550.2us
I20260709 13:18:25.634022 12783 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:25.635565 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:25.638420 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:25.642637 12780 raft_consensus.cc:2410] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:25.643607 12783 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:25.645803 14590 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:25.647593 12704 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:25.650763 14564 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:25.651345 14564 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.651769 14564 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:25.652719 14687 consensus_meta.cc:287] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.636s	user 0.007s	sys 0.003s
I20260709 13:18:25.653826 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:25.654428 12775 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:25.597764 14742 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:18:25.676527 12778 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:25.686537 12706 raft_consensus.cc:2393] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 4: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:25.688680 12709 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 4. (term mismatch)
I20260709 13:18:25.689085 12709 pending_rounds.cc:85] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 3
I20260709 13:18:25.689356 12709 pending_rounds.cc:107] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20260709 13:18:25.689638 12709 raft_consensus.cc:2889] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:25.653100 14687 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:25.691058 12775 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:25.691586 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:25.692971 12634 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:18:25.694518 12703 raft_consensus.cc:1217] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Deduplicated request from leader. Original: 2.2->[4.3-4.3]   Dedup: 4.3->[]
I20260709 13:18:25.695029 12703 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:18:25.632766 14740 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:25.697933 12778 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:18:25.639940 14711 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:25.640825 14689 raft_consensus.cc:697] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:25.703580 12673 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:25.688695 14588 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:25.711869 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:25.624269 14775 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:25.715016 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:25.702950 12701 raft_consensus.cc:2955] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index 2 to 4, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:25.717394 13384 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:25.699693 14590 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.718547 13383 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:18:25.626070 14714 consensus_queue.cc:237] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:25.740330 13383 pending_rounds.cc:85] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd: Aborting all ops after (but not including) 5
I20260709 13:18:25.740682 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:25.743868 13383 pending_rounds.cc:107] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:18:25.744251 13383 raft_consensus.cc:2889] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:25.712924 14682 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:25.711236 14711 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:25.742197 14779 log.cc:927] Time spent T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.220s	user 0.000s	sys 0.002s
I20260709 13:18:25.755990 12700 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:25.816200 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:25.763765 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.710999 14562 consensus_queue.cc:1048] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
W20260709 13:18:25.745549 14701 consensus_meta.cc:287] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.762s	user 0.011s	sys 0.011s
W20260709 13:18:25.820693 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 117200us
I20260709 13:18:25.741456 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:25.715481 14768 consensus_queue.cc:1059] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.100s
I20260709 13:18:25.833268 12779 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:25.743321 14564 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:25.833940 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:25.834878 12705 raft_consensus.cc:1275] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:18:25.841428 12705 pending_rounds.cc:85] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 2
I20260709 13:18:25.706491 14770 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:25.745429 14687 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:25.751297 14680 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:25.841727 12705 pending_rounds.cc:107] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:18:25.844203 12705 raft_consensus.cc:2889] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:25.756747 14588 raft_consensus.cc:697] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:25.822942 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:25.852205 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:25.829289 14560 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:25.852579 12628 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:25.825672 14613 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:25.845017 14720 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:18:25.867439 12709 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:25.742869 14590 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:25.869818 12779 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:25.870121 12700 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:25.871309 12701 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 13:18:25.855054 14781 log.cc:927] Time spent T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
W20260709 13:18:25.859066 14688 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:18:25.856199 14725 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.880607 12630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1036) took 1643 ms. Trace:
W20260709 13:18:25.851173 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 232268us
I20260709 13:18:25.884439 12747 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:25.858278 14564 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.881117 12630 rpcz_store.cc:276] 0709 13:18:24.232052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.233714 (+  1662us) service_pool.cc:224] Handling call
0709 13:18:24.240125 (+  6411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.738590 (+498465us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.874800 (+1136210us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b54002ce054. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:25.874969 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1125741440}
I20260709 13:18:25.860400 14777 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.860821 14761 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.860510 14768 consensus_queue.cc:1243] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:18:25.895462 12704 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:25.899612 12701 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:25.874408 14560 raft_consensus.cc:697] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:18:25.908859 14782 log.cc:927] Time spent T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:18:25.909377 14775 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:25.866552 14613 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:25.908154 14680 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:25.875257 14292 consensus_queue.cc:1059] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:18:25.923988 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:25.924669 12701 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:25.924794 14711 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:25.925449 14762 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:25.927482 14762 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:25.858628 14689 consensus_queue.cc:237] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:25.897274 14725 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:25.931810 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:25.932315 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.933138 14725 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:25.919348 14792 log.cc:927] Time spent T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:18:25.933723 14679 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:25.934311 14679 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:25.936556 14679 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:25.941242 14560 consensus_queue.cc:237] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:25.943415 14778 log.cc:927] Time spent T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
I20260709 13:18:25.905500 14777 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:25.920706 14789 log.cc:927] Time spent T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:18:25.922722 12672 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:25.923213 13377 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:25.905368 14761 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:25.951392 12701 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:25.952368 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:25.956866 12630 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:25.954429 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:25.969152 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:25.907651 14508 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:25.973250 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.974920 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:25.900231 14564 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:25.982224 12672 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:25.985335 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:25.986544 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:25.988356 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:25.950954 14523 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.957186 14720 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.973476 14680 raft_consensus.cc:697] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:25.977002 14602 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:25.995908 12780 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:26.000182 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:26.004930 12630 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:25.990190 14508 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:25.984448 13377 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51754 (request call id 407) took 1883 ms (client timeout 1598 ms). Trace:
I20260709 13:18:26.013386 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:25.945950 14588 consensus_queue.cc:237] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:26.015408 12701 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 4: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
W20260709 13:18:26.019102 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 62134.2us
I20260709 13:18:25.997190 14746 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:25.997875 14720 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:25.996459 14523 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:26.042265 12672 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:26.026247 14684 raft_consensus.cc:3055] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:18:26.056074 14799 log.cc:927] Time spent T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:18:26.057142 14523 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:26.016105 14508 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:26.014124 13377 rpcz_store.cc:269] 0709 13:18:24.094591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.112024 (+ 17433us) service_pool.cc:224] Handling call
0709 13:18:25.367912 (+1255888us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b5800095198. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c723780a0 00007f4c72377984 00007f4c6b8f26d4 00007f4c6b8f2662
0709 13:18:25.977449 (+609537us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4797824}
I20260709 13:18:26.063376 14802 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.064211 14777 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:26.064095 14802 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:26.044384 12672 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:26.067433 14777 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:26.068610 12672 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:26.052716 14684 raft_consensus.cc:740] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:26.054780 14776 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:26.074954 14775 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:26.077275 14613 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:26.078151 14684 consensus_queue.cc:260] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:26.079516 14684 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:26.086143 14564 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:26.086225 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:26.087128 12777 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:26.090896 14508 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:26.091174 12701 raft_consensus.cc:1240] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:18:26.091421 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:26.066177 14680 consensus_queue.cc:237] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:26.092722 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:26.093274 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:26.094435 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:26.095274 14761 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:26.095485 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:26.097636 14720 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:26.097716 14655 consensus_queue.cc:260] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:26.098999 14655 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:26.099112 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:26.100212 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:26.104559 14756 raft_consensus.cc:3055] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:26.105047 14756 raft_consensus.cc:740] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:26.105880 14756 consensus_queue.cc:260] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:26.108286 14802 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:18:26.100190 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:26.110726 12701 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:26.106827 14756 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:26.068816 14523 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.122501 14523 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:26.124589 14523 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:26.124817 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:26.125604 12700 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:26.126155 14767 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.130744 12783 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:26.134557 12708 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:26.138705 12782 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:26.142805 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:26.142931 12672 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:26.144629 12782 pending_rounds.cc:85] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 2
I20260709 13:18:26.146739 12782 pending_rounds.cc:107] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:18:26.147173 12782 raft_consensus.cc:2889] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:26.149957 12597 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:26.150178 12701 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:26.152279 12597 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:26.154862 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:26.149544 14740 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:26.163025 14740 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:26.154109 14746 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:18:26.165369 14591 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:26.165848 14591 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:26.166208 14591 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:26.159212 14758 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:26.159148 14743 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 13:18:26.170352 14743 consensus_queue.cc:1243] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer 3d539f8d95d343d8b1696803f18b30d2 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:18:26.171813 12783 raft_consensus.cc:1240] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:26.159325 14767 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:26.176928 14767 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:26.177397 14749 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.178153 14749 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:18:26.179685 14590 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:26.186471 14749 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:26.187204 12626 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:26.117822 12777 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:26.192126 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:26.193795 12777 pending_rounds.cc:85] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 1
I20260709 13:18:26.196097 12777 pending_rounds.cc:107] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260709 13:18:26.196696 12777 raft_consensus.cc:2889] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:26.198647 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.197444 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:26.200695 12783 raft_consensus.cc:2410] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:18:26.158756 14460 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:26.205272 13348 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:26.158991 14720 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:26.244499 12598 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:26.242822 13348 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:26.246655 12598 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:26.247725 12776 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:26.252187 12598 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:26.201467 14523 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.263908 12707 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:26.265810 12598 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:26.270697 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:26.201678 14802 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:26.271821 12707 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:26.246273 14460 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:26.258344 14613 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:26.278318 14183 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.256449 14720 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:26.256997 14725 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:26.261106 14749 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:26.242660 14655 consensus_queue.cc:1059] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:18:26.274137 14743 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:26.267306 14724 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:26.248808 14679 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:18:26.289842 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:26.284785 14183 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:26.206295 14738 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 13:18:26.283553 14802 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:26.302640 14814 consensus_queue.cc:1059] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:26.281634 14740 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:26.258149 14602 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:18:26.267637 14552 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:18:26.281327 14808 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:26.274232 14758 consensus_queue.cc:1059] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:18:26.315757 14613 raft_consensus.cc:697] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:26.276829 14523 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:26.318635 14479 raft_consensus.cc:3055] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:26.289214 14460 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:26.346135 12633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39590 (request call id 215) took 1351 ms. Trace:
I20260709 13:18:26.346593 12633 rpcz_store.cc:276] 0709 13:18:24.995142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.996498 (+  1356us) service_pool.cc:224] Handling call
0709 13:18:26.345922 (+1349424us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b54003458d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:26.346074 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1065657216}
W20260709 13:18:26.326961 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 61297.3us
I20260709 13:18:26.348790 12598 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:26.317423 12707 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:26.282919 14591 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:26.335819 14523 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:26.338363 14805 log.cc:927] Time spent T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:18:26.300315 14725 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:26.359753 14774 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:26.360384 14336 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:26.360813 14758 raft_consensus.cc:3055] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:26.361795 12598 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:26.369877 14591 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:26.348275 14479 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:26.370781 14720 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:26.371152 14479 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:26.372572 14479 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:26.379418 12598 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:26.381721 14820 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:26.368547 14758 raft_consensus.cc:740] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:26.362948 14749 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:18:26.387944 12598 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:26.390350 14758 consensus_queue.cc:260] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:26.390933 14552 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:26.398053 14802 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:26.399015 14725 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:26.386829 14724 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:26.415230 14183 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:26.415529 14718 log.cc:927] Time spent T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 1.680s	user 0.010s	sys 0.014s
I20260709 13:18:26.410822 14758 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:26.421906 12598 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:26.396845 14613 consensus_queue.cc:237] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:26.433841 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.437964 12598 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:26.432893 14724 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:26.461736 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:26.476814 12705 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:18:26.441263 12598 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:26.488029 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 46908.2us
I20260709 13:18:26.486560 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:26.496388 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:26.498344 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:26.512378 12632 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:26.513017 12629 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:18:26.493409 12705 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:26.514755 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:26.515811 12705 raft_consensus.cc:2393] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 6: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:26.522509 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:26.523334 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:26.527604 12626 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:18:26.527601 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:26.482053 14725 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:26.529466 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:26.530061 12625 raft_consensus.cc:2376] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5. Re-sending same reply.
I20260709 13:18:26.494100 14183 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:26.520533 14738 consensus_queue.cc:1048] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.029s
W20260709 13:18:26.558276 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 223924us
I20260709 13:18:26.583091 12632 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:18:26.584136 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:26.592866 12626 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:26.536334 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:26.526671 14767 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.595183 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:26.571277 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:26.599035 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:26.599678 12777 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:26.605384 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:26.607347 12632 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:18:26.532130 14725 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:26.616088 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:26.617517 13379 raft_consensus.cc:2376] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6. Re-sending same reply.
I20260709 13:18:26.618991 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:26.619240 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:26.619798 13382 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:26.568840 14336 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.538409 14816 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:26.598109 14824 log.cc:927] Time spent T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.064s	user 0.000s	sys 0.003s
W20260709 13:18:26.598326 14821 log.cc:927] Time spent T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.070s	user 0.000s	sys 0.003s
I20260709 13:18:26.531334 14552 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:26.621224 12629 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 3. Current term is 5. Ops: []
I20260709 13:18:26.639142 13382 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:26.619040 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:26.621605 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:26.542840 14183 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.640228 12632 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 4: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:26.640110 14183 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:26.623432 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:26.641393 14552 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:26.642073 13377 raft_consensus.cc:2376] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3. Re-sending same reply.
I20260709 13:18:26.627357 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:26.541796 14817 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.606388 14820 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.649830 14820 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:26.650514 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1106) took 1186 ms. Trace:
I20260709 13:18:26.650930 12702 rpcz_store.cc:276] 0709 13:18:25.464274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.467993 (+  3719us) service_pool.cc:224] Handling call
0709 13:18:25.468327 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:25.486651 (+ 18324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.650243 (+1163592us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b5400107cd4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:26.650450 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40507648}
I20260709 13:18:26.652431 14820 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:26.652557 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:26.633244 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:26.654835 12632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:18:26.655467 12632 raft_consensus.cc:606] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Received request to transfer leadership to TS 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:26.655810 12632 raft_consensus.cc:614] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:26.655895 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:26.633955 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:26.642724 14183 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:26.658512 14817 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:26.650882 13381 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:26.634186 12626 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:26.631839 14816 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:26.620803 14725 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:26.542451 14460 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:26.646850 14712 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:18:26.648784 12708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1451) took 1040 ms. Trace:
I20260709 13:18:26.663921 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:26.598017 14724 raft_consensus.cc:697] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:26.665107 12702 raft_consensus.cc:2410] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 7 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:18:26.657343 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:26.654861 13379 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:26.664562 12708 rpcz_store.cc:276] 0709 13:18:25.608303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.608417 (+   114us) service_pool.cc:224] Handling call
0709 13:18:26.648612 (+1040195us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b540021e3d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:26.648725 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1132886400}
I20260709 13:18:26.615377 14767 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:26.663558 14588 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.016s	sys 0.006s
I20260709 13:18:26.680117 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:26.681110 13379 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:26.629995 14336 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:26.687873 12705 raft_consensus.cc:1275] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:18:26.678371 14588 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.016s	sys 0.006s
I20260709 13:18:26.685773 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:26.676718 14183 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:26.692701 12709 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:26.699024 14744 raft_consensus.cc:3055] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:26.699738 14724 consensus_queue.cc:237] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:26.712381 13379 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:26.742376 14820 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.743456 14708 consensus_queue.cc:260] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:26.746156 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:26.759872 12709 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:26.764720 12626 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:18:26.765591 12626 raft_consensus.cc:606] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Received request to transfer leadership to TS 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:26.766084 12626 raft_consensus.cc:614] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:26.746498 14744 raft_consensus.cc:740] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:26.768759 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:26.770565 13379 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:26.761337 14817 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:26.757229 14183 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:26.778146 14767 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:26.779289 12707 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:18:26.778138 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:26.764483 14708 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:26.786625 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:26.787287 13379 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:26.793069 12709 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:26.761085 14820 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:26.780609 14770 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:26.802510 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:26.803285 13385 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:26.807077 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:26.810915 14820 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:26.799052 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:26.816581 14183 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:26.803889 14744 consensus_queue.cc:260] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:26.818517 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:26.819317 14744 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:26.819818 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:26.820567 12702 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:26.812232 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:26.813871 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:26.827103 13379 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:26.830482 12598 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:26.832351 12598 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:26.833334 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:26.834082 12598 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:26.835583 12598 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:26.831777 14183 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:26.836614 14740 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:26.836987 14740 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:26.837378 14740 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:26.837633 14738 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:26.838166 14738 raft_consensus.cc:697] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:26.839011 14738 consensus_queue.cc:237] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:26.842252 14336 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:26.843885 14731 log.cc:927] Time spent T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 1.734s	user 0.011s	sys 0.010s
I20260709 13:18:26.845675 12703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1105) took 1381 ms. Trace:
I20260709 13:18:26.834707 14820 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:18:26.846120 12703 rpcz_store.cc:276] 0709 13:18:25.463942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.467390 (+  3448us) service_pool.cc:224] Handling call
0709 13:18:25.467757 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:25.695398 (+227641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.845412 (+1150014us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b5400109ad4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:26.845605 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":521227520}
I20260709 13:18:26.848001 14523 raft_consensus.cc:3055] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:26.848407 14523 raft_consensus.cc:740] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:26.849195 14523 consensus_queue.cc:260] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:26.850186 14523 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:26.835197 14817 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:26.852150 14817 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:26.852509 14817 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:26.855780 12779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36488 (request call id 1587) took 1294 ms. Trace:
I20260709 13:18:26.855896 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:26.856353 12779 rpcz_store.cc:276] 0709 13:18:25.561842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.585915 (+ 24073us) service_pool.cc:224] Handling call
0709 13:18:26.855584 (+1269669us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54000cc954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:26.855699 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":539009920}
I20260709 13:18:26.857417 12703 raft_consensus.cc:2376] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6. Re-sending same reply.
I20260709 13:18:26.870702 12598 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:26.874456 14724 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:26.886196 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:26.887408 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:26.887923 12629 raft_consensus.cc:1275] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:26.888052 13378 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:26.888911 14746 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.889022 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:26.889503 14746 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:26.889956 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:26.890338 14552 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:26.891230 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:26.891579 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:26.891963 12631 raft_consensus.cc:2393] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 6: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:26.891999 14746 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:26.892611 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:26.892887 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:26.893435 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:26.894779 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:26.890218 12633 raft_consensus.cc:1240] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:18:26.889437 12626 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:26.865331 14725 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:26.911859 12633 raft_consensus.cc:1275] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:18:26.912184 13384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51762 (request call id 454) took 1173 ms. Trace:
I20260709 13:18:26.912643 13384 rpcz_store.cc:276] 0709 13:18:25.739018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.739157 (+   139us) service_pool.cc:224] Handling call
0709 13:18:26.891433 (+1152276us) spinlock_profiling.cc:243] Waited 998 us on lock 0x7b5800095198. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c723780a0 00007f4c72377984 00007f4c6b8f26d4 00007f4c6b8f2662
0709 13:18:26.912109 (+ 20676us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2296192}
I20260709 13:18:26.928591 12631 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:26.931468 12626 raft_consensus.cc:1275] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:26.932416 12673 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:26.868705 14740 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:26.933871 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:26.946492 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:26.948926 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.879731 14724 raft_consensus.cc:697] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:18:26.949393 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.934567 12783 raft_consensus.cc:1275] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:18:26.949872 12783 pending_rounds.cc:85] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 2
I20260709 13:18:26.950125 12783 pending_rounds.cc:107] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:18:26.950356 12783 raft_consensus.cc:2889] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:26.950424 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:18:26.952246 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.899612 14588 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.932796 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:26.965523 12782 raft_consensus.cc:1275] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:26.972487 12634 raft_consensus.cc:2393] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 6: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:18:26.976879 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.979601 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:26.980496 12782 raft_consensus.cc:2393] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:26.980095 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:26.982177 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:26.956741 14762 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.962177 14564 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:26.965070 14732 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:26.967716 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:26.994585 12775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1586) took 1433 ms. Trace:
I20260709 13:18:26.994894 14336 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:26.995378 12777 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "866523bf2d2f442f971d504267ed200f"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:18:27.001933 12777 raft_consensus.cc:606] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Received request to transfer leadership to TS 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:27.002358 12777 raft_consensus.cc:614] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:27.003193 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:26.988452 14738 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.997627 12775 rpcz_store.cc:276] 0709 13:18:25.561526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.853945 (+292419us) service_pool.cc:224] Handling call
0709 13:18:25.874060 (+ 20115us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:25.880530 (+  6470us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:18:26.187378 (+306848us) spinlock_profiling.cc:243] Waited 6.66 ms on lock 0x7b180000f940. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c2455 00007f4c70daa3c7 00007f4c6d3bc9a7 00007f4c6d3cc0bf 00007f4c6d3cc038
0709 13:18:26.299061 (+111683us) log.cc:844] Serialized 14 byte log entry
0709 13:18:26.947477 (+648416us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:26.947962 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 13:18:26.948105 (+   143us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:26.948137 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.948197 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:26.994302 (+ 46105us) raft_consensus.cc:1685] finished
0709 13:18:26.994400 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:26.994523 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":494714,"prepare.queue_time_us":26160,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":951,"raft.queue_time_us":472511,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":17124,"spinlock_wait_cycles":15310592,"thread_start_us":264550,"threads_started":2,"wal-append.queue_time_us":352787}
I20260709 13:18:27.001123 12673 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:27.003870 12629 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
W20260709 13:18:26.993448 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 58227us
I20260709 13:18:27.009629 12747 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:26.972977 14755 consensus_queue.cc:1059] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:18:27.011139 12597 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:26.970330 14588 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:27.004863 14776 consensus_queue.cc:1048] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:18:27.012878 12673 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:27.014197 13349 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:27.017361 14823 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:27.000509 14564 raft_consensus.cc:697] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:27.000329 14732 raft_consensus.cc:697] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:27.020308 14761 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:27.021169 14755 raft_consensus.cc:3055] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:27.021610 14755 raft_consensus.cc:740] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:27.022462 14755 consensus_queue.cc:260] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:27.023761 14755 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:27.021049 14825 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.011s
W20260709 13:18:27.026094 14840 log.cc:927] Time spent T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:18:27.026679 14823 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:27.021019 14732 consensus_queue.cc:237] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:27.028352 12777 raft_consensus.cc:1275] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:27.029008 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:27.029810 12781 raft_consensus.cc:1275] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:18:27.011085 14336 raft_consensus.cc:697] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:27.031150 14336 consensus_queue.cc:237] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:27.031859 14816 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:27.015618 14591 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:27.034003 14591 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.034356 14591 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:27.035418 14835 log.cc:927] Time spent T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.234s	user 0.002s	sys 0.000s
I20260709 13:18:27.013963 12597 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:27.015444 14744 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:27.036661 12597 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:27.040184 14774 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:27.040642 14774 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.041018 14774 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:27.044054 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:27.045682 14776 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:27.046314 12673 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:27.049095 14460 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:27.046463 14816 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:27.049983 14816 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:27.050390 14816 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:27.051179 14839 log.cc:927] Time spent T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:18:27.051430 12597 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:27.052922 14564 consensus_queue.cc:237] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:27.055073 14460 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:27.056497 14724 consensus_queue.cc:237] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:27.058301 12627 raft_consensus.cc:1240] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:26.990556 14762 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:27.059947 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.061170 14740 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:27.061939 12783 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:27.062920 14752 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:27.070070 12701 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:27.064913 13349 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:27.071506 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:27.071659 12597 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:27.077939 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:27.078198 12708 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:27.078599 12701 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:27.018417 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:27.079535 12709 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
I20260709 13:18:27.068922 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:27.082239 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:27.082501 12705 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:27.069811 12673 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:27.080855 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:27.096264 12627 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
I20260709 13:18:27.098807 12707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 196) took 1176 ms. Trace:
I20260709 13:18:27.100582 12630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:18:27.101444 12630 raft_consensus.cc:606] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Received request to transfer leadership to TS 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:27.102953 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 84629.9us
I20260709 13:18:27.103772 12672 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:27.104292 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:27.100831 12597 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:27.106693 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:27.099225 12707 rpcz_store.cc:276] 0709 13:18:25.922096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.350305 (+428209us) service_pool.cc:224] Handling call
0709 13:18:26.778811 (+428506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:26.783240 (+  4429us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.098641 (+315401us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b540006e350. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:27.098755 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":984669568}
I20260709 13:18:27.108978 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:27.109634 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:27.110729 12597 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:27.013252 14738 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:27.016152 14712 consensus_queue.cc:1059] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.102s
I20260709 13:18:27.106432 14825 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:27.113260 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:27.124039 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 52299.8us
I20260709 13:18:27.125154 12632 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:27.126281 12634 raft_consensus.cc:1240] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:27.126596 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:27.127879 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:27.126174 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:27.128706 13384 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:27.133611 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:27.137308 14712 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:27.099076 14613 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:27.085772 14761 raft_consensus.cc:697] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:27.108690 14740 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:27.120566 14704 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:27.130055 14571 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:18:27.122347 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.116163 14460 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:27.058781 14523 consensus_queue.cc:1059] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 13:18:27.116427 14756 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:27.123281 14680 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:18:27.155462 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 201036us
I20260709 13:18:27.138586 14712 raft_consensus.cc:3055] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:27.150946 12529 catalog_manager.cc:5697] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 2 to 3, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 3 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
W20260709 13:18:27.168368 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.175951 12746 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:27.145758 14508 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:27.115553 14817 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:27.132419 14808 consensus_queue.cc:1059] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:18:27.150012 14770 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:18:27.174108 14571 raft_consensus.cc:697] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:27.139884 14758 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:18:27.139703 14772 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:18:27.195070 12625 raft_consensus.cc:2393] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 7: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:18:27.179464 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 67292.7us
W20260709 13:18:27.170122 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 43643.1us
I20260709 13:18:27.172104 14712 raft_consensus.cc:740] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:27.185122 14771 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:27.197685 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 93990.8us
I20260709 13:18:27.182773 14762 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:27.189252 14823 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:27.206305 12672 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:27.222229 12672 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:27.180820 12746 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:27.177876 12529 catalog_manager.cc:5697] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 2 to 3, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 3 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:27.212839 14744 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:18:27.234371 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 53624.1us
I20260709 13:18:27.244092 12746 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:27.219214 14752 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:27.227483 14756 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.239396 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:27.200083 12747 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:27.229403 14588 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:27.199507 14761 consensus_queue.cc:237] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:27.238494 14762 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:27.243793 14767 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:18:27.259421 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 59446.2us
I20260709 13:18:27.231894 12529 catalog_manager.cc:5697] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:18:27.226459 14774 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:27.292865 12672 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:27.296574 12747 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:27.279606 12746 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:27.303637 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:27.303036 12529 catalog_manager.cc:5697] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 1 to 3, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 3 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:27.305786 12529 catalog_manager.cc:5697] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 2 to 3, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 3 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:27.309285 14774 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:27.259132 14758 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:27.279645 14762 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:27.237202 14571 consensus_queue.cc:237] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:27.291268 14732 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:27.296820 14846 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:27.254038 14756 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:27.300200 14564 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:27.291016 14479 consensus_queue.cc:1048] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 13:18:27.238813 14712 consensus_queue.cc:260] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:27.312650 14704 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:27.329463 14817 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:27.366200 12706 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:18:27.366581 12706 pending_rounds.cc:85] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 2
I20260709 13:18:27.366909 12706 pending_rounds.cc:107] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:18:27.320456 14767 consensus_queue.cc:1048] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:18:27.367368 12630 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:27.367733 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:27.334748 14816 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:27.335908 12597 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:27.367125 12706 raft_consensus.cc:2889] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:27.290864 14588 consensus_queue.cc:1048] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:18:27.314703 14776 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:27.324067 14508 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:18:27.333694 14844 log.cc:927] Time spent T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 13:18:27.326970 14708 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:18:27.353844 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 74350.4us
W20260709 13:18:27.329821 14854 log.cc:927] Time spent T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
W20260709 13:18:27.356204 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 251874us
I20260709 13:18:27.358119 12625 raft_consensus.cc:1275] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:18:27.378700 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 75124.9us
I20260709 13:18:27.322171 14746 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:18:27.339241 14680 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:27.349983 12529 catalog_manager.cc:5697] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 1 to 3, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 3 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:27.352753 14762 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:27.370752 12634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:18:27.390554 12634 raft_consensus.cc:606] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Received request to transfer leadership to TS 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:27.390900 12634 raft_consensus.cc:614] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:27.392091 14816 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:27.392645 14849 log.cc:927] Time spent T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:18:27.376071 14508 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.378703 12746 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:27.380093 12673 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:27.364488 14564 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.374477 14774 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:27.384377 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:27.399523 12707 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:27.363137 14847 consensus_queue.cc:1048] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:18:27.400441 12705 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:27.385905 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:27.401228 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 272587us
I20260709 13:18:27.401778 12708 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:27.402016 12746 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:27.402722 13349 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:27.386210 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:27.385210 12529 catalog_manager.cc:5697] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:27.387993 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:27.388622 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:27.403795 12701 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:27.389228 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:27.389449 14680 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.378041 14846 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.397128 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:27.397872 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:27.416793 12703 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:27.403237 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:27.421415 13379 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:27.404925 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:18:27.424477 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:27.405418 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:27.362948 14732 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:27.406711 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:27.372411 12627 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:27.408887 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:27.428402 13385 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:27.390482 14708 raft_consensus.cc:697] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:27.394012 14317 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:27.404438 12529 catalog_manager.cc:5697] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:27.443220 12774 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1601) took 1066 ms. Trace:
I20260709 13:18:27.414072 12783 raft_consensus.cc:2393] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 5: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:27.411388 12708 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:27.437135 12628 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:27.413149 14684 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:27.437289 14771 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:27.399793 14508 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:27.373772 14745 consensus_queue.cc:1175] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "639ab2db265f4747bb7afb93e833f530" responder_term: 4 status { last_received { term: 2 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: [3.3-3.3]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:18:27.430229 14862 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:27.440623 14689 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:27.395589 14523 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:18:27.446245 14682 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:27.443655 12529 catalog_manager.cc:5697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:27.404940 14774 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.384178 14704 consensus_queue.cc:1048] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:18:27.400319 14564 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:27.447365 12707 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:27.442369 14861 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:18:27.465951 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:27.443653 12774 rpcz_store.cc:276] 0709 13:18:26.376999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.893767 (+516768us) service_pool.cc:224] Handling call
0709 13:18:26.894145 (+   378us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:26.894541 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:26.894568 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:26.895162 (+   594us) log.cc:844] Serialized 52 byte log entry
0709 13:18:27.092640 (+197478us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:27.136628 (+ 43988us) log.cc:844] Serialized 14 byte log entry
0709 13:18:27.169588 (+ 32960us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b1800484fc0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a23fc0a 00007f4c7397f219 00007f4c6a2c7a40 00007f4c6a2c24e4 00007f4c6a2c2455 00007f4c6d3aa060 00007f4c6d3bc716 00007f4c6d3cd287 00007f4c6d3cd208
0709 13:18:27.169813 (+   225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.169895 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:27.241686 (+ 71791us) spinlock_profiling.cc:243] Waited 2.16 ms on lock 0x7b1800484fc0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c53f3 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:18:27.443000 (+201314us) raft_consensus.cc:1685] finished
0709 13:18:27.443021 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:27.443161 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5048,"prepare.queue_time_us":46731,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":2978,"raft.queue_time_us":65625,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":190,"spinlock_wait_cycles":34248832,"thread_start_us":196993,"threads_started":1,"wal-append.queue_time_us":256354}
I20260709 13:18:27.449906 14708 consensus_queue.cc:237] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:27.450240 14317 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:27.498634 12780 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
W20260709 13:18:27.451929 14867 log.cc:927] Time spent T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:18:27.416100 14680 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:27.459990 12703 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:27.460424 13379 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:27.416325 14714 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:27.463912 12598 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:27.407444 14591 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:27.470037 14856 log.cc:927] Time spent T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
W20260709 13:18:27.442123 14850 log.cc:927] Time spent T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:18:27.482316 12529 catalog_manager.cc:5697] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 1 to 3, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 3 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:27.507004 12598 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:27.519137 12774 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:27.505738 12597 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:27.490906 14774 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:27.519887 12598 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:27.521399 12529 catalog_manager.cc:5697] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 1 to 2, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 2 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:27.434173 14846 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:27.522842 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 124732us
I20260709 13:18:27.537385 12707 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:27.443840 14808 consensus_queue.cc:1059] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:18:27.488353 14816 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:18:27.522301 12774 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:27.515482 14552 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:27.515154 14714 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.528122 12529 catalog_manager.cc:5697] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:18:27.542898 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 122916us
I20260709 13:18:27.541206 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:27.581318 12707 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:27.533624 14859 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:27.578601 14714 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:27.539577 14825 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:27.529738 14725 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:27.533469 14857 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:27.583328 12529 catalog_manager.cc:5697] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 1 to 4, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 4 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:18:27.667547 12529 catalog_manager.cc:5697] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:27.574388 12704 raft_consensus.cc:1275] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:18:27.583046 14848 log.cc:927] Time spent T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.188s	user 0.000s	sys 0.003s
I20260709 13:18:27.594726 12700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1476) took 1010 ms. Trace:
I20260709 13:18:27.545049 14552 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.679318 12529 catalog_manager.cc:5697] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 2 to 3, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 3 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:18:27.683666 12529 catalog_manager.cc:5697] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 1 to 2, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 2 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:18:27.582782 14571 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:27.674634 12700 rpcz_store.cc:276] 0709 13:18:26.583880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.691765 (+107885us) service_pool.cc:224] Handling call
0709 13:18:26.692287 (+   522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:26.757723 (+ 65436us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:18:26.764652 (+  6929us) log.cc:844] Serialized 14 byte log entry
0709 13:18:27.045656 (+281004us) spinlock_profiling.cc:243] Waited 3.42 ms on lock 0x7b18003f1f60. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf5223a 00007f4c6d3bca2a
0709 13:18:27.257157 (+211501us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:27.277973 (+ 20816us) log.cc:844] Serialized 26 byte log entry
0709 13:18:27.314454 (+ 36481us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:27.314509 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.315948 (+  1439us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:27.579391 (+263443us) raft_consensus.cc:1685] finished
0709 13:18:27.594426 (+ 15035us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:27.594624 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18739,"prepare.queue_time_us":11213,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":4066,"raft.queue_time_us":67391,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":28486,"spinlock_wait_cycles":7876096,"thread_start_us":210679,"threads_started":1,"wal-append.queue_time_us":551719}
I20260709 13:18:27.540858 14761 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.125s
W20260709 13:18:27.588433 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 186479us
I20260709 13:18:27.670801 13349 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:27.671523 14851 log.cc:927] Time spent T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.290s	user 0.000s	sys 0.001s
I20260709 13:18:27.686583 14552 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:27.743471 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.597659 14862 consensus_queue.cc:1059] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.131s
I20260709 13:18:27.748138 14862 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:27.729658 14846 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:27.724665 14863 log.cc:927] Time spent T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.322s	user 0.000s	sys 0.001s
I20260709 13:18:27.748594 14862 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.749275 14862 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:27.750066 12779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 1826) took 1373 ms. Trace:
I20260709 13:18:27.750597 12779 rpcz_store.cc:276] 0709 13:18:26.376469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.863959 (+487490us) service_pool.cc:224] Handling call
0709 13:18:26.864415 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:27.185743 (+321328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:27.185775 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:27.186328 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 13:18:27.313225 (+126897us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:27.313284 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.313375 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:27.749806 (+436431us) raft_consensus.cc:1685] finished
0709 13:18:27.749833 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:27.749991 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":124642,"raft.queue_time_us":129391,"raft.run_cpu_time_us":62,"raft.run_wall_time_us":63,"thread_start_us":313804,"threads_started":2,"wal-append.queue_time_us":187247}
I20260709 13:18:27.712250 14571 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:27.753954 14317 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:27.755201 14591 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:27.755761 14571 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:27.755862 14732 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:27.686966 14857 raft_consensus.cc:697] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:27.757318 14857 consensus_queue.cc:237] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:27.764329 13385 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:27.767213 12597 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:27.752457 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:27.813102 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 46013.6us
I20260709 13:18:27.670332 14523 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:18:27.812701 14571 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:27.822264 14882 log.cc:927] Time spent T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:18:27.821985 14591 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:27.730906 14762 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:27.803397 14825 raft_consensus.cc:697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:27.854317 13379 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:18:27.855819 12779 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:27.859177 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:27.861012 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:27.861775 13385 raft_consensus.cc:2376] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4. Re-sending same reply.
I20260709 13:18:27.862895 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:27.863605 13384 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:27.856477 14591 raft_consensus.cc:697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:27.866739 14591 consensus_queue.cc:237] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:27.867904 12778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 1822) took 1607 ms. Trace:
I20260709 13:18:27.868376 12778 rpcz_store.cc:276] 0709 13:18:26.260149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.706279 (+446130us) service_pool.cc:224] Handling call
0709 13:18:26.706682 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:26.977815 (+271133us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:26.977879 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:26.978513 (+   634us) log.cc:844] Serialized 26 byte log entry
0709 13:18:27.264946 (+286433us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:27.310387 (+ 45441us) log.cc:844] Serialized 14 byte log entry
0709 13:18:27.322427 (+ 12040us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.322536 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:27.456935 (+134399us) spinlock_profiling.cc:243] Waited 7.98 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:18:27.867576 (+410641us) raft_consensus.cc:1685] finished
0709 13:18:27.867680 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:27.867828 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":286853,"prepare.queue_time_us":47791,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":301920,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":14386,"spinlock_wait_cycles":18356096,"thread_start_us":259780,"threads_started":1,"wal-append.queue_time_us":387002}
I20260709 13:18:27.870769 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:27.871361 12778 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:27.736373 14869 consensus_queue.cc:1059] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.110s
I20260709 13:18:27.874233 14869 raft_consensus.cc:3055] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:27.874590 14869 raft_consensus.cc:740] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:27.875491 14869 consensus_queue.cc:260] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:27.876684 14869 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:27.879137 14762 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:27.879776 14816 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.880086 12778 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:27.880311 14816 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:27.881093 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:27.881670 12778 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:27.882484 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:27.883448 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:27.883929 13377 raft_consensus.cc:3055] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:27.884443 13377 raft_consensus.cc:740] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:27.891142 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:27.892989 14823 raft_consensus.cc:3055] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:27.893429 14823 raft_consensus.cc:740] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:27.894397 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:27.894270 14823 consensus_queue.cc:260] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:27.895066 12778 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:27.895710 14823 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:18:27.895599 14564 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:27.897006 12776 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:27.897948 12783 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:27.898903 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:27.906121 12778 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:27.911358 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:27.912107 13379 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:27.842625 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:27.913018 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:27.914826 14183 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.930349 14825 consensus_queue.cc:237] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:27.933470 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:27.835032 14571 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:27.936215 14613 raft_consensus.cc:3055] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:27.936929 14613 raft_consensus.cc:740] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:18:27.938212 14872 log.cc:927] Time spent T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:18:27.938133 14613 consensus_queue.cc:260] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:27.939612 14613 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:27.947938 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:27.948431 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.928970 14183 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:27.883076 14816 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:27.920440 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:27.849303 14859 raft_consensus.cc:697] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:18:27.953926 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 356373us
I20260709 13:18:27.962738 14183 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:27.972749 12597 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:27.973505 14744 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.974064 14744 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:27.974543 14879 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.975028 14879 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:27.976179 14744 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:27.976832 14771 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.977734 14762 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:27.978300 14762 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:27.977356 14771 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:27.982445 14571 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:27.983485 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:27.971346 12746 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:27.985373 14571 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:27.985942 14571 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.986287 14571 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:27.988032 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:27.988708 12700 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:27.990834 14876 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:27.950273 13381 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:27.951970 14689 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.001042 13349 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:27.885227 13377 consensus_queue.cc:260] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:28.032549 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 59908.4us
I20260709 13:18:28.041337 12597 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:28.044117 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:28.037111 13385 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:28.031193 14689 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:28.067003 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:28.067842 14564 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:28.038868 14870 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:28.068574 14689 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:28.068830 14870 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:27.983242 14771 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:28.069295 14870 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:28.070698 14682 consensus_meta.cc:287] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.578s	user 0.009s	sys 0.000s
I20260709 13:18:28.071141 14682 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:28.072170 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:28.073805 12783 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:18:27.952950 14680 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:27.954631 14714 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:27.994565 14756 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:27.970557 14874 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:28.081049 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:28.031399 14847 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:28.081810 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:28.083899 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:28.109860 12700 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:28.060364 14876 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:28.074283 14888 log.cc:927] Time spent T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.084s	user 0.005s	sys 0.000s
I20260709 13:18:28.109715 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:28.098441 12598 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:27.972558 14508 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:28.114529 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:28.032917 14523 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:18:28.136592 12708 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:18:28.078135 14860 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:28.118934 12777 raft_consensus.cc:2393] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 6: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:28.100476 13377 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:28.101491 14857 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.148455 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:28.150800 12707 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:28.152693 12707 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:18:28.161834 12781 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:28.162887 12709 raft_consensus.cc:2393] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 5: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:27.998536 14890 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.146s
I20260709 13:18:28.147812 14860 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:28.168532 14857 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:28.189738 14714 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:28.190397 14860 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:28.191197 14857 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:28.169695 14884 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:28.170248 12672 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:28.171170 14873 log.cc:927] Time spent T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.224s	user 0.000s	sys 0.000s
I20260709 13:18:28.172012 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:28.172955 12779 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:28.193334 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:28.113867 14892 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:28.174561 14680 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:28.175163 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:28.177644 12700 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:28.208062 12707 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:18:28.208949 14858 consensus_queue.cc:1059] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:28.209865 14588 raft_consensus.cc:3055] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:28.163617 14756 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:28.207284 12672 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:28.220432 12672 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:28.178200 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 79858.8us
I20260709 13:18:27.983354 14859 consensus_queue.cc:237] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:28.201715 12706 raft_consensus.cc:1275] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:18:28.196642 13349 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:28.222812 12706 pending_rounds.cc:85] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 2
I20260709 13:18:28.223199 12706 pending_rounds.cc:107] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:18:28.223515 12706 raft_consensus.cc:2889] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:28.223788 13349 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:28.210225 14588 raft_consensus.cc:740] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:28.116915 14862 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:28.193638 14708 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.181795 14752 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.231578 12707 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:28.136937 14876 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:28.197284 14823 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:28.128180 14847 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:28.206609 14884 raft_consensus.cc:697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:28.210582 14732 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:28.258404 14891 consensus_queue.cc:1059] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:28.229053 14714 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:28.286504 14884 consensus_queue.cc:237] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:28.287499 14823 raft_consensus.cc:3055] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:28.287920 14823 raft_consensus.cc:740] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:28.288728 14823 consensus_queue.cc:260] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:28.289312 14887 log.cc:927] Time spent T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 13:18:28.293872 14860 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:28.306865 14746 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:28.307441 14746 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:28.308194 14847 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:28.255724 14752 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:28.309430 14746 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:28.310791 14860 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:28.311393 14762 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:28.311926 13378 raft_consensus.cc:2410] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:18:28.313766 14879 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:28.265549 12628 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:18:28.266218 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:28.266821 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.268364 12747 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:28.333144 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 64850.1us
W20260709 13:18:28.270854 14894 log.cc:927] Time spent T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:18:28.264226 14870 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:28.270587 14588 consensus_queue.cc:260] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:28.274446 14886 log.cc:927] Time spent T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:18:28.274981 13381 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:28.277817 12632 raft_consensus.cc:1275] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:28.278473 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:28.280642 13348 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:28.284979 12779 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:28.262526 14508 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:28.291045 14862 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:28.233496 14808 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:28.291946 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:28.263178 14876 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:28.295550 14857 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:28.248853 14613 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:28.298784 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:28.300040 14571 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:28.374248 12634 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:28.231175 14756 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:28.210537 14817 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.092s
I20260709 13:18:28.230276 14680 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:28.116282 14874 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:28.235208 14708 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:28.255453 14892 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.233176 14858 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:28.305807 14552 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:28.284332 14774 consensus_meta.cc:287] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.748s	user 0.009s	sys 0.000s
I20260709 13:18:28.268282 14689 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:28.336504 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:28.337272 12630 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:28.374673 14508 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:28.338819 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:18:28.361043 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 80481.3us
I20260709 13:18:28.362418 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:28.408283 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:28.310789 14752 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:28.365150 13382 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:28.334550 14879 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:28.364328 14183 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.300503 14738 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:28.376060 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:28.381256 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.364362 14869 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:28.364598 14808 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:28.386113 14876 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:28.375378 14588 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:28.359117 14761 consensus_queue.cc:1048] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:18:28.377288 14613 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:28.399358 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 62909.9us
I20260709 13:18:28.445408 12747 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:28.337630 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:28.412230 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 187309us
I20260709 13:18:28.413169 14183 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:28.447137 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:28.400666 14712 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:28.418864 12597 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:28.377449 14857 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:28.420548 12633 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:18:28.402390 14862 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.438802 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:28.439782 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:28.464129 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:28.412890 14847 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:28.447726 12627 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:28.415177 14879 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:28.455906 13348 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.583s (SENT) [suppressed 9 similar messages]
I20260709 13:18:28.392095 14680 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:28.465288 12707 raft_consensus.cc:2393] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 6: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:28.520465 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:28.466029 12775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 1839) took 1030 ms. Trace:
I20260709 13:18:28.521255 12775 rpcz_store.cc:276] 0709 13:18:27.435351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.666094 (+230743us) service_pool.cc:224] Handling call
0709 13:18:27.705055 (+ 38961us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:27.728155 (+ 23100us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:27.728193 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:27.728777 (+   584us) log.cc:844] Serialized 26 byte log entry
0709 13:18:27.847810 (+119033us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:27.856167 (+  8357us) log.cc:844] Serialized 14 byte log entry
0709 13:18:27.918393 (+ 62226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.918472 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:28.465766 (+547294us) raft_consensus.cc:1685] finished
0709 13:18:28.465793 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:28.465953 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32019,"prepare.queue_time_us":8493,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":92637,"raft.run_cpu_time_us":314,"raft.run_wall_time_us":24093,"thread_start_us":92965,"threads_started":1,"wal-append.queue_time_us":343772}
I20260709 13:18:28.436030 14808 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:28.475793 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:28.531908 12703 raft_consensus.cc:2393] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 6: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:28.532521 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:28.533252 12707 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:28.533543 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:28.524597 12700 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:18:28.535864 12706 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:18:28.537751 14183 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:28.475175 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.525677 12702 raft_consensus.cc:1240] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:18:28.433715 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:28.541174 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:28.439181 14869 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:28.456251 14613 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:28.543093 12632 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:18:28.402556 14774 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:28.397498 14892 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:28.538910 14847 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:28.414182 14508 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:28.468559 14725 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:28.453979 14876 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:28.463472 14684 consensus_queue.cc:1048] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.088s
I20260709 13:18:28.471812 12634 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:28.604575 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:28.605959 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:28.607156 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:28.553824 14689 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:28.555019 14680 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:28.555101 14900 log.cc:927] Time spent T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
W20260709 13:18:28.556099 14895 log.cc:927] Time spent T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 13:18:28.556510 14552 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:28.556576 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:28.471741 14682 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:28.571625 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:28.622365 12702 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:28.543949 12631 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:28.558378 14571 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:28.528093 14899 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:28.597072 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:28.607348 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:28.618654 12629 raft_consensus.cc:2393] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 5: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:28.619653 12628 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:28.529076 14862 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:28.562346 14869 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:28.621860 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:28.640623 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 184696us
I20260709 13:18:28.529348 14879 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:28.624109 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:28.626031 12625 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:18:28.596680 13381 raft_consensus.cc:2368] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4.
W20260709 13:18:28.627521 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 177469us
I20260709 13:18:28.629438 12634 raft_consensus.cc:1275] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:28.596541 14847 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:28.641688 12778 raft_consensus.cc:2410] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:18:28.598644 14906 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:28.618002 14725 raft_consensus.cc:697] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:18:28.632834 14896 log.cc:927] Time spent T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 13:18:28.664116 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:28.668840 12634 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:28.653893 14870 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:28.627624 14877 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:18:28.682855 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:28.683252 14876 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:28.660270 14761 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.684055 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:28.642694 14682 raft_consensus.cc:697] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:28.670636 14571 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:28.721331 12701 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:18:28.700369 14907 log.cc:927] Time spent T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:18:28.700824 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:28.702350 14682 consensus_queue.cc:237] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:28.754649 12709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1125) took 1354 ms (client timeout 1645 ms). Trace:
I20260709 13:18:28.708151 12776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 1840) took 1255 ms. Trace:
I20260709 13:18:28.709492 14892 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:28.710531 14857 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:28.711364 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:28.712510 12781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 213) took 1315 ms. Trace:
I20260709 13:18:28.712680 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:28.760429 13383 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
W20260709 13:18:28.713290 14855 log.cc:927] Time spent T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 1.176s	user 0.003s	sys 0.009s
I20260709 13:18:28.714181 14916 consensus_queue.cc:1059] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:28.715461 12708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1129) took 1166 ms. Trace:
W20260709 13:18:28.717840 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.764909 12746 leader_election.cc:400] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4.
W20260709 13:18:28.694041 14902 log.cc:927] Time spent T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:18:28.793341 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:18:28.793970 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:28.794070 12777 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54346 (request call id 209) took 1405 ms (client timeout 1583 ms). Trace:
W20260709 13:18:28.755322 12709 rpcz_store.cc:269] 0709 13:18:27.367045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.367514 (+   469us) service_pool.cc:224] Handling call
0709 13:18:28.720883 (+1353369us) spinlock_profiling.cc:243] Waited 794 ms on lock 0x7b54003b33d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:28.721065 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1826675584}
I20260709 13:18:28.794675 14892 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.759306 12781 rpcz_store.cc:276] 0709 13:18:27.397047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.447385 (+ 50338us) service_pool.cc:224] Handling call
0709 13:18:27.447679 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:28.162427 (+714748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:28.712289 (+549862us) spinlock_profiling.cc:243] Waited 714 ms on lock 0x7b540011ce54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:28.712451 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1641852672}
I20260709 13:18:28.795192 14892 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:28.795717 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:28.764396 12708 rpcz_store.cc:276] 0709 13:18:27.548957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.549139 (+   182us) service_pool.cc:224] Handling call
0709 13:18:27.549471 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:28.146011 (+596540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:28.715198 (+569187us) spinlock_profiling.cc:243] Waited 587 ms on lock 0x7b5400172ad4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:28.715400 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1349706624}
I20260709 13:18:28.796147 14756 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:28.794523 12777 rpcz_store.cc:269] 0709 13:18:27.388632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.389677 (+  1045us) service_pool.cc:224] Handling call
0709 13:18:28.793878 (+1404201us) spinlock_profiling.cc:243] Waited 689 ms on lock 0x7b54001fe354. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:28.794014 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1585573760}
I20260709 13:18:28.765202 12746 leader_election.cc:403] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:18:28.798377 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 74527.7us
I20260709 13:18:28.798976 14874 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:28.792650 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:28.799544 14874 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:28.794235 13381 raft_consensus.cc:2393] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 6: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:28.807729 12783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 216) took 1219 ms. Trace:
I20260709 13:18:28.808166 12783 rpcz_store.cc:276] 0709 13:18:27.588160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.900603 (+312443us) service_pool.cc:224] Handling call
0709 13:18:27.900872 (+   269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:28.074289 (+173417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:28.807474 (+733185us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54001ff254. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:28.807665 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":396580736}
I20260709 13:18:28.641754 14899 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:28.809885 14689 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:28.810773 14899 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:28.810489 14689 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:28.811442 14317 consensus_queue.cc:1048] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:28.812909 14317 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:28.813125 14689 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:28.815747 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:28.817966 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:28.819392 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:28.820407 12707 raft_consensus.cc:2410] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:18:28.724067 14761 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:28.797295 14892 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:28.836561 12782 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.017s	sys 0.027s
I20260709 13:18:28.833834 12707 raft_consensus.cc:1275] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:28.756239 12776 rpcz_store.cc:276] 0709 13:18:27.452875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.899403 (+446528us) service_pool.cc:224] Handling call
0709 13:18:27.899738 (+   335us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:27.954877 (+ 55139us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:27.954923 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:27.969381 (+ 14458us) log.cc:844] Serialized 52 byte log entry
0709 13:18:28.306407 (+337026us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b1800313600. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf520d1 00007f4c6d36da0f
0709 13:18:28.467298 (+160891us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:28.467365 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:28.467435 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:28.707806 (+240371us) raft_consensus.cc:1685] finished
0709 13:18:28.707831 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:28.708081 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49632,"raft.queue_time_us":58150,"raft.run_cpu_time_us":555,"raft.run_wall_time_us":6246,"spinlock_wait_cycles":41914752,"thread_start_us":160706,"threads_started":1,"wal-append.queue_time_us":679723}
I20260709 13:18:28.721500 14684 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:28.724254 14571 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:28.832175 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 127701us
I20260709 13:18:28.818760 12702 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:28.842096 12747 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:28.843623 12747 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:28.843705 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:28.838187 12782 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.017s	sys 0.027s
I20260709 13:18:28.846406 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:28.847754 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:28.850024 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:28.850797 12629 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:28.851938 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:28.852346 12628 raft_consensus.cc:2393] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 4: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:28.764693 14708 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:28.813503 14317 consensus_queue.cc:1243] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:18:28.853900 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:28.855059 12778 raft_consensus.cc:2393] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 5: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:28.870111 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:28.870751 13383 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:28.872303 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:28.872954 12706 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:28.844789 14571 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:28.876700 14912 log.cc:927] Time spent T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:18:28.883683 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:28.884414 12706 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
W20260709 13:18:28.881385 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.852039 14689 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:28.888566 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:28.862807 14745 consensus_queue.cc:1048] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:18:28.855353 14876 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:28.889010 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:28.848220 14756 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:18:28.926568 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:28.845091 14684 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:28.856355 14879 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:28.839201 14725 consensus_queue.cc:237] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:28.963850 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:28.920913 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.894512 14874 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4.
I20260709 13:18:28.929831 14508 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:28.894658 14708 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:28.890343 14714 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:28.899266 14774 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:28.986467 14714 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:28.989164 14714 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:28.984686 12783 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:18:28.991643 12782 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:28.991956 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:28.997957 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:28.999506 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:29.001682 12775 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:29.006533 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:29.007691 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:28.996795 14317 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:29.010751 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:28.931316 14921 consensus_queue.cc:1059] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.099s
I20260709 13:18:29.011094 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:29.221642 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 367789us
I20260709 13:18:29.221756 14899 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:29.222182 12776 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:29.000630 14714 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.222517 14899 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:29.222842 14714 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:29.160751 14771 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.022s	sys 0.022s
I20260709 13:18:29.224959 14899 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:29.206990 14756 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:29.225040 14771 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.026s	sys 0.029s
I20260709 13:18:29.097977 12703 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:18:29.227298 12747 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:29.258989 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:29.261096 12747 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:29.266037 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:29.273511 12703 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:29.274998 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:29.127355 14523 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:29.077606 14924 log.cc:927] Time spent T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:18:29.020813 12782 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:29.099201 14877 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.275777 14922 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:29.280858 14877 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:29.125115 14745 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:18:29.273640 13377 consensus_meta.cc:287] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 1.110s	user 0.012s	sys 0.000s
I20260709 13:18:29.210538 14774 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:29.266250 14588 consensus_meta.cc:287] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.819s	user 0.009s	sys 0.002s
I20260709 13:18:29.280088 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:29.099617 14708 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:29.086768 14879 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:29.288287 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:29.297695 12703 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:29.257140 14874 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:18:29.272336 14682 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:18:29.289641 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 284624us
I20260709 13:18:29.292953 12782 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:29.313323 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:29.232009 14761 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:29.282965 14869 consensus_meta.cc:287] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.616s	user 0.007s	sys 0.002s
W20260709 13:18:29.283138 14808 consensus_meta.cc:287] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.736s	user 0.010s	sys 0.001s
I20260709 13:18:29.021736 14892 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:29.098181 14876 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:29.279151 14921 consensus_queue.cc:1243] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:18:29.296779 14684 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:29.184303 14820 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:29.311347 12708 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:29.266426 14756 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:29.013900 14772 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.282s
I20260709 13:18:29.021751 14317 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:29.294226 14870 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:29.326630 14682 raft_consensus.cc:697] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:29.298734 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:29.325207 13377 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:29.382452 13377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 440) took 1974 ms. Trace:
I20260709 13:18:29.383488 12779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1617) took 1531 ms. Trace:
I20260709 13:18:29.384080 12779 rpcz_store.cc:276] 0709 13:18:27.852294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.195360 (+343066us) service_pool.cc:224] Handling call
0709 13:18:28.196841 (+  1481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:28.360903 (+164062us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.383175 (+1022272us) spinlock_profiling.cc:243] Waited 87.7 ms on lock 0x7b54000cc954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:29.383415 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":201770880}
I20260709 13:18:29.386675 12701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 206) took 1238 ms. Trace:
I20260709 13:18:29.387194 12701 rpcz_store.cc:276] 0709 13:18:28.148628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.153696 (+  5068us) service_pool.cc:224] Handling call
0709 13:18:28.721045 (+567349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:28.721997 (+   952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.386437 (+664440us) spinlock_profiling.cc:243] Waited 567 ms on lock 0x7b54003b33d0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:29.386604 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1303858048}
I20260709 13:18:29.349834 14820 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:29.392498 14820 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:29.312306 14771 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:29.347158 13383 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
W20260709 13:18:29.328377 14847 consensus_meta.cc:287] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.661s	user 0.008s	sys 0.002s
I20260709 13:18:29.349795 12703 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:29.282609 14523 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:29.292871 14745 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:29.353022 14930 log.cc:927] Time spent T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:18:29.353353 14317 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:29.353957 14870 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:29.355819 14682 consensus_queue.cc:237] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:29.357491 14932 log.cc:927] Time spent T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 13:18:29.358783 14877 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:29.361908 14756 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:29.362763 12707 raft_consensus.cc:1275] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:18:29.363929 14712 consensus_meta.cc:287] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.895s	user 0.007s	sys 0.003s
I20260709 13:18:29.332532 14808 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:29.410487 12781 raft_consensus.cc:2410] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:18:29.342665 14921 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:29.327275 14892 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:29.358186 14571 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:29.365221 14891 raft_consensus.cc:3055] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:29.330189 14869 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:29.321245 14879 consensus_queue.cc:1048] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:18:29.337380 14876 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:29.294601 14774 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:29.373507 14772 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.302953 14708 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:29.383721 14861 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:18:29.314152 14508 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:29.396174 13383 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:29.437151 13383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 448) took 1002 ms. Trace:
W20260709 13:18:29.437240 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 122612us
I20260709 13:18:29.437865 13383 rpcz_store.cc:276] 0709 13:18:28.434577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.395398 (+960821us) service_pool.cc:224] Handling call
0709 13:18:29.395767 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:29.436827 (+ 41060us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.437081 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:29.399803 12705 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:29.333411 14761 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:29.396999 14771 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:29.382973 13377 rpcz_store.cc:276] 0709 13:18:27.408180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.412910 (+  4730us) service_pool.cc:224] Handling call
0709 13:18:27.883496 (+470586us) spinlock_profiling.cc:243] Waited 1.86 ms on lock 0x7b5800095198. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c7238bfc0 00007f4c72376929 00007f4c723763d2 00007f4c6b8edbd4
0709 13:18:27.883770 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:29.382040 (+1498270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.382355 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4282624}
W20260709 13:18:29.413136 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 141568us
W20260709 13:18:29.444341 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.399444 14847 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:29.419948 13385 raft_consensus.cc:1240] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:29.446424 13382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51762 (request call id 464) took 1300 ms. Trace:
I20260709 13:18:29.447649 12775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1627) took 1036 ms. Trace:
I20260709 13:18:29.448158 12775 rpcz_store.cc:276] 0709 13:18:28.410763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.802298 (+391535us) service_pool.cc:224] Handling call
0709 13:18:28.802608 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:29.002104 (+199496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.447357 (+445253us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54003985d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:29.447567 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":441484544}
W20260709 13:18:29.420909 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 157864us
I20260709 13:18:29.452319 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:29.423198 12777 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:29.431123 14922 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:29.413151 14523 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:29.435781 12633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1091) took 1160 ms. Trace:
I20260709 13:18:29.416236 14891 raft_consensus.cc:740] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:18:29.434427 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.412845 14921 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:29.454763 12747 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:29.479089 12633 rpcz_store.cc:276] 0709 13:18:28.275724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.277894 (+  2170us) service_pool.cc:224] Handling call
0709 13:18:28.278219 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:28.465664 (+187445us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.432393 (+966729us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b54000de154. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:29.435700 (+  3307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326775552}
I20260709 13:18:29.446790 13382 rpcz_store.cc:276] 0709 13:18:28.145928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.148208 (+  2280us) service_pool.cc:224] Handling call
0709 13:18:29.446248 (+1298040us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b54003890d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:29.446360 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":496894208}
I20260709 13:18:29.443565 14771 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:29.442025 14508 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:29.494840 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.479302 14680 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:29.495633 14680 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:29.496313 14869 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:29.495100 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:29.498656 12634 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
I20260709 13:18:29.499656 13349 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:29.500134 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:29.482771 14756 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:29.445859 14761 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:29.481452 14689 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:29.433370 14772 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:29.488715 14317 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:29.488444 14939 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.488013 13378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51762 (request call id 457) took 1446 ms. Trace:
I20260709 13:18:29.503381 14756 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:29.580204 14756 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:29.579896 14939 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:29.580960 14508 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:29.582024 12777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1624) took 1259 ms. Trace:
I20260709 13:18:29.582195 14939 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:29.582417 12777 rpcz_store.cc:276] 0709 13:18:28.322275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.798989 (+476714us) service_pool.cc:224] Handling call
0709 13:18:28.799242 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:29.486592 (+687350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.581762 (+ 95170us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b54003700d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:29.581957 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1434555136}
I20260709 13:18:29.579823 14523 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:29.584614 14523 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:29.584973 14523 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:29.585840 14919 log.cc:927] Time spent T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.822s	user 0.005s	sys 0.000s
I20260709 13:18:29.587342 14761 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:29.587720 13380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 447) took 1165 ms. Trace:
I20260709 13:18:29.588089 13377 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:29.588346 14756 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:29.588932 13377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 451) took 1133 ms. Trace:
I20260709 13:18:29.550625 14689 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:29.590092 12700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1143) took 1042 ms. Trace:
I20260709 13:18:29.590607 12700 rpcz_store.cc:276] 0709 13:18:28.547668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.547846 (+   178us) service_pool.cc:224] Handling call
0709 13:18:28.548216 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:28.548602 (+   386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:28.548627 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:28.549168 (+   541us) log.cc:844] Serialized 26 byte log entry
0709 13:18:28.622952 (+ 73784us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:28.622990 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:28.623139 (+   149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:29.585087 (+961948us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:18:29.589782 (+  4695us) raft_consensus.cc:1685] finished
0709 13:18:29.589807 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:29.590028 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1137,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"spinlock_wait_cycles":64630144,"thread_start_us":43488,"threads_started":1,"wal-append.queue_time_us":124366}
I20260709 13:18:29.588132 13380 rpcz_store.cc:276] 0709 13:18:28.421821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.367779 (+945958us) service_pool.cc:224] Handling call
0709 13:18:29.368125 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:29.411600 (+ 43475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:29.411646 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:29.411725 (+    79us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:29.411778 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.587341 (+175563us) spinlock_profiling.cc:243] Waited 43.2 ms on lock 0x7b5400730354. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:29.587394 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:29.587538 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":99365632}
I20260709 13:18:29.548908 14317 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:29.542479 14820 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:29.604298 14317 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:29.604569 14689 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:29.604799 14820 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:29.550386 14712 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.607100 14712 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:29.603587 14745 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:29.589375 13377 rpcz_store.cc:276] 0709 13:18:28.455618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.480217 (+1024599us) service_pool.cc:224] Handling call
0709 13:18:29.503227 (+ 23010us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:29.588611 (+ 85384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.588867 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:29.613682 14847 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:29.633437 14756 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:29.639935 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:29.634625 14932 log.cc:927] Time spent T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:18:29.638142 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:29.569608 13378 rpcz_store.cc:276] 0709 13:18:28.036618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.060277 (+ 23659us) service_pool.cc:224] Handling call
0709 13:18:29.446031 (+1385754us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b540026cad4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:29.483164 (+ 37133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":549152640}
I20260709 13:18:29.655061 14921 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:29.653978 14571 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:29.570079 14891 consensus_queue.cc:260] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:29.681723 14808 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
W20260709 13:18:29.704070 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 265359us
I20260709 13:18:29.704435 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:29.705421 12700 raft_consensus.cc:2410] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:18:29.706740 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:29.718664 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:29.691062 14892 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:29.724436 12700 raft_consensus.cc:2410] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:18:29.690593 14712 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:29.735327 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:29.736179 12704 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:29.738616 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:29.701719 14946 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.722800 14772 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:29.732275 14820 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:29.723089 14708 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:29.750511 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 305406us
I20260709 13:18:29.738034 14571 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:18:29.773010 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 225360us
I20260709 13:18:29.775455 14808 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:29.775584 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:29.739045 14876 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:29.784475 12747 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:29.730998 14317 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:29.786823 13383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 449) took 1351 ms. Trace:
I20260709 13:18:29.789988 12747 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:29.778236 14921 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.795734 14921 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:29.797703 12780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1612) took 1973 ms. Trace:
I20260709 13:18:29.797969 14921 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:29.790110 14876 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:29.793529 14689 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:29.798157 12780 rpcz_store.cc:276] 0709 13:18:27.824108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.141055 (+316947us) service_pool.cc:224] Handling call
0709 13:18:28.141431 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:28.403555 (+262124us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:18:28.419733 (+ 16178us) log.cc:844] Serialized 14 byte log entry
0709 13:18:28.738497 (+318764us) log.cc:844] Serialized 14 byte log entry
0709 13:18:28.739269 (+   772us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:28.739318 (+    49us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:28.739371 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.451504 (+712133us) spinlock_profiling.cc:243] Waited 5.46 ms on lock 0x7b1800294a00. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2932a6 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419
0709 13:18:29.797446 (+345942us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b54001fd954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:29.797491 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:29.797641 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16565,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":87,"raft.queue_time_us":22288,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":28118,"spinlock_wait_cycles":614907520,"thread_start_us":317336,"threads_started":2,"wal-append.queue_time_us":293171}
I20260709 13:18:29.805295 14876 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.805886 14876 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:29.806324 12633 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:29.806485 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:29.807129 12626 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:29.807433 14892 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:29.808094 14876 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:29.807964 14892 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:29.774619 14708 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:29.810271 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:29.809780 12706 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:18:29.837885 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:29.837507 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:29.837422 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:29.787537 14745 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:29.818351 14921 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:29.783389 14756 consensus_queue.cc:1059] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:18:29.808631 14876 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:29.850351 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.783859 14808 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:29.778110 14571 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:29.854118 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:29.818620 14820 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:29.833653 14947 log.cc:927] Time spent T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:18:29.790108 14955 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:29.749469 14946 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:29.881913 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:29.818352 14952 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:29.890666 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:29.891640 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:29.908753 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:29.909807 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 118976us
I20260709 13:18:29.914057 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:29.926781 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:29.851610 14745 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:18:29.906939 14613 consensus_meta.cc:287] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 1.309s	user 0.010s	sys 0.004s
I20260709 13:18:29.931149 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:29.932850 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:29.845764 14708 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:29.927415 12780 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:29.918936 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:29.919440 12630 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:29.938488 14756 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:29.941874 12672 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:29.886507 14876 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:29.917213 14943 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.945797 12777 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:29.944629 14823 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:29.996671 12776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 1848) took 1002 ms. Trace:
I20260709 13:18:29.997407 12630 raft_consensus.cc:1240] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:29.939491 14745 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:30.006258 12776 rpcz_store.cc:276] 0709 13:18:28.994326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.222912 (+228586us) service_pool.cc:224] Handling call
0709 13:18:29.223265 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:29.226296 (+  3031us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:29.226346 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:29.226790 (+   444us) log.cc:844] Serialized 26 byte log entry
0709 13:18:29.510551 (+283761us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:29.510608 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.510703 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:29.996423 (+485720us) raft_consensus.cc:1685] finished
0709 13:18:29.996444 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:29.996600 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2641,"raft.queue_time_us":7844,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":8459,"thread_start_us":231247,"threads_started":1,"wal-append.queue_time_us":452778}
I20260709 13:18:30.011089 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:30.011713 12776 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:29.918738 14921 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:30.008826 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.022677 13348 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:29.939738 14820 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:30.034687 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:30.035420 12777 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:29.955799 14756 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:29.924078 14955 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:29.940939 14952 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:30.017854 14892 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:30.019676 14964 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:29.945623 14613 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:30.056779 12776 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:30.029246 14965 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:18:30.017525 14523 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:30.055756 14950 log.cc:927] Time spent T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.278s	user 0.000s	sys 0.002s
I20260709 13:18:30.010572 14943 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:30.069288 12704 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:30.037906 14508 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:18:30.057982 14934 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.059748 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:30.078433 13379 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 7: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
W20260709 13:18:30.082275 14959 log.cc:927] Time spent T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 13:18:30.068689 14971 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:18:30.068948 14771 consensus_meta.cc:287] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.525s	user 0.007s	sys 0.004s
I20260709 13:18:30.105026 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:30.105741 12747 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:30.108031 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:30.111199 12705 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:30.099776 14862 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:30.101131 14876 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:30.113969 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:30.103755 14708 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:30.092247 14820 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:30.102834 14943 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:30.115360 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 118220us
I20260709 13:18:30.115465 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:30.120788 12709 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:30.095268 14974 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:30.104573 14808 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:30.123194 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:30.135154 12625 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:30.123492 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:30.119349 12627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56838 (request call id 1099) took 1839 ms (client timeout 1684 ms). Trace:
I20260709 13:18:30.104592 14571 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:30.114053 14876 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:30.163635 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:30.107960 14771 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:30.155016 12673 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:30.119709 14960 log.cc:927] Time spent T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.161s	user 0.001s	sys 0.001s
I20260709 13:18:30.170715 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:30.163520 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:30.137523 12627 rpcz_store.cc:269] 0709 13:18:28.279924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.361165 (+ 81241us) service_pool.cc:224] Handling call
0709 13:18:30.119099 (+1757934us) spinlock_profiling.cc:243] Waited 31.4 ms on lock 0x7b54000deb54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:30.119252 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":72331008}
I20260709 13:18:30.106985 14971 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:30.094224 14934 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:30.246877 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 191583us
I20260709 13:18:30.256619 12704 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:30.268683 13348 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:30.242435 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:30.279345 12701 raft_consensus.cc:2393] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 5: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:30.221297 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:30.237583 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1148) took 1365 ms. Trace:
W20260709 13:18:30.259014 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 136418us
I20260709 13:18:30.259644 12709 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:30.125651 14946 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:30.288925 12775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 1855) took 1280 ms. Trace:
I20260709 13:18:30.290596 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:30.293183 12673 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:30.289374 12775 rpcz_store.cc:276] 0709 13:18:29.008375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.449332 (+440957us) service_pool.cc:224] Handling call
0709 13:18:29.491119 (+ 41787us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:29.497664 (+  6545us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:29.497699 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:29.497742 (+    43us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:29.497773 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:30.288685 (+790912us) spinlock_profiling.cc:243] Waited 6.32 ms on lock 0x7b540011c954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:30.288729 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:30.288869 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14537472}
I20260709 13:18:30.293814 13385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 354) took 1876 ms. Trace:
I20260709 13:18:30.281702 12702 rpcz_store.cc:276] 0709 13:18:28.871803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.876338 (+  4535us) service_pool.cc:224] Handling call
0709 13:18:28.876655 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:28.881465 (+  4810us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:28.881496 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:28.883887 (+  2391us) log.cc:844] Serialized 26 byte log entry
0709 13:18:29.633849 (+749962us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b1800262c60. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf520d1 00007f4c6d36da0f
0709 13:18:30.097007 (+463158us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:30.097047 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:30.097098 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:30.237263 (+140165us) raft_consensus.cc:1685] finished
0709 13:18:30.237287 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:30.237497 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25746,"raft.queue_time_us":6882,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":7593,"spinlock_wait_cycles":6671744,"thread_start_us":462952,"threads_started":1,"wal-append.queue_time_us":1285411}
I20260709 13:18:30.296018 12673 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:30.240198 14955 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:30.205581 14876 raft_consensus.cc:697] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:30.237656 14756 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:30.302671 13385 rpcz_store.cc:276] 0709 13:18:28.417671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.926846 (+509175us) service_pool.cc:224] Handling call
0709 13:18:28.927270 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:29.446533 (+519263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:30.289590 (+843057us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b54005d89d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:30.293749 (+  4159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1132439936}
W20260709 13:18:30.305315 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 198447us
I20260709 13:18:30.306700 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:30.307365 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:30.320394 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:30.320870 12634 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:30.238023 14523 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:30.308967 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:30.328023 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:30.240008 14745 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:30.310415 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:30.283838 14965 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:30.316926 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:30.317216 12704 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
W20260709 13:18:30.317874 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 360011us
I20260709 13:18:30.335839 12672 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:30.321161 13385 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:30.322846 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:30.308126 13376 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:30.324129 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:30.296164 14808 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:30.332000 12629 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:30.337750 12779 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
I20260709 13:18:30.323089 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:30.338254 13380 raft_consensus.cc:2368] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6.
I20260709 13:18:30.358316 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:30.357203 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:30.359831 13382 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:30.355173 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:30.363888 13382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 356) took 1700 ms. Trace:
I20260709 13:18:30.358304 12777 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:18:30.364264 13382 rpcz_store.cc:276] 0709 13:18:28.663744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.491718 (+827974us) service_pool.cc:224] Handling call
0709 13:18:30.359331 (+867613us) spinlock_profiling.cc:243] Waited 6.46 ms on lock 0x7b5800095198. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c7238bfc0 00007f4c72376929 00007f4c723763d2 00007f4c6b8edbd4
0709 13:18:30.359503 (+   172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:30.363642 (+  4139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:30.363826 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14851456}
I20260709 13:18:30.365353 13385 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:30.318697 14682 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:30.325845 14571 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.363281 12775 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
I20260709 13:18:30.326303 14712 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:30.368669 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 51800.1us
I20260709 13:18:30.338285 14965 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:30.337270 14920 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.374382 14712 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:30.378666 14571 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:30.381747 12776 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:30.381291 14571 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:30.383018 12629 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:30.363125 14808 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.361011 14921 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:30.338143 14756 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:18:30.389935 13349 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:30.386232 12746 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:30.391482 14808 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:30.376789 12634 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:30.393867 12746 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:30.397034 13349 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:30.398167 14317 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:30.397910 14876 consensus_queue.cc:237] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:30.398307 14847 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:30.402021 14774 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:30.402364 14680 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:30.403674 13348 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:30.407223 14870 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:30.383798 14965 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:30.410925 13379 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51768 (request call id 351) took 2004 ms (client timeout 1501 ms). Trace:
W20260709 13:18:30.411531 13379 rpcz_store.cc:269] 0709 13:18:28.406491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.700529 (+294038us) service_pool.cc:224] Handling call
0709 13:18:30.410645 (+1710116us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400486054. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:30.410828 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3104586624}
I20260709 13:18:30.413112 14934 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:30.405176 13384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 450) took 1962 ms. Trace:
I20260709 13:18:30.415131 13384 rpcz_store.cc:276] 0709 13:18:28.442594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.450630 (+1008036us) service_pool.cc:224] Handling call
0709 13:18:29.451056 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:29.502093 (+ 51037us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:29.502126 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:29.502168 (+    42us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:29.502211 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:30.404796 (+902585us) spinlock_profiling.cc:243] Waited 50.8 ms on lock 0x7b540001aed4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:30.404839 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:30.405117 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116911360}
I20260709 13:18:30.417785 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:30.404919 12746 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:30.371019 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:30.438681 13377 raft_consensus.cc:2410] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:18:30.440011 14965 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:30.414734 14946 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:30.442310 14946 raft_consensus.cc:697] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:30.443224 14946 consensus_queue.cc:237] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:30.405968 14714 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:30.445444 14680 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:30.375487 14682 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:30.446053 14680 raft_consensus.cc:697] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:30.446949 14680 consensus_queue.cc:237] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:30.447494 14682 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:30.448911 14745 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:30.424314 13348 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:30.374758 14823 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:30.414345 14971 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:30.426030 14870 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:30.426630 14771 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:30.390954 13376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 349) took 1983 ms. Trace:
I20260709 13:18:30.422071 14571 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:30.428843 14613 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:30.431854 14317 consensus_queue.cc:1059] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:30.433313 12746 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:30.474581 12746 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:30.433763 14523 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:30.387149 14712 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:30.384817 14920 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:30.455206 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:30.463034 14921 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:30.451457 13348 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:30.391752 13378 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:30.476331 12746 leader_election.cc:400] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6.
I20260709 13:18:30.459841 12777 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:30.489432 12746 leader_election.cc:403] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:30.484773 12634 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:30.487941 14523 raft_consensus.cc:697] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:30.492244 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:30.492142 14523 consensus_queue.cc:237] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:30.494357 14984 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:30.494774 14984 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:30.495144 14984 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:30.472518 14571 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.498112 14571 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:30.474434 14934 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:30.501904 14934 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:30.502341 14934 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:30.508754 13349 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:30.510048 14823 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:30.513286 12746 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:30.516342 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:30.517477 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:30.478133 14870 raft_consensus.cc:697] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:30.486510 14921 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.488051 14971 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.486059 14991 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:30.526587 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:30.511126 14712 raft_consensus.cc:3055] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:30.518376 14317 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6.
I20260709 13:18:30.480633 14771 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:18:30.524219 14876 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:30.464154 13376 rpcz_store.cc:276] 0709 13:18:28.405854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.420005 (+ 14151us) service_pool.cc:224] Handling call
0709 13:18:30.233820 (+1813815us) spinlock_profiling.cc:243] Waited 7.4 ms on lock 0x7b5800095198. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c7238bfc0 00007f4c72376929 00007f4c723763d2 00007f4c6b8edbd4
0709 13:18:30.288955 (+ 55135us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:30.370934 (+ 81979us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:30.389500 (+ 18566us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17025792}
I20260709 13:18:30.526387 14965 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:30.528662 14921 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:30.529855 14991 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:30.530969 14712 raft_consensus.cc:740] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:30.545548 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:30.530696 14971 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:30.575877 12781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36488 (request call id 1623) took 2253 ms. Trace:
I20260709 13:18:30.576303 12781 rpcz_store.cc:276] 0709 13:18:28.322003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.795471 (+473468us) service_pool.cc:224] Handling call
0709 13:18:30.575649 (+1780178us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b540040e554. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:30.575810 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1411859712}
I20260709 13:18:30.576484 14712 consensus_queue.cc:260] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:30.577595 14712 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:30.578984 12778 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:18:30.579342 12778 pending_rounds.cc:85] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 2
I20260709 13:18:30.579566 12778 pending_rounds.cc:107] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:18:30.579917 12778 raft_consensus.cc:2889] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:30.566744 14991 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:30.575086 14992 log.cc:927] Time spent T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:18:30.611055 12783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 218) took 2065 ms. Trace:
I20260709 13:18:30.605329 13377 raft_consensus.cc:2410] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:18:30.611500 12783 rpcz_store.cc:276] 0709 13:18:28.546612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.809078 (+262466us) service_pool.cc:224] Handling call
0709 13:18:28.809408 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:28.994962 (+185554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:30.610644 (+1615682us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b54002cae54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:30.610989 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":402548608}
I20260709 13:18:30.614928 14808 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:30.617105 14808 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:30.593156 14984 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:30.624825 14988 log.cc:927] Time spent T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.102s	user 0.001s	sys 0.001s
I20260709 13:18:30.625137 12783 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:33732
W20260709 13:18:30.626441 14990 log.cc:927] Time spent T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:18:30.628484 14920 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:30.630450 12782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 1850) took 1635 ms. Trace:
I20260709 13:18:30.631007 12782 rpcz_store.cc:276] 0709 13:18:28.994966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.326511 (+331545us) service_pool.cc:224] Handling call
0709 13:18:29.326884 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:29.352099 (+ 25215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:29.352132 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:29.352175 (+    43us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:29.352210 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:30.630115 (+1277905us) spinlock_profiling.cc:243] Waited 25 ms on lock 0x7b540040ea54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:30.630190 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:30.630365 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57502080}
I20260709 13:18:30.591576 14870 consensus_queue.cc:237] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:29.787532 13383 rpcz_store.cc:276] 0709 13:18:28.435913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.439879 (+1003966us) service_pool.cc:224] Handling call
0709 13:18:29.440254 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:29.440502 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:29.709073 (+268571us) log.cc:844] Serialized 14 byte log entry
0709 13:18:29.786496 (+ 77423us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:29.786543 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:29.786585 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.786650 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:29.786767 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":335928,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":69,"thread_start_us":345240,"threads_started":2}
I20260709 13:18:30.648995 14680 consensus_queue.cc:1059] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:30.649930 14876 raft_consensus.cc:3055] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:30.650373 14876 raft_consensus.cc:740] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:30.651315 14876 consensus_queue.cc:260] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:30.652576 14876 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:30.662525 14994 log.cc:927] Time spent T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:18:30.665673 14934 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:30.672479 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 70121.2us
I20260709 13:18:30.674640 12634 raft_consensus.cc:1240] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:18:30.683162 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:30.683835 12634 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:30.676137 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:30.676079 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:30.676380 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:30.689357 12626 raft_consensus.cc:2410] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:18:30.690680 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:30.692001 12634 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:30.693286 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:30.693802 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:30.694147 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:30.694823 12634 raft_consensus.cc:2393] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 6: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:30.674649 14892 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:30.696019 12627 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:30.695778 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:30.697522 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:30.696782 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:30.698310 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:30.698662 12625 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:30.698778 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:30.698853 12628 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:30.698800 12631 raft_consensus.cc:2368] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6.
I20260709 13:18:30.695323 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:30.700133 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:30.699616 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:30.699935 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:30.672715 14774 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:30.701375 12625 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:30.703303 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:30.705947 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:30.714550 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:30.715646 12631 raft_consensus.cc:2393] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 4: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:30.695797 14892 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:30.673930 14820 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.719277 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:30.673710 15004 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:30.720510 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:30.728437 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:30.721143 14774 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:30.744246 12625 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:30.730598 14971 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:30.730001 14745 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:30.767751 14745 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:30.731005 15004 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:30.770427 12626 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
W20260709 13:18:30.773355 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 60772.5us
I20260709 13:18:30.774410 12598 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:30.756735 15002 log.cc:927] Time spent T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:18:30.784106 15007 consensus_queue.cc:1048] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:30.762913 14892 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:30.789528 14744 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:30.739639 14934 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:30.730031 14820 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:30.790699 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 494750us
I20260709 13:18:30.791244 14955 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:30.791525 14774 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:30.794718 14980 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:18:30.819782 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 483983us
W20260709 13:18:30.822279 14963 log.cc:927] Time spent T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.794s	user 0.003s	sys 0.000s
I20260709 13:18:30.822557 12628 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:30.831108 12673 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:30.833290 12673 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:30.835728 12630 pending_rounds.cc:85] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 1
I20260709 13:18:30.836146 12630 pending_rounds.cc:107] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:18:30.836464 12630 raft_consensus.cc:2889] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:30.837688 14983 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
W20260709 13:18:30.840516 15006 log.cc:927] Time spent T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:18:30.842510 14948 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:30.843065 14948 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:30.843536 14948 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:30.738054 14571 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:30.832152 12672 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:30.822291 14745 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:30.853729 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.831207 14980 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.863001 14571 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:30.866721 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:30.868322 12747 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:30.846741 14714 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:30.870242 14714 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.832839 14984 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:30.870682 14714 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:30.872939 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:30.879262 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:18:30.881937 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 49906.2us
I20260709 13:18:30.883065 12672 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:30.865109 14980 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:30.890873 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 116577us
I20260709 13:18:30.891281 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:30.892130 12627 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:30.884765 14508 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:30.876060 14876 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:30.876528 14571 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:30.881958 14955 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:30.879531 14934 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:30.907178 12673 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:30.856715 14938 consensus_queue.cc:1059] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:18:30.901233 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:30.897535 14776 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:30.897688 15004 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:30.917936 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:30.923686 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:30.905665 14955 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:30.912524 14986 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.929944 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:30.901855 15007 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:18:30.916060 15015 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:30.934291 12747 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:30.913918 14934 raft_consensus.cc:697] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:30.918288 14680 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:30.902453 14508 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:30.918835 15014 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:30.918465 14876 raft_consensus.cc:697] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:30.923064 14938 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:30.930081 14955 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:30.929008 14847 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:30.947324 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:30.948132 12705 raft_consensus.cc:2393] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 6: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:30.948873 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:30.949618 12705 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:18:30.931494 14317 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:30.948014 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:30.950872 12704 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:30.934424 14682 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:30.960459 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:30.961529 12704 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:30.963637 15025 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:30.964169 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:30.965770 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:30.954114 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:30.924275 14921 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:30.970090 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:30.971069 12705 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:30.972857 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:30.973706 12705 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:30.974296 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:30.943992 14938 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:30.980607 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:30.921828 14991 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:30.921028 14776 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:31.006335 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 72095.5us
I20260709 13:18:31.008951 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:31.009975 12783 raft_consensus.cc:606] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Received request to transfer leadership to TS 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:31.010532 12783 raft_consensus.cc:614] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:30.983239 14938 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:30.977746 14820 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:31.014885 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:31.016610 12702 raft_consensus.cc:2393] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 6: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:30.986800 15024 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:30.940374 14986 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:31.019163 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:30.945653 15014 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:31.003487 15020 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:31.017452 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 45164.7us
I20260709 13:18:31.019654 15026 consensus_queue.cc:1059] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:18:31.026342 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:31.022521 15011 log.cc:927] Time spent T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:18:31.029069 12708 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:31.030872 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:31.031917 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:31.032482 12701 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:31.033891 15026 consensus_queue.cc:1243] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:18:31.034379 15012 log.cc:927] Time spent T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:18:31.018074 14744 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:31.040140 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:31.020016 15024 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:31.040902 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:31.046789 12705 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:31.047890 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:31.025410 15020 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:31.043077 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:31.043596 14986 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:31.045442 12701 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:31.049835 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:31.052752 12705 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:31.053817 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:31.054704 12705 raft_consensus.cc:2410] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:18:31.055250 15026 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:31.055408 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:31.056042 12705 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:31.051266 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:31.051407 14776 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:31.052014 15024 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:31.058583 12701 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:18:31.059140 12747 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:31.011479 14934 consensus_queue.cc:237] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:31.066092 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:31.013650 14876 consensus_queue.cc:237] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:31.068231 12747 leader_election.cc:400] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6.
I20260709 13:18:31.068817 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:31.039242 14965 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:31.070523 12747 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:31.068317 15025 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:18:31.071919 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 112282us
I20260709 13:18:31.072444 12747 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:31.077230 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:31.079383 12705 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:31.055685 15026 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:31.086212 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:31.090008 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:31.112160 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:31.116925 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.117509 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:31.121203 12780 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
W20260709 13:18:31.072461 15019 log.cc:927] Time spent T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
W20260709 13:18:31.121995 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 49635.8us
I20260709 13:18:31.142153 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:31.143255 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:31.145632 15025 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:31.147588 12747 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:31.148877 15025 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:31.149240 15025 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:31.149595 15025 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:31.151311 14714 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:31.156761 15020 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:31.138083 14680 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:31.073793 14965 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:31.161501 14965 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:31.161688 14961 log.cc:927] Time spent T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 1.140s	user 0.004s	sys 0.000s
I20260709 13:18:31.161826 14714 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:31.163303 15025 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:31.165436 15025 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:31.165725 14991 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:31.069289 15017 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:31.166445 15017 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:31.168180 15014 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:31.168576 15017 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:31.073462 15024 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:31.173430 14948 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:31.112041 15027 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:31.177845 15021 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:31.175556 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:31.179282 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:31.179899 12746 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:31.117709 14508 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:31.207285 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:31.208182 12777 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:31.207955 12597 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:31.209039 12774 raft_consensus.cc:1240] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:18:31.125957 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:31.191434 13379 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:18:31.177356 14712 raft_consensus.cc:3055] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:31.119273 12627 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:31.195211 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:31.215420 12701 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:31.196477 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:31.140192 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 47541.4us
I20260709 13:18:31.120016 15026 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:31.203725 12708 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:31.211969 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:31.213310 12703 raft_consensus.cc:1240] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:31.214015 12597 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:31.217742 12747 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:31.240377 13348 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:31.239619 12775 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:31.242758 13384 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:31.243209 12747 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:31.222029 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:31.244966 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:31.245253 12779 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:31.218730 12780 raft_consensus.cc:2410] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:18:31.220253 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:31.220623 13379 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:18:31.221429 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:31.191643 15027 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:31.250283 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:31.254977 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 339313us
I20260709 13:18:31.273828 13380 raft_consensus.cc:1240] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:31.276589 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:31.278043 12777 raft_consensus.cc:2410] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:18:31.277840 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:31.278852 12780 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:31.280150 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:31.284555 13381 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:18:31.284585 12775 raft_consensus.cc:2393] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 4: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:31.290697 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:31.212378 14774 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:31.213306 14976 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:31.294445 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:31.302716 13380 raft_consensus.cc:1240] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:18:31.239387 14991 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:31.309757 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:31.297745 12629 raft_consensus.cc:1275] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:31.310972 12705 raft_consensus.cc:2393] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 6: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:31.311486 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:31.312203 12629 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:31.214507 14712 raft_consensus.cc:740] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:31.294468 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:31.295068 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:31.299396 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:31.293912 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:31.321642 13379 raft_consensus.cc:1275] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:31.321512 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:31.244963 15021 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:31.287744 14984 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:31.289312 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:31.288149 14571 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:31.285879 14921 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:31.245678 14682 consensus_queue.cc:1059] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.079s
I20260709 13:18:31.295733 13381 pending_rounds.cc:85] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd: Aborting all ops after (but not including) 5
I20260709 13:18:31.332722 13379 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:31.250108 14862 consensus_queue.cc:1059] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.085s
I20260709 13:18:31.215627 15025 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 13:18:31.334493 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:31.333074 12629 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:31.287488 15010 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:31.335361 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:31.380607 13380 raft_consensus.cc:2368] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7.
I20260709 13:18:31.287289 15027 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:31.318002 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:31.284192 14771 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:31.368142 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:31.446023 13384 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 6: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:31.369555 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:31.222398 14680 consensus_queue.cc:1059] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 13:18:31.293208 15028 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:31.304206 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:31.323032 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:31.450897 12780 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:31.310407 14991 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:31.372865 13381 pending_rounds.cc:107] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:18:31.431212 12777 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:31.386443 14984 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:31.387219 12632 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.012s	sys 0.004s
I20260709 13:18:31.371176 14869 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:31.444754 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:31.446635 13385 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:31.445467 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:31.447659 12701 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:18:31.446139 14771 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:31.458847 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:31.362192 14774 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.095s
I20260709 13:18:31.451861 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:31.452966 14980 consensus_meta.cc:287] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.550s	user 0.009s	sys 0.003s
I20260709 13:18:31.455206 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:31.456597 12706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1507) took 2559 ms. Trace:
W20260709 13:18:31.457218 12632 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.019s	sys 0.004s
I20260709 13:18:31.336387 12626 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:31.461468 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:31.470482 12706 rpcz_store.cc:276] 0709 13:18:28.894468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.919437 (+ 24969us) service_pool.cc:224] Handling call
0709 13:18:29.809369 (+889932us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:29.837514 (+ 28145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:31.453817 (+1616303us) spinlock_profiling.cc:243] Waited 889 ms on lock 0x7b5400249ad0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:31.453980 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2045812480}
I20260709 13:18:31.454555 14991 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:31.460846 14771 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:31.493662 14944 raft_consensus.cc:3055] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:31.494212 14944 raft_consensus.cc:740] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:31.495114 14944 consensus_queue.cc:260] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:31.496351 14944 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:31.497998 14948 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:31.498239 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.471041 14869 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:31.500929 14869 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:31.502632 14955 consensus_meta.cc:287] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.548s	user 0.005s	sys 0.005s
I20260709 13:18:31.503036 14955 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:31.505553 14317 consensus_meta.cc:287] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.551s	user 0.007s	sys 0.002s
I20260709 13:18:31.506120 14317 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:31.506481 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:31.506860 12781 raft_consensus.cc:1240] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:31.508754 15026 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:31.472383 14980 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:31.511161 15026 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:31.513273 12777 raft_consensus.cc:2393] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 5: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:31.514792 12704 raft_consensus.cc:1240] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:18:31.457015 14984 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:31.474735 14892 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:31.463541 12746 leader_election.cc:400] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7.
I20260709 13:18:31.468080 13385 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:31.453413 13381 raft_consensus.cc:2889] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:31.474869 14680 consensus_queue.cc:1048] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:18:31.473912 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:31.462337 12701 raft_consensus.cc:2393] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 4: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:31.476014 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:31.452301 15028 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:31.483922 13376 raft_consensus.cc:2393] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 5: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:31.536051 12780 raft_consensus.cc:2393] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 6: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:31.540391 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:31.541020 12780 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:31.541157 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:31.529274 14964 raft_consensus.cc:3055] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:31.545111 12777 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:31.546344 14938 consensus_meta.cc:287] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.000s
I20260709 13:18:31.547475 15028 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:31.542831 14965 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:31.551250 14964 raft_consensus.cc:740] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:31.519356 14879 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:18:31.609819 14964 consensus_queue.cc:260] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:31.611209 15035 log.cc:927] Time spent T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:18:31.562764 14952 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:31.611366 14964 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:31.631038 14965 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:31.553038 14938 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:31.634649 14938 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:31.638899 15027 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:31.647678 14991 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:31.656359 14984 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:31.699352 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 401449us
I20260709 13:18:31.700299 12632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1118) took 1023 ms. Trace:
I20260709 13:18:31.700984 12632 rpcz_store.cc:276] 0709 13:18:30.676354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.676475 (+   121us) service_pool.cc:224] Handling call
0709 13:18:30.693162 (+ 16687us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:30.770626 (+ 77464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:30.770663 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:30.822545 (+ 51882us) log.cc:844] Serialized 26 byte log entry
0709 13:18:31.471622 (+649077us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:31.471659 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:31.512104 (+ 40445us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:31.699335 (+187231us) raft_consensus.cc:1685] finished
0709 13:18:31.699423 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:31.699944 (+   521us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":78856,"raft.run_cpu_time_us":64,"raft.run_wall_time_us":64,"thread_start_us":725724,"threads_started":2,"wal-append.queue_time_us":717043}
I20260709 13:18:31.731392 12702 raft_consensus.cc:2368] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 5. Current term is 6.
I20260709 13:18:31.732409 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:31.734763 12705 raft_consensus.cc:2393] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 7: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:31.735284 12782 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:31.541641 12746 leader_election.cc:403] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:18:31.737628 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:31.737741 12672 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:31.740561 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:31.747882 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 284391us
I20260709 13:18:31.749830 12746 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:18:31.753896 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 510627us
I20260709 13:18:31.761502 12672 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:31.764295 14479 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:31.765102 14479 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:31.764448 14866 consensus_queue.cc:1059] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:31.765604 14479 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:31.766155 14866 consensus_queue.cc:1059] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:31.767464 14866 raft_consensus.cc:3055] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:31.767890 14866 raft_consensus.cc:740] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:31.769385 14866 consensus_queue.cc:260] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:31.770448 14708 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:31.771482 14866 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:31.771327 14708 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:31.779990 15028 raft_consensus.cc:3055] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:31.790875 14991 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:31.792640 14984 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:31.792996 15027 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:31.796096 14965 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:31.780671 15028 raft_consensus.cc:740] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:18:31.806113 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 44945.8us
I20260709 13:18:31.812224 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:31.814147 14944 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:31.815019 14944 raft_consensus.cc:697] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:31.817183 14944 consensus_queue.cc:237] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:31.818290 14708 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:31.822405 12777 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:31.826926 14879 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:31.827756 15028 consensus_queue.cc:260] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:31.830760 15028 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:31.840658 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:31.839394 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:31.844712 14934 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:31.845229 14934 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:31.845638 14934 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:31.847097 12747 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:31.848229 14708 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:18:31.851614 12598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.010s	sys 0.013s
W20260709 13:18:31.852113 12598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.010s	sys 0.013s
I20260709 13:18:31.856730 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:31.858805 14708 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:31.859827 12781 raft_consensus.cc:1240] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:31.859869 14708 raft_consensus.cc:697] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:31.860522 12778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 1843) took 2869 ms. Trace:
I20260709 13:18:31.864153 12778 rpcz_store.cc:276] 0709 13:18:28.990999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.991492 (+   493us) service_pool.cc:224] Handling call
0709 13:18:30.577431 (+1585939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:30.580336 (+  2905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:31.860087 (+1279751us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b540040e550. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:31.860313 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3609654528}
I20260709 13:18:31.856827 14934 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:31.867837 12779 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:31.869549 12783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:33732 (request call id 5) took 1244 ms. Trace:
I20260709 13:18:31.870484 12783 rpcz_store.cc:276] 0709 13:18:30.624736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.624960 (+   224us) service_pool.cc:224] Handling call
0709 13:18:31.869086 (+1244126us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b54000b04d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3ac853 00007f4c7237a60a 00007f4c6b9007d4 00007f4c6b900762 00007f4c6b900691 00007f4c6b90060c 00007f4c6b8ff8c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:31.869288 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":882895104}
I20260709 13:18:31.869063 14708 consensus_queue.cc:237] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:31.879200 12779 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 5. Current term is 6. Ops: []
W20260709 13:18:31.880076 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 46726.2us
I20260709 13:18:31.885892 15027 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:18:31.889423 12747 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:31.891793 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:31.897166 12747 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:31.899544 14934 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:31.897503 12783 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:31.897751 15027 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:31.925915 15027 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:31.926568 15027 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:31.934381 14771 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:31.936206 12672 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:31.937461 14973 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:31.941651 12672 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:31.900153 14479 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:31.906641 14892 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:18:31.916860 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 215520us
I20260709 13:18:31.918146 14820 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:31.918638 14712 consensus_queue.cc:260] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:31.923368 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:31.923712 12598 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:31.924295 12630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1111) took 1250 ms. Trace:
W20260709 13:18:31.949797 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 52874.6us
I20260709 13:18:31.950428 12782 raft_consensus.cc:1275] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:31.951987 15038 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:31.962133 14892 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:31.971412 14973 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:31.971582 14479 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:31.971966 14973 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:31.971876 15038 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:31.972617 14973 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:31.973877 15008 raft_consensus.cc:2764] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:18:31.975855 12597 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:31.976770 14820 consensus_queue.cc:1059] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:18:31.967737 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 44190.7us
I20260709 13:18:31.982796 12597 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:31.954985 12630 rpcz_store.cc:276] 0709 13:18:30.673715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.673861 (+   146us) service_pool.cc:224] Handling call
0709 13:18:30.674192 (+   331us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:30.849843 (+175651us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:30.849888 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:30.850608 (+   720us) log.cc:844] Serialized 52 byte log entry
0709 13:18:31.287263 (+436655us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:31.332628 (+ 45365us) log.cc:844] Serialized 14 byte log entry
0709 13:18:31.875854 (+543226us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b1800318e20. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c53f3 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:18:31.916549 (+ 40695us) spinlock_profiling.cc:243] Waited 23.4 ms on lock 0x7b1800318e20. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a23fc0a 00007f4c7397f219 00007f4c6a2c7a40 00007f4c6a2c24e4 00007f4c6a2c2455 00007f4c6d3aa060 00007f4c6d3bc716 00007f4c6d3cd287 00007f4c6d3cd208
0709 13:18:31.916904 (+   355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:31.922917 (+  6013us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54002cea54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:31.922971 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:31.923039 (+    68us) raft_consensus.cc:1685] finished
0709 13:18:31.923051 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:31.924041 (+   990us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7486,"prepare.queue_time_us":43166,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":69,"raft.queue_time_us":587582,"raft.run_cpu_time_us":486,"raft.run_wall_time_us":10329,"spinlock_wait_cycles":431770368,"thread_start_us":406483,"threads_started":1,"wal-append.queue_time_us":448981}
I20260709 13:18:31.990653 15027 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:31.993973 15027 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:31.997486 14820 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:31.998054 14820 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:31.998479 14820 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:32.001861 15026 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.002758 12700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1173) took 1056 ms. Trace:
I20260709 13:18:32.002518 15026 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:32.003191 12700 rpcz_store.cc:276] 0709 13:18:30.946713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.946841 (+   128us) service_pool.cc:224] Handling call
0709 13:18:30.947120 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:31.037485 (+ 90365us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:31.037505 (+    20us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:31.037533 (+    28us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:31.037554 (+    21us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:32.002250 (+964696us) spinlock_profiling.cc:243] Waited 87.9 ms on lock 0x7b540021edd4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:32.002322 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:32.002564 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":202070784}
I20260709 13:18:32.001170 15038 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.005491 15038 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:32.009955 15007 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:32.017802 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:32.021018 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:32.021967 14971 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:32.022758 14971 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:32.023840 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:32.027357 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:32.028612 15040 consensus_queue.cc:1048] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:32.029258 15051 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:32.031730 12700 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:32.023451 14973 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:31.987339 15008 raft_consensus.cc:3055] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:32.013919 12628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56838 (request call id 1129) took 1332 ms. Trace:
I20260709 13:18:32.035076 14955 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:32.037935 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 96621.7us
I20260709 13:18:32.037053 12628 rpcz_store.cc:276] 0709 13:18:30.678655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.699730 (+ 21075us) service_pool.cc:224] Handling call
0709 13:18:32.010470 (+1310740us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54000e1ad4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:32.010693 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":277125888}
I20260709 13:18:32.036849 15008 raft_consensus.cc:740] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:18:32.050781 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.050674 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:32.051697 12703 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:32.054991 15008 consensus_queue.cc:260] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:32.056377 15008 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:32.052718 15016 log.cc:927] Time spent T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 1.035s	user 0.000s	sys 0.029s
I20260709 13:18:32.078069 14906 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.079015 14906 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:32.082877 14906 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:32.057507 14771 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:32.103745 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 126237us
I20260709 13:18:32.109830 14944 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.110755 14879 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7.
I20260709 13:18:32.110651 14944 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:32.109757 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:32.111228 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:32.115300 12704 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 7: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:32.116765 12703 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:32.119611 12631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1121) took 1439 ms. Trace:
I20260709 13:18:32.120237 12631 rpcz_store.cc:276] 0709 13:18:30.677024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.723392 (+ 46368us) service_pool.cc:224] Handling call
0709 13:18:30.723817 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:30.827542 (+103725us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:30.827578 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:30.850922 (+ 23344us) log.cc:844] Serialized 26 byte log entry
0709 13:18:30.912391 (+ 61469us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:30.912420 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:30.912472 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:32.052211 (+1139739us) spinlock_profiling.cc:243] Waited 6.76 ms on lock 0x7b2400114b78. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a14b9c4 00007f4c6a149acb 00007f4c6a304dc9 00007f4c6a304a55 00007f4c6a30409d 00007f4c6cf3610e 00007f4c6cf4d88c 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221
0709 13:18:32.111898 (+ 59687us) raft_consensus.cc:1685] finished
0709 13:18:32.111926 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:32.116598 (+  4672us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":59398,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":100,"spinlock_wait_cycles":15547776,"thread_start_us":134593,"threads_started":2,"wal-append.queue_time_us":100132}
I20260709 13:18:32.123081 13381 raft_consensus.cc:2393] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 7: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:32.123559 12776 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:32.125257 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:32.126113 12628 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:18:32.126863 15038 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:32.130632 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:32.130869 15026 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:32.131106 15013 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:18:32.133831 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.121803 14971 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:32.142597 12597 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:32.144892 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:32.149667 13379 raft_consensus.cc:3055] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:32.150161 13379 raft_consensus.cc:740] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:32.150230 12632 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:32.151423 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:32.151335 13379 consensus_queue.cc:260] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:32.152236 12703 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:32.153026 13379 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:32.154291 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:32.155495 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:32.155825 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:32.164048 15009 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:32.167591 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:32.169057 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:32.169771 12626 raft_consensus.cc:1240] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:32.170572 14820 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:32.173110 14820 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:32.150377 12630 raft_consensus.cc:3055] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:32.182960 12630 raft_consensus.cc:740] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:32.183295 14944 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:32.184193 12630 consensus_queue.cc:260] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:32.185392 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:32.113248 14892 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:32.185406 12630 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:32.194864 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:32.196568 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.199149 12628 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:32.204202 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:32.205353 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 86440.4us
I20260709 13:18:32.206296 12597 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:32.207173 12632 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:18:32.207623 14980 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:32.208592 12597 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:32.209671 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:32.210166 14891 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:32.211064 15009 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:32.211141 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:32.215266 12776 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:32.217929 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 52430.5us
I20260709 13:18:32.222504 14892 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:32.215454 15051 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:18:32.228797 15054 log.cc:927] Time spent T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 13:18:32.230090 13349 connection.cc:256] RPC call timeout handler was delayed by -1.1802966019999985s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23530000000000004s for a response
I20260709 13:18:32.232451 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:32.233368 13378 raft_consensus.cc:2368] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7.
I20260709 13:18:32.234064 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:32.234655 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:32.235008 13381 raft_consensus.cc:2393] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 7: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:32.236191 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:32.236433 12781 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:18:32.236374 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:32.236937 12781 pending_rounds.cc:85] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 6
I20260709 13:18:32.237308 12781 pending_rounds.cc:107] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:18:32.237491 12782 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 6: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:32.237617 12781 raft_consensus.cc:2889] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:32.237741 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:32.238271 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:32.239030 12783 raft_consensus.cc:2393] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:32.239061 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:32.239427 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:32.239826 12779 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:32.218889 15008 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:32.241271 15008 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:32.241698 15008 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:32.241961 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:32.242337 15025 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.243652 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:32.244560 12780 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:32.245783 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:32.244969 15025 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:32.248605 12775 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:32.245482 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:32.254612 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:32.260504 15053 log.cc:927] Time spent T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
I20260709 13:18:32.211719 15009 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:32.262158 15009 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:32.263614 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:32.265842 12597 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:32.266757 15051 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:32.267429 15051 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:32.267864 15051 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:32.269531 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:32.269860 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:32.270438 13380 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:32.271492 12633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 1940) took 2369 ms. Trace:
I20260709 13:18:32.271744 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:32.272486 13380 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:32.271229 15025 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:32.274960 14712 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:32.277309 12630 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:32.285071 14969 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:32.287043 12775 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:18:32.287348 15009 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:32.288394 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:32.271862 12633 rpcz_store.cc:276] 0709 13:18:29.901729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.901901 (+   172us) service_pool.cc:224] Handling call
0709 13:18:30.119822 (+217921us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:30.127451 (+  7629us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:30.127496 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:30.163312 (+ 35816us) log.cc:844] Serialized 26 byte log entry
0709 13:18:30.322439 (+159127us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:30.322484 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:30.322558 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:30.629268 (+306710us) raft_consensus.cc:1685] finished
0709 13:18:30.629290 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:32.271066 (+1641776us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b54000deb50. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:32.271290 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1737,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":36164,"spinlock_wait_cycles":452464384,"thread_start_us":62268,"threads_started":1,"wal-append.queue_time_us":273280,"wal-append.run_cpu_time_us":2529,"wal-append.run_wall_time_us":1147017}
I20260709 13:18:32.290081 15008 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:32.290571 12780 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:32.291016 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:32.292590 15052 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.265s	user 0.002s	sys 0.000s
I20260709 13:18:32.294294 12703 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:32.239064 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:32.307243 12779 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
I20260709 13:18:32.307648 12778 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:32.272226 12597 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:32.308332 13348 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:32.309300 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:32.310431 15051 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:32.295740 15009 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:32.313657 12774 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1630) took 1090 ms. Trace:
I20260709 13:18:32.314421 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:32.318938 12776 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:32.322130 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:32.321239 12774 rpcz_store.cc:276] 0709 13:18:31.222775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:31.934839 (+712064us) service_pool.cc:224] Handling call
0709 13:18:31.935615 (+   776us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:31.969099 (+ 33484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:31.969131 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:31.970027 (+   896us) log.cc:844] Serialized 52 byte log entry
0709 13:18:31.984686 (+ 14659us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:31.984750 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:31.984832 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:32.259815 (+274983us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:18:32.296632 (+ 36817us) raft_consensus.cc:1685] finished
0709 13:18:32.296652 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:32.313448 (+ 16796us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45966,"raft.queue_time_us":34877,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":119,"spinlock_wait_cycles":24668416,"thread_start_us":14179,"threads_started":1,"wal-append.queue_time_us":54509}
I20260709 13:18:32.325927 12707 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:32.326287 12709 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:32.327054 12703 raft_consensus.cc:1240] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:32.328109 13383 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:32.327600 12528 catalog_manager.cc:5697] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 2 to 4, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:32.329744 15008 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:32.329824 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:32.333159 14613 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:32.333514 12630 raft_consensus.cc:1240] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:32.329406 12528 catalog_manager.cc:5697] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:32.333617 14613 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:32.342233 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.343441 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:32.348694 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:32.323872 14973 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:32.345549 12781 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:32.336637 14588 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.352425 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:32.354174 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:32.354949 12704 raft_consensus.cc:2393] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 7: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
W20260709 13:18:32.356317 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 84212.6us
I20260709 13:18:32.341382 14974 consensus_queue.cc:1059] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:18:32.358072 14974 raft_consensus.cc:3055] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:32.358532 14974 raft_consensus.cc:740] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:32.359706 14974 consensus_queue.cc:260] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:32.339005 14613 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:32.363212 14588 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:32.338622 15008 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:32.357379 12703 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:18:32.365711 12528 catalog_manager.cc:5697] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 3 to 5, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 5 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:32.367847 12528 catalog_manager.cc:5697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:32.368767 12707 raft_consensus.cc:1240] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:32.368839 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:32.370050 14978 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.370301 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:32.370581 14978 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:32.371228 12702 raft_consensus.cc:2410] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:18:32.362564 14969 consensus_queue.cc:1059] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:32.362893 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:32.363052 13349 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:32.376452 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:32.379503 12707 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:32.380092 15051 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:32.343441 13378 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:32.380725 14978 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:32.381815 13377 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:18:32.382208 13377 pending_rounds.cc:85] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: Aborting all ops after (but not including) 7
I20260709 13:18:32.382472 13377 pending_rounds.cc:107] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:18:32.382714 13377 raft_consensus.cc:2889] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:32.385118 13377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 455) took 1202 ms. Trace:
W20260709 13:18:32.385890 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 66498.5us
I20260709 13:18:32.387192 12780 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:32.387681 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:32.388005 12597 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:32.388926 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:32.391041 14588 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:32.392010 12704 raft_consensus.cc:2393] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:32.392293 12703 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:18:32.397249 12747 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:32.397296 12631 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:32.399845 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:32.378136 15025 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:32.401510 12528 catalog_manager.cc:5697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 1 to 5, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 5 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:32.401731 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:32.405427 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:32.404855 14680 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:32.407414 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:32.407510 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:32.404306 15051 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:32.413128 14891 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:18:32.414566 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.414853 14771 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:32.415359 14771 raft_consensus.cc:697] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:32.415741 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:32.416247 14771 consensus_queue.cc:237] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:32.416783 15025 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:32.408584 14680 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:32.417969 14680 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:32.418521 12629 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:32.418816 12777 raft_consensus.cc:2410] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:18:32.420751 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:32.422300 12597 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:32.421667 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 66123.6us
I20260709 13:18:32.426260 12747 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:32.428201 12672 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:32.428385 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:32.429071 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:32.430815 15038 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:32.431329 15038 raft_consensus.cc:697] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:32.432278 15038 consensus_queue.cc:237] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:32.432906 14934 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:32.435345 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:32.436364 13383 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:32.414709 14891 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:32.438285 12702 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:32.385507 13377 rpcz_store.cc:276] 0709 13:18:31.183071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:31.185715 (+  2644us) service_pool.cc:224] Handling call
0709 13:18:31.194530 (+  8815us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:32.383174 (+1188644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:32.384814 (+  1640us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54003ab954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:32.385045 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2683237632}
I20260709 13:18:32.442695 13379 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:18:32.443053 13379 pending_rounds.cc:85] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: Aborting all ops after (but not including) 5
I20260709 13:18:32.443274 13379 pending_rounds.cc:107] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:18:32.443513 13379 raft_consensus.cc:2889] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:32.444069 12633 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:32.447434 12631 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:32.447935 13379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 375) took 1004 ms. Trace:
I20260709 13:18:32.448581 12597 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:32.439338 14891 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:32.448360 13379 rpcz_store.cc:276] 0709 13:18:31.443979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:31.444118 (+   139us) service_pool.cc:224] Handling call
0709 13:18:31.444360 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:32.149364 (+705004us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b5400059ea0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d329dc0 00007f4c6d3b41c0 00007f4c6d3b5231 00007f4c6d3b1d8f 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c
0709 13:18:32.447669 (+298305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:32.447885 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":774775552}
I20260709 13:18:32.466787 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:32.467046 12629 raft_consensus.cc:2410] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:18:32.468418 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:18:32.474002 14921 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:18:32.474586 15009 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:32.475045 15009 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:32.475431 15009 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:32.479233 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:32.480167 14978 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:18:32.481717 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.482829 14974 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:32.483671 14588 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.484386 14983 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.484455 12628 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:32.484989 14983 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:32.488121 14939 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:32.488829 14942 consensus_queue.cc:1059] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:32.489080 15014 consensus_queue.cc:1059] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:32.484143 14588 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:32.456557 12707 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
W20260709 13:18:32.491636 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 46288.4us
I20260709 13:18:32.487433 12673 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:32.453792 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:32.496006 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:32.444293 14708 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:32.500550 12597 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:32.453648 15008 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:32.500875 12632 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:32.507170 14892 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:32.507776 13377 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:18:32.506290 15027 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:18:32.509986 14317 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:32.497399 14991 consensus_queue.cc:1059] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:18:32.490419 14980 consensus_queue.cc:1059] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:18:32.511497 15027 raft_consensus.cc:3055] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:32.512391 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:32.453903 14973 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:32.514194 12597 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:32.499323 14708 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:32.523348 14680 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:32.511904 15027 raft_consensus.cc:740] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:32.534019 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:32.534509 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 47148.4us
I20260709 13:18:32.537357 14772 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:32.538702 12632 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:18:32.539726 14980 raft_consensus.cc:3055] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:32.540160 14980 raft_consensus.cc:740] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:32.541911 15039 consensus_queue.cc:1059] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:32.520429 14991 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:18:32.542994 15013 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:32.546150 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:32.547020 12631 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:32.546214 14680 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:32.550251 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.535936 12627 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
W20260709 13:18:32.554908 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 58968.1us
I20260709 13:18:32.522886 14973 raft_consensus.cc:697] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:32.555979 12672 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:32.556771 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:32.564143 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.534762 14708 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:32.566314 14939 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:32.575081 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:32.530467 14317 consensus_queue.cc:1048] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:18:32.578610 14939 raft_consensus.cc:697] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:32.541145 14980 consensus_queue.cc:260] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:32.597872 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:32.598532 12780 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:32.598610 14969 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:32.603457 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:32.605878 12780 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:32.606189 14939 consensus_queue.cc:237] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:32.587343 14942 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:32.612227 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:32.613343 12781 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
W20260709 13:18:32.614282 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.614652 14847 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:18:32.623445 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.454171 14879 raft_consensus.cc:2955] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:32.584470 14613 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:32.631256 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:32.629902 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 59138.8us
I20260709 13:18:32.624382 15025 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:32.617758 12747 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:32.665139 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:32.629515 15051 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:32.626573 14588 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:32.628583 14613 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:32.675222 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 93532.7us
I20260709 13:18:32.733587 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:32.678553 12776 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:18:32.664075 14983 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:32.664899 15039 raft_consensus.cc:3055] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:32.739913 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:32.667819 15025 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:32.625931 14891 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:32.756426 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:32.672693 14942 raft_consensus.cc:697] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:18:32.760923 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 95912us
I20260709 13:18:32.764801 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:32.736059 14847 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:18:32.766392 15067 log.cc:927] Time spent T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:18:32.753134 12700 raft_consensus.cc:2376] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6. Re-sending same reply.
I20260709 13:18:32.752753 15039 raft_consensus.cc:740] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:32.677502 14980 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:32.763650 15072 consensus_queue.cc:1059] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:18:32.794065 14971 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:18:32.794821 15074 log.cc:927] Time spent T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:18:32.630007 14973 consensus_queue.cc:237] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:32.804540 14708 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:32.807572 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:32.809134 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 255853us
I20260709 13:18:32.813076 15025 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:32.812592 12528 catalog_manager.cc:5697] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 3 to 4, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 4 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:32.811833 12705 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
W20260709 13:18:32.826762 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 242137us
I20260709 13:18:32.806275 14983 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:32.809279 14479 raft_consensus.cc:3055] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:32.876304 12597 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:32.827208 14971 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:32.887544 14971 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:32.887593 14776 raft_consensus.cc:3055] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:32.888274 14776 raft_consensus.cc:740] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:32.888779 13349 leader_election.cc:400] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 5. Current term is 6.
I20260709 13:18:32.889130 13349 leader_election.cc:403] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:18:32.889169 14776 consensus_queue.cc:260] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:32.890888 14969 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 5. Current term is 6.
I20260709 13:18:32.891047 13349 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:32.892006 14969 consensus_queue.cc:1059] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:18:32.893720 15048 log.cc:927] Time spent T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:18:32.894718 14942 consensus_queue.cc:237] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:32.897136 14978 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:32.897827 14613 raft_consensus.cc:3055] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:32.898238 14613 raft_consensus.cc:740] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:32.899546 15038 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.672276 15027 consensus_queue.cc:260] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:32.900112 15038 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:32.900734 15027 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:32.900542 14776 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:32.811295 14317 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.094s
I20260709 13:18:32.906999 14983 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:32.907642 14891 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:32.908824 12705 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:18:32.909193 14983 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:32.902333 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.835191 14847 raft_consensus.cc:697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:32.881244 14479 raft_consensus.cc:740] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:32.930680 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:32.911600 14708 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:32.913017 15025 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:32.873941 15070 log.cc:927] Time spent T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 13:18:32.913800 15009 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:32.918092 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:32.917984 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:32.918265 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:32.919174 12705 raft_consensus.cc:1240] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:18:32.919082 14892 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:32.919430 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:32.899152 14613 consensus_queue.cc:260] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:32.902288 15038 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:32.893190 13349 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:32.903687 14969 consensus_queue.cc:1059] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
W20260709 13:18:32.927438 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 190813us
I20260709 13:18:32.928356 12528 catalog_manager.cc:5697] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:32.930600 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:32.951933 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 75786.8us
I20260709 13:18:32.933699 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:32.953262 12777 raft_consensus.cc:2393] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 6: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:32.934602 14479 consensus_queue.cc:260] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:32.935778 15039 consensus_queue.cc:260] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:32.937007 14847 consensus_queue.cc:237] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:32.941154 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 52106.9us
I20260709 13:18:32.945873 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:32.960001 12709 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:32.947117 14712 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:32.948056 12747 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:32.933111 12780 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: []
W20260709 13:18:32.956434 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 63301.7us
I20260709 13:18:32.940829 15072 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:32.958173 12776 raft_consensus.cc:2410] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:18:32.964079 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.965629 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:32.967216 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.966284 13349 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:32.955484 15068 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.979115 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:32.962734 14708 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:32.965065 14921 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:32.984506 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:33.003305 12747 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:32.974324 14479 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:33.005921 12747 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:32.969384 15025 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.024s
W20260709 13:18:33.023869 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 57653us
W20260709 13:18:32.971647 14892 consensus_queue.cc:1175] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3d539f8d95d343d8b1696803f18b30d2" responder_term: 6 status { last_received { term: 5 index: 3 } last_committed_idx: 3 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:18:33.026280 15076 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.050487 15076 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:33.053483 12783 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:18:33.054078 12782 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:33.054664 14820 raft_consensus.cc:3055] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:33.055034 14820 raft_consensus.cc:740] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:33.055828 14820 consensus_queue.cc:260] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:33.056655 12775 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:18:33.057003 12775 pending_rounds.cc:85] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 5
I20260709 13:18:33.056941 14820 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:33.057451 12775 pending_rounds.cc:107] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:18:33.057747 12775 raft_consensus.cc:2889] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:33.052734 15076 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:33.043812 15025 consensus_queue.cc:1243] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:18:33.046028 12774 raft_consensus.cc:1240] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:33.078260 12778 raft_consensus.cc:1240] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:18:33.046947 12777 raft_consensus.cc:1240] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:33.047402 14978 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:33.083479 14978 raft_consensus.cc:697] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:33.083431 14971 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:33.047829 15009 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:33.085692 15079 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.099s
I20260709 13:18:33.086036 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:33.086627 15079 raft_consensus.cc:3055] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:33.087010 15079 raft_consensus.cc:740] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:33.087862 15079 consensus_queue.cc:260] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:33.089041 15079 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:33.094305 12709 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:33.007850 14613 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:33.084776 14978 consensus_queue.cc:237] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:32.983407 15077 consensus_queue.cc:1059] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 5.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.067s
I20260709 13:18:33.030084 14745 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:33.066179 13349 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:32.997879 14944 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:18:33.067685 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 61871.5us
I20260709 13:18:33.114075 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:32.997340 14712 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:33.068416 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:33.044275 13377 raft_consensus.cc:2955] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:18:33.080327 12597 connection.cc:256] RPC call timeout handler was delayed by -1.9535885370000869s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1539s for a response
W20260709 13:18:33.116528 12597 connection.cc:256] RPC call timeout handler was delayed by -1.7267885369999476s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1791s for a response
W20260709 13:18:33.117008 12597 connection.cc:256] RPC call timeout handler was delayed by -1.6277885370000149s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19010000000000002s for a response
W20260709 13:18:33.117348 12597 connection.cc:256] RPC call timeout handler was delayed by -1.596288536999964s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1936s for a response
I20260709 13:18:33.081105 14934 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:33.081732 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:18:33.082690 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 73717.5us
I20260709 13:18:33.010107 14771 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:32.988550 15039 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:33.122888 12673 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:33.009696 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:33.118165 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:33.119009 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 84317.6us
I20260709 13:18:33.134233 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:33.121707 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 180841us
I20260709 13:18:33.122495 12702 raft_consensus.cc:1240] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:33.136191 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:33.126857 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:33.135627 14969 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:33.120538 14934 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:33.146278 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:33.144498 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:33.145146 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:33.122521 14771 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:33.120630 14971 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:33.127568 14921 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:32.997609 15068 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:33.169891 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:33.170338 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:33.170615 12775 raft_consensus.cc:2376] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5. Re-sending same reply.
I20260709 13:18:33.170948 12708 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:33.171025 12634 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:33.171864 15077 consensus_queue.cc:1059] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:33.174098 12703 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:18:33.175078 15073 raft_consensus.cc:3055] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:33.175246 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:33.175472 15073 raft_consensus.cc:740] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:33.176234 15073 consensus_queue.cc:260] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:33.178123 12626 raft_consensus.cc:1240] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: []
W20260709 13:18:33.180061 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 44428.5us
I20260709 13:18:33.180827 12672 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:33.181061 12627 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:33.167044 14934 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:33.182250 14983 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:33.182355 12672 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:33.182871 14983 raft_consensus.cc:697] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:33.183058 14941 raft_consensus.cc:3055] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:33.183547 14941 raft_consensus.cc:740] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:18:33.185926 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:33.186609 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.189242 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:33.191483 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:33.195082 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 57783.7us
I20260709 13:18:33.176779 12703 raft_consensus.cc:2955] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:33.199426 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:33.167728 14771 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:33.231283 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:33.255442 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 89841.5us
W20260709 13:18:33.256090 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.255132 14921 raft_consensus.cc:697] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 7 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:18:33.257704 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 61523.5us
I20260709 13:18:33.257261 12673 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:33.164734 14588 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:33.202329 13378 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 5: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
W20260709 13:18:33.262197 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 52734us
I20260709 13:18:33.183944 15073 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:33.139964 14974 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:18:33.276981 14891 raft_consensus.cc:3055] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:33.277470 14891 raft_consensus.cc:740] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:33.183787 14983 consensus_queue.cc:237] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:33.259591 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:33.264415 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 68276.9us
I20260709 13:18:33.266796 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:33.267628 13348 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:33.208626 15008 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:18:33.167047 15009 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.093s
I20260709 13:18:33.285876 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:33.168113 12630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:18:33.287623 12630 raft_consensus.cc:606] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Received request to transfer leadership to TS 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:33.288044 12630 raft_consensus.cc:614] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:33.274026 14973 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:33.284452 15068 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:33.243203 14942 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:18:33.316753 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 57214.9us
I20260709 13:18:33.279505 15025 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:33.321512 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 137795us
I20260709 13:18:33.322858 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:33.314571 12528 catalog_manager.cc:5697] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 1 to 2, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 2 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:33.324177 12630 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:33.325534 14682 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.326306 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:33.327054 12705 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:18:33.260579 15076 consensus_queue.cc:1059] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:18:33.300935 14973 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:33.265748 14969 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
W20260709 13:18:33.337695 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 46529.7us
I20260709 13:18:33.321266 14974 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:33.345698 13349 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:33.334653 14891 consensus_queue.cc:260] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:33.334478 15025 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:33.315383 15009 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
W20260709 13:18:33.350564 15093 log.cc:927] Time spent T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:18:33.351312 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:33.353993 15055 log.cc:927] Time spent T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:18:33.355960 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 65204.5us
I20260709 13:18:33.326329 14991 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.359258 15095 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:33.322232 14921 consensus_queue.cc:237] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:33.367973 12672 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:33.368414 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:33.367321 12597 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:33.326233 14971 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:33.380584 13349 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:33.339283 14973 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:33.388230 12631 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:33.393605 12598 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:33.326073 14682 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:33.395480 14680 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.395864 14991 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:33.375790 14613 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:18:33.568786 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.601853 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:33.602551 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.450963 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:33.532872 12625 raft_consensus.cc:3055] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:33.587291 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:33.604003 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:33.536952 15099 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.606047 14317 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:33.606593 15099 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:33.606663 14317 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:33.607228 14991 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:33.608811 14317 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:33.609012 15099 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:33.606269 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:33.609797 14934 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:33.610857 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:33.612468 12631 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:33.614596 12779 raft_consensus.cc:1240] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 5. Ops: []
I20260709 13:18:33.501241 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:33.395891 14980 consensus_meta.cc:287] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.560s	user 0.011s	sys 0.001s
W20260709 13:18:33.572484 12672 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.68:36021 timed out after 1.704s (SENT) [suppressed 12 similar messages]
W20260709 13:18:33.616279 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 43845.4us
I20260709 13:18:33.617426 12775 raft_consensus.cc:1240] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:33.618054 14934 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:33.623014 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:33.626863 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 254955us
I20260709 13:18:33.628558 12746 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:33.524127 12627 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:18:33.534103 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:33.630753 12632 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:33.584641 15095 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.568989 15102 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:33.572103 15008 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:33.526368 14939 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:33.523499 12708 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:33.578027 12705 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
W20260709 13:18:33.536238 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 169001us
I20260709 13:18:33.377645 15027 consensus_queue.cc:1059] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:18:33.367281 15077 consensus_queue.cc:1059] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:18:33.568426 14971 raft_consensus.cc:697] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:33.602512 12673 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:33.537901 15025 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:33.592247 14891 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:33.603574 12780 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:33.582541 14771 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:33.601799 14978 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:33.583334 14588 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:33.604360 12625 raft_consensus.cc:740] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:33.632763 14613 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:33.633042 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:33.700953 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:33.634657 14682 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:33.603407 14680 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:33.636415 14941 consensus_queue.cc:260] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:33.603752 14847 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.637622 14317 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:33.637873 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:33.636451 13379 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 13:18:33.656497 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:33.657948 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:33.698546 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 96111.6us
W20260709 13:18:33.699272 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 310158us
I20260709 13:18:33.701596 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:33.702306 12597 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:33.705982 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:33.701490 13385 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:33.740301 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:33.646422 14934 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:33.646580 15008 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:33.787878 12709 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:33.787432 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:33.788048 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 43514.4us
I20260709 13:18:33.788427 13379 pending_rounds.cc:85] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd: Aborting all ops after (but not including) 5
I20260709 13:18:33.744841 12783 raft_consensus.cc:1240] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:18:33.789000 12701 raft_consensus.cc:2393] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 6: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
W20260709 13:18:33.697865 14956 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.001s	sys 0.007s
W20260709 13:18:33.794023 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.797921 13379 pending_rounds.cc:107] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:18:33.799782 13379 raft_consensus.cc:2889] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:33.800858 12709 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:33.797464 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:33.802342 12747 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:33.700886 15027 raft_consensus.cc:3055] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:33.808274 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:33.808969 12627 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:33.714613 15009 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:33.813118 14708 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:18:33.814069 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 111844us
I20260709 13:18:33.814539 14680 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:33.739315 15073 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:33.816267 12672 leader_election.cc:400] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7.
I20260709 13:18:33.821502 12672 leader_election.cc:403] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:18:33.816578 13349 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:33.816918 12705 raft_consensus.cc:1275] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 13:18:33.806286 14956 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.004s	sys 0.011s
I20260709 13:18:33.811131 12625 consensus_queue.cc:260] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:33.818773 12702 raft_consensus.cc:2393] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 4: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:33.820012 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:33.818184 14892 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.810436 14934 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:33.823769 14708 raft_consensus.cc:697] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:33.802052 15008 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:33.826180 12701 raft_consensus.cc:1240] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:33.787433 14682 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:33.646382 15095 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:33.699939 15102 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:33.828902 12629 raft_consensus.cc:3055] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:33.822731 13349 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:33.847347 12629 raft_consensus.cc:740] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:33.832765 12625 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:33.812438 14941 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:33.809862 13376 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:33.828697 12597 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:33.835362 15039 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:18:33.831420 14892 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:33.861893 14317 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:33.862751 14892 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:33.846318 14682 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:33.865890 14682 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:33.866542 14682 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.867121 14682 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:33.869175 15081 log.cc:927] Time spent T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.873s	user 0.011s	sys 0.000s
I20260709 13:18:33.819142 14944 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:33.852358 15100 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:33.867579 14317 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.869755 14944 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:33.869994 14317 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:33.870371 14944 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:33.870404 15100 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:33.872074 14317 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:33.852451 14974 consensus_queue.cc:1059] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:18:33.875017 14974 raft_consensus.cc:3055] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:33.875641 14974 raft_consensus.cc:740] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:18:33.878196 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 49576.7us
I20260709 13:18:33.879594 12597 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:33.744421 14939 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:33.830691 14969 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:33.831472 15069 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:33.891259 12630 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:33.820679 15073 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:33.794785 14847 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:33.828413 15108 log.cc:927] Time spent T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 13:18:33.809511 15027 raft_consensus.cc:740] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:18:33.920845 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.920285 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:33.921538 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.921501 13385 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:33.880815 15077 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7.
I20260709 13:18:33.876772 14974 consensus_queue.cc:260] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:33.893797 14820 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:33.894003 14680 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.869438 14682 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:33.934881 12780 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:33.921077 15073 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:33.922988 14892 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:33.832376 14971 consensus_queue.cc:237] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:33.929827 12629 consensus_queue.cc:260] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:33.854393 15009 consensus_queue.cc:1059] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.082s
W20260709 13:18:33.939788 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 91081.5us
I20260709 13:18:33.922740 15039 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
W20260709 13:18:33.944859 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 87825.9us
I20260709 13:18:33.943293 14708 consensus_queue.cc:237] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:33.946892 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 136861us
I20260709 13:18:33.947458 12629 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:33.947857 15111 log.cc:927] Time spent T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:18:33.943279 14613 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
W20260709 13:18:33.948338 15075 log.cc:927] Time spent T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.928s	user 0.000s	sys 0.013s
I20260709 13:18:33.952334 14934 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:33.953984 14974 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:33.955302 14771 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:33.955698 15027 consensus_queue.cc:260] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:33.956107 12632 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:33.956569 15095 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:33.968642 14980 consensus_queue.cc:1059] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:33.954015 14847 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:33.940752 14680 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:33.980315 14680 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:33.981400 12780 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:18:33.983914 14978 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:33.984287 13383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 390) took 1021 ms. Trace:
I20260709 13:18:33.984728 13383 rpcz_store.cc:276] 0709 13:18:32.962799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:32.969238 (+  6439us) service_pool.cc:224] Handling call
0709 13:18:32.971072 (+  1834us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:32.977287 (+  6215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:32.977307 (+    20us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:33.011554 (+ 34247us) log.cc:844] Serialized 26 byte log entry
0709 13:18:33.123529 (+111975us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:33.123569 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:33.123641 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:33.981241 (+857600us) spinlock_profiling.cc:243] Waited 26.7 ms on lock 0x7b2400317948. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a14b9c4 00007f4c6a149acb 00007f4c6a304dc9 00007f4c6a304a55 00007f4c6a303fb0 00007f4c6cf7a488 00007f4c6cf7abe8 00007f4c6cf52398 00007f4c6cf4d76a 00007f4c6cf4cbb6
0709 13:18:33.983988 (+  2747us) raft_consensus.cc:1685] finished
0709 13:18:33.984009 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:33.984226 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7399,"raft.run_cpu_time_us":352,"raft.run_wall_time_us":5184,"spinlock_wait_cycles":61473152,"thread_start_us":73498,"threads_started":1,"wal-append.queue_time_us":338397}
I20260709 13:18:33.958164 14939 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:33.958482 14973 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:33.959064 15078 log.cc:927] Time spent T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.929s	user 0.005s	sys 0.005s
I20260709 13:18:33.960462 14944 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:33.942420 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 105722us
I20260709 13:18:34.016069 12629 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:34.016024 12598 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:33.964766 15072 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.931s	user 0.013s	sys 0.011s
W20260709 13:18:34.021101 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 57495.6us
I20260709 13:18:34.023525 12746 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:34.027886 12634 raft_consensus.cc:1275] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:18:34.028213 12634 pending_rounds.cc:85] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 2
I20260709 13:18:34.028466 12634 pending_rounds.cc:107] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:18:34.028680 12634 raft_consensus.cc:2889] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:34.022112 15013 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:34.022550 15027 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:34.024498 15072 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.990s	user 0.013s	sys 0.011s
I20260709 13:18:34.022845 15102 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:34.038344 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.029366 14939 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:34.035316 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 155835us
I20260709 13:18:34.018898 15008 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:34.013217 15117 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:18:34.098057 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:34.144176 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:34.115152 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:34.141541 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:34.093194 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 77422.6us
I20260709 13:18:34.101497 12625 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:18:34.101310 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:34.089872 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:34.041440 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:34.097124 15039 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:18:34.165238 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.186381 12630 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:34.187554 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:34.165880 12673 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:34.174074 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.100137 12634 pending_rounds.cc:85] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 1
I20260709 13:18:34.188701 12634 pending_rounds.cc:107] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:18:34.189064 12634 raft_consensus.cc:2889] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:34.100899 12629 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:18:34.184495 13385 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:34.039641 14934 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:34.183876 12625 pending_rounds.cc:85] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 4
I20260709 13:18:34.191701 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:34.191753 12672 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:34.191960 12625 pending_rounds.cc:107] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:18:34.192752 12625 raft_consensus.cc:2889] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:34.090361 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:34.194911 12631 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:34.194109 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:34.195888 12628 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:34.130203 15013 raft_consensus.cc:697] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:34.088315 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:34.197670 12625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1162) took 1054 ms. Trace:
I20260709 13:18:34.197785 12673 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:34.159281 15113 raft_consensus.cc:3055] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:18:34.198812 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.198951 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:34.197638 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 156268us
I20260709 13:18:34.202461 12632 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
W20260709 13:18:34.203608 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.208562 12673 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:34.203712 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:34.205888 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:34.204735 12625 rpcz_store.cc:276] 0709 13:18:33.143516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.143621 (+   105us) service_pool.cc:224] Handling call
0709 13:18:33.180390 (+ 36769us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:33.532774 (+352384us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54002795a0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d329dc0 00007f4c6d3b41c0 00007f4c6d3b5231 00007f4c6d3b1d8f 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c
0709 13:18:34.197396 (+664622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.197612 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244992384}
I20260709 13:18:34.209852 12629 raft_consensus.cc:2393] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 6: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:34.210999 12632 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:34.211395 12628 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:34.205801 12746 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:34.103879 12626 raft_consensus.cc:2955] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 2 to 5, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:34.213938 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:34.214700 12627 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:34.195216 14934 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.198331 14479 raft_consensus.cc:3055] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:34.202219 15102 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:34.221771 12747 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:34.222167 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:34.223726 15009 raft_consensus.cc:3055] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:34.227734 15009 raft_consensus.cc:740] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:34.228588 14771 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:34.229084 14771 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.228804 15009 consensus_queue.cc:260] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:34.229486 14771 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:34.230104 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:34.233160 15008 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:34.233326 14965 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.234133 14965 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:34.207583 15014 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:34.160995 14939 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:34.224262 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:34.204784 14680 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:34.253876 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:34.223063 12625 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:34.226436 12632 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:34.208716 15113 raft_consensus.cc:740] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:18:34.251582 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.252948 15079 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.218186 14980 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:34.214638 15099 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:34.247944 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:34.192436 14820 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:18:34.215072 15069 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:34.254688 12782 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:34.347122 12783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 246) took 1232 ms. Trace:
I20260709 13:18:34.402275 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:34.415742 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:34.432672 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:34.344179 12708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1548) took 1428 ms (client timeout 1805 ms). Trace:
W20260709 13:18:34.315042 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.218071 15077 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.098s
I20260709 13:18:34.345551 12707 raft_consensus.cc:1240] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:34.207842 15039 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:34.224002 14847 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:34.360853 12776 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:34.253860 14680 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.229206 15026 consensus_queue.cc:1059] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:18:34.223760 14479 raft_consensus.cc:740] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:34.242267 15014 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.220664 14934 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:34.314635 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:34.472669 15099 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.433775 12704 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:18:34.437307 12709 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:18:34.415549 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:34.337893 14973 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:34.346179 15122 log.cc:927] Time spent T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
W20260709 13:18:34.342658 15121 log.cc:927] Time spent T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 13:18:34.317814 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:34.538566 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 222114us
I20260709 13:18:34.539351 12747 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:34.320297 12672 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:34.540057 15026 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:34.468588 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:34.469357 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.372299 12632 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:18:34.541028 12705 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:34.383117 12625 raft_consensus.cc:1275] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:18:34.541611 12625 pending_rounds.cc:85] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 3
I20260709 13:18:34.541884 12625 pending_rounds.cc:107] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:18:34.541723 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:34.542148 12625 raft_consensus.cc:2889] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:34.542372 12632 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:34.543586 12625 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:34.449700 14965 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:34.483592 12703 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:34.480458 12701 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:34.481122 12706 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
W20260709 13:18:34.480865 12708 rpcz_store.cc:269] 0709 13:18:32.916008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:32.917670 (+  1662us) service_pool.cc:224] Handling call
0709 13:18:34.343994 (+1426324us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b540021edd4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:34.344104 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":521203712}
I20260709 13:18:34.546643 12673 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:34.547479 14965 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:34.547549 12673 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:34.548216 14965 raft_consensus.cc:697] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:34.548373 15039 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:34.486498 14891 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:34.549919 14891 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:34.552716 14891 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:34.547111 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:34.471117 14745 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:34.553929 12708 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:34.554289 12777 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:34.549197 14965 consensus_queue.cc:237] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:34.497262 12775 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:34.348004 14944 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:34.499882 15014 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:34.501956 15124 log.cc:927] Time spent T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:18:34.557883 14955 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:34.558375 14955 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.558754 14955 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:34.499136 14847 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:34.560734 14847 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:34.517340 15073 consensus_meta.cc:287] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.555s	user 0.010s	sys 0.000s
I20260709 13:18:34.564298 15073 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:34.502821 15123 log.cc:927] Time spent T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 13:18:34.480306 12783 rpcz_store.cc:276] 0709 13:18:33.114665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.202229 (+ 87564us) service_pool.cc:224] Handling call
0709 13:18:33.202557 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:33.806083 (+603526us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.346833 (+540750us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b54000cba54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:34.347040 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1246502656}
I20260709 13:18:34.473680 14980 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:34.454912 15008 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:18:34.523092 14939 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:34.576159 14317 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:34.525525 14682 raft_consensus.cc:3055] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:18:34.527230 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.528307 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:34.577656 12630 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:34.529084 14712 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.532717 12631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 1992) took 1156 ms. Trace:
I20260709 13:18:34.582666 12631 rpcz_store.cc:276] 0709 13:18:33.376749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.614647 (+237898us) service_pool.cc:224] Handling call
0709 13:18:33.614880 (+   233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:34.197055 (+582175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.532461 (+335406us) spinlock_profiling.cc:243] Waited 577 ms on lock 0x7b540004b5d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:34.532660 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1326836608}
I20260709 13:18:34.493813 14680 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:34.534591 15133 log.cc:927] Time spent T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:18:34.536209 15038 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:34.515415 15099 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:34.469575 15013 consensus_queue.cc:237] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:34.452401 15113 consensus_queue.cc:260] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:34.539459 15027 consensus_meta.cc:287] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.500s	user 0.008s	sys 0.001s
W20260709 13:18:34.540534 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 220349us
I20260709 13:18:34.544903 14973 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:34.506165 14479 consensus_queue.cc:260] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:34.572999 12747 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:34.573974 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:34.520928 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:34.581030 12673 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:34.670893 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 89993.9us
I20260709 13:18:34.586766 12626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1170) took 1419 ms. Trace:
W20260709 13:18:34.585354 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.475500 15079 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:34.614293 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:34.637427 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.552735 15009 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:34.576222 14980 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:34.582967 14682 raft_consensus.cc:740] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:34.670643 12709 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:34.682742 12672 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:34.671833 12626 rpcz_store.cc:276] 0709 13:18:33.167611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.534896 (+367285us) service_pool.cc:224] Handling call
0709 13:18:33.535538 (+   642us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:18:33.582405 (+ 46867us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:33.794028 (+211623us) log.cc:844] Serialized 14 byte log entry
0709 13:18:34.025061 (+231033us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:18:34.065236 (+ 40175us) log.cc:844] Serialized 608 byte log entry
0709 13:18:34.103750 (+ 38514us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:34.544966 (+441216us) log.cc:844] Serialized 14 byte log entry
0709 13:18:34.577205 (+ 32239us) log.cc:844] Serialized 14 byte log entry
0709 13:18:34.585606 (+  8401us) log.cc:844] Serialized 14 byte log entry
0709 13:18:34.586235 (+   629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.586349 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:34.586416 (+    67us) raft_consensus.cc:1685] finished
0709 13:18:34.586456 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:34.586684 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":168005,"prepare.queue_time_us":33636,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":76,"raft.queue_time_us":76671,"raft.run_cpu_time_us":462,"raft.run_wall_time_us":62066,"spinlock_wait_cycles":59897728,"thread_start_us":394521,"threads_started":2,"wal-append.queue_time_us":223790}
I20260709 13:18:34.589919 14847 consensus_queue.cc:1048] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.075s
I20260709 13:18:34.665771 14973 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:34.644141 15026 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:34.715498 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 142587us
I20260709 13:18:34.723362 12747 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:34.724606 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:34.725862 14317 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:34.644060 15038 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:34.720881 13383 raft_consensus.cc:1217] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[3.5-3.5]   Dedup: 3.5->[]
W20260709 13:18:34.720654 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 383957us
I20260709 13:18:34.730089 13383 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:18:34.688112 14978 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.669533 15102 consensus_queue.cc:1059] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 13:18:34.677903 14939 consensus_queue.cc:1059] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 13:18:34.670312 14712 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:34.707655 15095 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:34.666491 15137 consensus_queue.cc:1048] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.086s
I20260709 13:18:34.755932 14317 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:18:34.756567 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 87162.5us
I20260709 13:18:34.721727 14973 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:34.723337 15027 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:18:34.774642 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 67115.7us
I20260709 13:18:34.779385 12672 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:34.776153 12597 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:34.719653 15136 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:18:34.792100 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.757269 15116 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.804268 12626 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:34.777470 15077 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:34.809541 12528 catalog_manager.cc:5697] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 1 to 5, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 5 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:34.761327 14965 consensus_queue.cc:1059] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:18:34.742941 14771 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:34.818161 15072 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:18:34.818414 15140 log.cc:927] Time spent T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:18:34.822089 15072 consensus_queue.cc:1059] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:34.791808 15136 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.822292 12672 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:34.822937 15136 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:34.729089 15026 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:34.743566 14745 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 13:18:34.781533 15027 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:34.783660 15073 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:34.805516 12673 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:34.816684 14965 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:34.757819 15069 consensus_queue.cc:1059] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
W20260709 13:18:34.863083 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 80215.5us
I20260709 13:18:34.864342 12597 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:34.864559 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 134892us
W20260709 13:18:34.868654 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 46468.1us
I20260709 13:18:34.787088 15095 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:34.758914 14978 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:34.869537 12672 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:34.867962 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:34.888224 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:34.889061 14965 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:34.874956 15154 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:18:34.893991 14991 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:34.894483 14991 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.893643 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:34.894887 14991 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:34.895206 12700 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:34.899883 12709 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:34.901262 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:34.866534 15027 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:34.902046 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:34.906091 12632 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:18:34.810907 15077 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:34.865967 14745 consensus_queue.cc:1243] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:18:34.910388 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:34.910777 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 105362us
W20260709 13:18:34.911298 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 47045us
I20260709 13:18:34.912390 12672 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:34.912580 12627 raft_consensus.cc:1240] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:34.898917 14847 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:34.897261 15137 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:34.918663 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:34.918871 12597 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:34.812340 15116 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:34.795039 14682 consensus_queue.cc:260] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:34.909932 15069 raft_consensus.cc:3055] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:34.916401 14965 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:34.936295 15159 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:34.941073 15148 raft_consensus.cc:3055] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:34.919060 14847 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:34.916386 15141 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.925992 15154 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:34.941496 15148 raft_consensus.cc:740] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:34.943293 15149 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:34.936300 15155 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:35.006199 15155 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:35.006800 15155 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:35.006891 15148 consensus_queue.cc:260] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:34.947242 15139 log.cc:927] Time spent T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:18:34.936525 15151 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.009657 15151 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:34.944825 14944 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:35.011135 14712 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:34.949649 15153 raft_consensus.cc:3055] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:35.011934 15038 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:35.012022 15153 raft_consensus.cc:740] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:34.950173 15115 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:34.919148 15137 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:35.005123 14708 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 13:18:35.013638 15137 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:34.915730 15077 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:34.926247 15079 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:35.016964 12707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1214) took 1888 ms. Trace:
I20260709 13:18:35.017498 12707 rpcz_store.cc:276] 0709 13:18:33.128560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.128700 (+   140us) service_pool.cc:224] Handling call
0709 13:18:34.345363 (+1216663us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:34.482084 (+136721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.016732 (+534648us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b540021edd0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:35.016891 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2762958080}
I20260709 13:18:34.918048 14934 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:34.926949 15069 raft_consensus.cc:740] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:34.999217 15008 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.008306 15148 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:35.008646 15149 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:35.021075 12633 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 5. Current term is 6. Ops: []
W20260709 13:18:34.953403 15135 log.cc:927] Time spent T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.478s	user 0.002s	sys 0.000s
I20260709 13:18:35.005165 15154 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:35.023212 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:34.962641 14847 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:34.982355 15141 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:35.036842 15008 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:35.047135 12528 catalog_manager.cc:5697] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 1 to 4, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 4 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:35.038230 15069 consensus_queue.cc:260] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:35.039362 14745 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:35.121163 14745 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:35.123278 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.145902 14974 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:18:35.082497 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:35.148772 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.146787 14974 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:35.033294 15153 consensus_queue.cc:260] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:35.159915 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:35.182390 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.079181 14771 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:34.928572 12632 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
W20260709 13:18:35.200986 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:35.178488 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:35.210258 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 284281us
I20260709 13:18:35.211195 13383 raft_consensus.cc:1240] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:18:35.215588 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:35.218348 13383 raft_consensus.cc:2393] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 3: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:35.223209 13379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 465) took 1006 ms. Trace:
I20260709 13:18:35.215839 12632 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:18:35.222776 12528 catalog_manager.cc:5697] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:35.242465 12528 catalog_manager.cc:5697] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 2 to 4, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 4 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:35.224985 13379 rpcz_store.cc:276] 0709 13:18:34.216942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.217169 (+   227us) service_pool.cc:224] Handling call
0709 13:18:34.227143 (+  9974us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:34.243115 (+ 15972us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:34.243162 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:34.334081 (+ 90919us) log.cc:844] Serialized 26 byte log entry
0709 13:18:34.691252 (+357171us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:34.885660 (+194408us) log.cc:844] Serialized 14 byte log entry
0709 13:18:34.886032 (+   372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.886151 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:35.222984 (+336833us) raft_consensus.cc:1685] finished
0709 13:18:35.223006 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:35.223139 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17300,"prepare.queue_time_us":114248,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":20987,"raft.run_cpu_time_us":9420,"raft.run_wall_time_us":223145,"thread_start_us":355421,"threads_started":2,"wal-append.queue_time_us":736746}
I20260709 13:18:35.247501 14965 raft_consensus.cc:697] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:35.249496 12783 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:35.248231 12528 catalog_manager.cc:5697] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:18:35.252511 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.200248 14772 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:35.211566 15102 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.200616 14479 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:35.206060 14682 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:35.221431 15069 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:35.221632 14974 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:35.211723 15161 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:35.223429 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:35.271015 13385 raft_consensus.cc:1240] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:35.242724 14956 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.265375 13379 raft_consensus.cc:1240] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:35.206282 15073 consensus_queue.cc:1059] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
W20260709 13:18:35.259481 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 330460us
W20260709 13:18:35.267707 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.244843 14978 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:35.267599 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 348865us
I20260709 13:18:35.268986 15160 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:35.271854 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:35.287127 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:35.275593 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:35.282114 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:35.280982 12597 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:35.288043 12783 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:35.288614 12628 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:35.290354 12746 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:35.256809 14934 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:35.294083 12673 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:35.261317 15117 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.294955 13379 raft_consensus.cc:1240] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:35.249646 14941 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:18:35.248744 14965 consensus_queue.cc:237] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:35.301365 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:35.302075 12628 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:35.303238 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:35.274194 15113 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:35.279891 14956 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:35.309365 12597 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:35.311908 14971 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:35.312014 15141 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:35.314604 15163 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:35.310282 14978 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:35.296658 14983 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.267163 15102 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:35.304111 14941 consensus_queue.cc:1243] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:18:35.296655 14820 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:35.312224 14969 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:35.280175 15161 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:35.312815 15038 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:35.304739 15160 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:35.329275 12700 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:35.411187 12783 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:18:35.286415 14771 consensus_queue.cc:1048] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:18:35.309924 12747 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:35.299975 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:35.417850 13376 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:35.300133 15008 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:35.300093 15026 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:35.265377 14772 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:35.313936 15095 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:35.332697 13377 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:18:35.371632 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 71719.8us
I20260709 13:18:35.341885 15151 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:35.340315 15115 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:35.331048 12597 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:35.331844 14971 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:35.367412 14983 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:35.435716 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 125907us
I20260709 13:18:35.366114 14978 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:35.436663 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 133500us
I20260709 13:18:35.439723 15175 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:18:35.441161 12529 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54814 (request call id 219) took 11723 ms (client timeout 15000 ms). Trace:
I20260709 13:18:35.458663 14745 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:35.530755 15099 consensus_meta.cc:287] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.862s	user 0.008s	sys 0.014s
I20260709 13:18:35.436901 15152 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:35.365994 15154 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:35.365934 15166 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:35.503497 12778 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:35.508993 15161 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:35.509164 14479 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:35.512306 15148 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.512555 12629 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:18:35.586385 12629 pending_rounds.cc:85] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 5
I20260709 13:18:35.586737 12629 pending_rounds.cc:107] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:18:35.587016 12629 raft_consensus.cc:2889] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:35.516144 14944 consensus_meta.cc:287] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.505s	user 0.007s	sys 0.010s
I20260709 13:18:35.518091 15102 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:35.406487 15116 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:35.519083 14772 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:35.466187 14955 consensus_meta.cc:287] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.907s	user 0.001s	sys 0.020s
I20260709 13:18:35.420012 15141 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:18:35.465750 14980 consensus_meta.cc:287] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.755s	user 0.012s	sys 0.004s
W20260709 13:18:35.527684 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 196720us
I20260709 13:18:35.326668 15149 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:35.365469 15162 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.499184 14974 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.613550 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:35.538331 12747 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:35.540273 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:35.614899 13383 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:35.615254 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 77032.1us
I20260709 13:18:35.411828 14969 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:35.550607 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:35.550778 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:35.305287 15117 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:35.601050 14955 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:35.617729 15120 consensus_queue.cc:1059] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:18:35.623826 15120 raft_consensus.cc:3055] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:35.624022 15117 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:35.624214 15120 raft_consensus.cc:740] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:35.624819 14955 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:35.625705 14745 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:35.626300 14745 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:35.626819 14745 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:35.627795 15165 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:35.628533 15165 raft_consensus.cc:697] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:18:35.528283 14973 consensus_meta.cc:287] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.737s	user 0.005s	sys 0.018s
I20260709 13:18:35.530138 15115 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.569018 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:35.630347 14973 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:35.631353 13379 raft_consensus.cc:2393] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 7: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:35.630513 15115 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:35.569355 14956 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:35.569514 14991 consensus_meta.cc:287] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.674s	user 0.005s	sys 0.018s
I20260709 13:18:35.569870 14479 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:35.570901 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:35.569216 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:35.559716 15152 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:18:35.572012 15077 consensus_meta.cc:287] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.556s	user 0.009s	sys 0.010s
W20260709 13:18:35.572882 15155 consensus_meta.cc:287] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.565s	user 0.007s	sys 0.013s
I20260709 13:18:35.572803 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:18:35.546962 12529 rpcz_store.cc:269] 0709 13:18:23.671188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.673725 (+  2537us) service_pool.cc:224] Handling call
0709 13:18:35.312142 (+11638417us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:28.393621 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:28.628946 (+235325us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:28.633167 (+  4221us) write_op.cc:435] Acquired schema lock
0709 13:18:28.638938 (+  5771us) tablet.cc:662] Decoding operations
0709 13:18:28.790841 (+151903us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:28.790904 (+    63us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:28.790936 (+    32us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:18:28.792173 (+  1237us) tablet.cc:701] Row locks acquired
0709 13:18:28.792193 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:18:28.792359 (+   166us) write_op.cc:270] Start()
0709 13:18:28.792481 (+   122us) write_op.cc:276] Timestamp: P: 1783603108792305 usec, L: 0
0709 13:18:28.792504 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:18:28.793263 (+   759us) log.cc:844] Serialized 6343 byte log entry
0709 13:18:30.017448 (+1224185us) op_driver.cc:464] REPLICATION: finished
0709 13:18:30.521352 (+503904us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b18002499a0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c53f3 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:18:30.523775 (+  2423us) write_op.cc:301] APPLY: starting
0709 13:18:30.525465 (+  1690us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:30.575483 (+ 50018us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:30.578703 (+  3220us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:30.650113 (+ 71410us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:30.650414 (+   301us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 13:18:30.650469 (+    55us) write_op.cc:312] APPLY: finished
0709 13:18:30.650909 (+   440us) log.cc:844] Serialized 113 byte log entry
0709 13:18:30.651162 (+   253us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:30.651347 (+   185us) write_op.cc:454] Released schema lock
0709 13:18:30.651404 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:30.672931 (+ 21527us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:30.674055 (+  1124us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":158522,"reported_tablets":41,"child_traces":[["op",{"apply.queue_time_us":178772,"apply.run_cpu_time_us":5649,"apply.run_wall_time_us":147275,"num_ops":16,"prepare.queue_time_us":378317,"prepare.run_cpu_time_us":39822,"prepare.run_wall_time_us":697007,"raft.queue_time_us":528496,"raft.run_cpu_time_us":9570,"raft.run_wall_time_us":218605,"replication_time_us":1165224,"spinlock_wait_cycles":38635904,"thread_start_us":862608,"threads_started":4,"wal-append.queue_time_us":561929,"wal-append.run_cpu_time_us":23502,"wal-append.run_wall_time_us":2454771}]]}
W20260709 13:18:35.576752 14680 consensus_meta.cc:287] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.903s	user 0.013s	sys 0.004s
W20260709 13:18:35.580173 15009 consensus_meta.cc:287] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.897s	user 0.018s	sys 0.001s
I20260709 13:18:35.581522 14971 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:35.583159 14983 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:35.550010 15175 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:35.586428 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.548491 14879 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:18:35.601730 12597 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:18:35.746155 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 144601us
I20260709 13:18:35.746940 12700 raft_consensus.cc:1240] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:35.616434 12747 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:35.594689 15153 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:35.583154 14978 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:35.561085 15099 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:35.677614 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.634613 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:35.676430 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.596383 14944 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:35.597674 15141 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.633405 14973 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:35.634979 15178 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.742604 12778 raft_consensus.cc:2393] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 5: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
W20260709 13:18:35.670584 15027 consensus_meta.cc:287] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.758s	user 0.005s	sys 0.012s
I20260709 13:18:35.589250 15166 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:35.603725 15116 consensus_queue.cc:1059] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.152s
I20260709 13:18:35.750401 12597 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:35.635347 15145 log.cc:927] Time spent T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:18:35.752386 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:35.789113 12700 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:35.790827 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:35.753973 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:35.754838 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 138533us
I20260709 13:18:35.594467 15148 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:35.757493 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:35.798375 13380 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:35.798569 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:35.742475 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.662343 15160 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:35.736342 15152 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:35.812808 12707 raft_consensus.cc:2393] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 7: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:35.659735 14969 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:35.634848 15176 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.816990 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:35.759507 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:35.817981 12774 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:35.760346 12782 raft_consensus.cc:1240] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7. Ops: []
W20260709 13:18:35.788539 15137 consensus_meta.cc:287] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.774s	user 0.013s	sys 0.006s
I20260709 13:18:35.767846 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:35.770323 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:35.744517 14956 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.750064 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:35.745913 15159 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.784860 13383 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:35.745487 15117 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:35.793478 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:35.795063 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:35.799921 12702 raft_consensus.cc:1240] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:18:35.804562 12703 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:35.751520 15175 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:35.758227 15025 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:35.788442 15008 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:35.603138 14980 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:35.625250 15120 consensus_queue.cc:260] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:35.758747 15141 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:35.811656 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:35.808508 12747 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:35.843107 12708 raft_consensus.cc:2376] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5. Re-sending same reply.
I20260709 13:18:35.761909 12781 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:18:35.743108 14680 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:35.677433 15162 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:35.742152 15155 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:35.736573 14991 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:35.822304 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:35.855547 12705 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:35.646821 14974 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:35.777539 15072 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:35.831588 15181 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:35.736161 15077 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:35.831511 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:35.754295 15009 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:35.903126 12783 raft_consensus.cc:1275] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:18:35.836622 12673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.002s	sys 0.011s
I20260709 13:18:35.782589 14971 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:18:35.841550 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:35.746459 15165 consensus_queue.cc:237] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:35.820127 15079 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:35.847906 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:35.847376 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:35.782392 15027 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:35.830866 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:35.871167 12781 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:18:35.831398 15127 consensus_queue.cc:1059] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
W20260709 13:18:35.871577 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.780032 15178 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:35.832199 15025 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:35.818595 15038 consensus_queue.cc:1059] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:18:35.953765 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:35.832657 15137 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:35.954411 13384 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:35.837363 15116 raft_consensus.cc:3055] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:35.776978 14613 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:18:35.883041 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.847695 15154 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.893133 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:35.969197 12705 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:35.896777 12780 raft_consensus.cc:2368] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7.
I20260709 13:18:35.970013 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:35.970387 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.906494 12707 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:35.906574 12709 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:35.817289 15026 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.869390 14682 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:35.835759 14956 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:35.930944 12703 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:35.901754 15120 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:35.835568 15117 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:35.907969 15014 consensus_meta.cc:287] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 1.351s	user 0.013s	sys 0.010s
W20260709 13:18:35.938578 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 122264us
I20260709 13:18:35.949448 12630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1192) took 1045 ms. Trace:
I20260709 13:18:35.953044 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:35.900241 15009 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.948643 13380 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:35.907232 14479 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:35.959471 12702 raft_consensus.cc:2393] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 4: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
W20260709 13:18:35.958841 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 134561us
I20260709 13:18:35.968642 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:35.904563 12673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.002s	sys 0.011s
I20260709 13:18:35.954713 15025 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:35.827581 15176 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:35.974505 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:35.974997 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:35.831213 15159 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:35.980608 12704 raft_consensus.cc:1240] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:35.982254 12630 rpcz_store.cc:276] 0709 13:18:34.903630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.903781 (+   151us) service_pool.cc:224] Handling call
0709 13:18:34.904102 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:35.274031 (+369929us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:35.274064 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:35.290296 (+ 16232us) log.cc:844] Serialized 26 byte log entry
0709 13:18:35.839026 (+548730us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:35.839063 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.839128 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:35.949243 (+110115us) raft_consensus.cc:1685] finished
0709 13:18:35.949262 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:35.949395 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8582,"raft.queue_time_us":44479,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":8756,"thread_start_us":829125,"threads_started":2,"wal-append.queue_time_us":625436}
I20260709 13:18:35.983520 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:35.959146 15116 raft_consensus.cc:740] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:35.912211 15072 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:35.972679 15038 consensus_queue.cc:1243] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 639ab2db265f4747bb7afb93e833f530 log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 13:18:36.043658 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 58408.9us
I20260709 13:18:36.064798 14317 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.065362 14317 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.065609 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:36.047176 12700 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:35.971557 15192 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.910774 15170 consensus_queue.cc:1059] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
W20260709 13:18:36.068002 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.046674 12746 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:36.043260 12781 raft_consensus.cc:2410] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:18:36.060205 12705 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:36.069757 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:35.983979 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:35.986589 14682 raft_consensus.cc:697] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:35.971493 15079 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:36.119765 15079 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:36.121726 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:35.980394 15026 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.122849 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:36.123504 12780 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:36.124293 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:36.124540 15026 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:36.127219 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.128321 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.129164 12627 raft_consensus.cc:2393] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 6: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:36.129688 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.130426 12626 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 5: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:36.128352 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.130826 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.131197 12631 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 2.
I20260709 13:18:36.132750 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:36.133522 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.134305 12632 raft_consensus.cc:2410] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:18:36.135154 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.135664 12632 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:36.136430 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.138139 12633 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:36.139283 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.140462 12630 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:18:36.141748 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:36.074128 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:35.980276 15149 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:35.972407 14613 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:36.044471 14941 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.155200 12634 raft_consensus.cc:2393] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 6: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:36.076079 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:35.989626 15009 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:36.050704 15127 consensus_queue.cc:1059] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:18:36.078498 13381 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 5: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:36.078182 15116 consensus_queue.cc:260] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:36.143273 14847 consensus_meta.cc:287] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 1.066s	user 0.004s	sys 0.014s
I20260709 13:18:36.080749 15072 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:36.073935 15163 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:36.193795 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:36.081702 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:36.082095 15155 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:35.900599 15181 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:36.082738 14974 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:36.085247 14680 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:36.087543 15166 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:36.088019 15115 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.088979 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:36.090252 14479 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:36.091712 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:35.973335 15154 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.093129 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:36.094512 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:36.095474 15162 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:36.231709 15069 consensus_meta.cc:287] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.965s	user 0.006s	sys 0.004s
I20260709 13:18:36.231848 15154 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:36.095932 15148 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:36.234876 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:36.096542 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:36.097867 15136 consensus_meta.cc:287] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 1.228s	user 0.012s	sys 0.008s
I20260709 13:18:36.098588 14956 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:36.100642 14991 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:36.102088 15178 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:36.102869 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:36.256259 12781 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
W20260709 13:18:36.103102 15184 log.cc:927] Time spent T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
I20260709 13:18:36.106045 13384 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:36.102114 14980 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:36.106473 15137 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:36.106842 14317 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:36.106880 15152 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:36.108655 14682 consensus_queue.cc:237] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.110241 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:36.275919 12783 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:35.989461 15014 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.111562 15161 raft_consensus.cc:3055] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:36.112507 15077 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:36.112623 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:36.113644 15027 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:36.113768 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:36.281576 12703 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 5: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:36.060669 14891 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.144562 12705 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:36.137413 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.283913 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.284754 12632 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:36.150219 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:36.285900 12702 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:36.153036 12673 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:36.287629 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 134707us
I20260709 13:18:36.054291 15099 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:36.154330 12780 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:18:36.157294 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 86271.1us
I20260709 13:18:36.162961 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:36.293151 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:36.293413 12597 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:36.163601 12672 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:36.167354 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:36.170254 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:36.170663 15175 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:36.295059 12701 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
I20260709 13:18:36.171427 14941 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:36.080446 15170 raft_consensus.cc:3055] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:36.189119 12774 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:36.181916 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:36.195206 13382 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:36.143055 14745 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.191790 14944 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.228072 12775 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:36.225836 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:36.230165 14479 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.231194 15181 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:36.322441 15165 consensus_queue.cc:1059] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:18:36.180454 15026 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.342963 15026 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.344641 14941 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:36.200625 15116 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:36.345651 14944 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.347750 14944 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:36.347906 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.239108 15166 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.349550 14479 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:36.350040 15166 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.352246 15166 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:36.134653 15038 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.352954 15038 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:36.355195 12776 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
W20260709 13:18:36.201085 15190 consensus_queue.cc:1175] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "19cd864c053744bebd96093a7982537e" responder_term: 6 status { last_received { term: 6 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 6 index: 7 } }
I20260709 13:18:36.075234 15192 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:36.223913 15163 raft_consensus.cc:697] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:36.197786 14847 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.274295 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:36.366223 12626 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:18:36.278209 12781 raft_consensus.cc:1240] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:18:36.279944 12707 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:36.289543 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.367825 12632 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:36.343137 14745 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:36.171646 14934 raft_consensus.cc:3055] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:36.308141 12597 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:36.308260 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:36.295091 12704 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:36.248342 15117 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:36.310029 15141 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.311134 15175 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:36.379171 12598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.038s	sys 0.028s
I20260709 13:18:36.313859 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:36.379906 12703 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:36.379841 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:36.380513 12598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.038s	sys 0.028s
I20260709 13:18:36.380616 12707 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:36.247071 15136 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.296053 12778 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:36.279491 15161 raft_consensus.cc:740] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:36.330602 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:36.331986 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 168444us
I20260709 13:18:36.384394 12672 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:36.333348 12628 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
W20260709 13:18:36.334014 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.307662 14980 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:36.326879 15077 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:36.337896 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:36.322104 15148 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:36.360196 12702 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:36.361014 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:36.423208 15148 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:36.423558 12701 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:36.423729 15148 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:36.423954 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:36.424662 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:36.425289 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:36.345347 15026 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:36.355302 15009 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.328534 15170 raft_consensus.cc:740] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:36.441388 12709 raft_consensus.cc:2368] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7.
W20260709 13:18:36.441978 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 57647.2us
I20260709 13:18:36.443011 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:36.368980 12632 pending_rounds.cc:85] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 2
I20260709 13:18:36.444648 12632 pending_rounds.cc:107] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:18:36.445030 12632 raft_consensus.cc:2889] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:36.293826 12631 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:36.163290 13349 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:36.307227 14891 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:36.371030 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 62963.4us
I20260709 13:18:36.371222 12705 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:36.311169 13385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 473) took 1024 ms. Trace:
I20260709 13:18:36.316962 15014 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:36.392104 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:36.370568 14934 raft_consensus.cc:740] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:36.379305 15069 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:36.393332 14980 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:36.382124 14613 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:36.406358 15161 consensus_queue.cc:260] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:36.410878 15176 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:36.411902 15153 consensus_meta.cc:287] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.658s	user 0.003s	sys 0.006s
I20260709 13:18:36.415825 12598 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:36.419566 15159 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:36.420720 14820 raft_consensus.cc:3055] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:36.436985 15077 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:36.420812 15115 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:36.422190 15152 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:36.426098 12702 raft_consensus.cc:2393] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 7: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:36.426955 12779 raft_consensus.cc:2393] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:36.425915 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:36.437431 12703 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:36.334414 15183 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 13:18:36.456473 14980 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:36.455950 15178 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:36.447351 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:36.489681 12705 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:36.449361 12704 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:36.490243 12673 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:36.451025 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 287802us
W20260709 13:18:36.404806 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.493165 12597 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:36.494808 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:36.496407 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.497481 13348 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:36.446714 13385 rpcz_store.cc:276] 0709 13:18:35.286783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.292621 (+  5838us) service_pool.cc:224] Handling call
0709 13:18:35.310531 (+ 17910us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:35.567428 (+256897us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:35.567467 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:35.616376 (+ 48909us) log.cc:844] Serialized 26 byte log entry
0709 13:18:36.005389 (+389013us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:36.064730 (+ 59341us) log.cc:844] Serialized 14 byte log entry
0709 13:18:36.064973 (+   243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.065051 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:36.119920 (+ 54869us) spinlock_profiling.cc:243] Waited 7.67 ms on lock 0x7b180033dd80. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c5522 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:18:36.310838 (+190918us) raft_consensus.cc:1685] finished
0709 13:18:36.310951 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.311104 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39662,"prepare.queue_time_us":62016,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":61,"raft.queue_time_us":129525,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":8491,"spinlock_wait_cycles":17640320,"thread_start_us":450751,"threads_started":2,"wal-append.queue_time_us":567133}
I20260709 13:18:36.467368 12630 raft_consensus.cc:2393] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 7: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:18:36.475370 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 59617.6us
I20260709 13:18:36.453018 14479 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:36.453119 15026 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.507117 12703 raft_consensus.cc:1240] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:36.511569 12673 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:36.458766 15038 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:36.523309 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.523940 12627 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:36.470376 15014 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:36.571349 15014 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:36.571884 15014 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:36.507485 15176 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:36.517442 15190 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:18:36.573689 15170 consensus_queue.cc:260] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:36.479251 14941 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.574318 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.574772 14941 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:36.575058 12625 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:36.507411 15025 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:36.576155 15153 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:36.576301 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.576680 15153 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:36.576864 14941 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:36.576978 12625 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:36.577272 12777 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:36.517701 15175 consensus_queue.cc:1059] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:18:36.577867 12705 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:18:36.574355 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 50990.7us
I20260709 13:18:36.578446 15175 raft_consensus.cc:3055] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:36.578598 15141 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:36.578852 15175 raft_consensus.cc:740] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:36.574731 15179 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:18:36.579946 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:36.579717 15175 consensus_queue.cc:260] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:36.580190 12704 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:18:36.580894 15175 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:36.574743 15170 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:36.583055 14613 consensus_queue.cc:1175] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "19cd864c053744bebd96093a7982537e" responder_term: 6 status { last_received { term: 6 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 6 index: 6 } }
I20260709 13:18:36.524780 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:36.584209 12627 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:18:36.523102 15099 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:36.585952 14934 consensus_queue.cc:260] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.587230 14934 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:36.470639 15069 raft_consensus.cc:697] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:36.589480 15155 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:36.590266 15069 consensus_queue.cc:237] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:36.590804 14891 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:36.591334 14891 consensus_queue.cc:1059] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:36.592072 15153 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:36.592115 14891 raft_consensus.cc:3055] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:36.592639 14891 raft_consensus.cc:740] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:18:36.594070 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.468633 14820 raft_consensus.cc:740] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:36.570312 12702 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:36.521680 15026 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.488732 15163 consensus_queue.cc:237] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:36.569013 15136 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:36.622102 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 99477.2us
I20260709 13:18:36.621111 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:36.628419 12672 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:36.603176 12701 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
W20260709 13:18:36.596655 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 70571.4us
W20260709 13:18:36.630122 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.630188 12672 leader_election.cc:400] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7.
I20260709 13:18:36.614907 12673 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:36.615414 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 98314.3us
I20260709 13:18:36.507267 15178 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:36.632990 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:36.615478 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:36.633947 12634 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:36.619630 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:36.479614 15199 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:36.634229 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.635578 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.636240 12625 raft_consensus.cc:2393] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 6: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:36.635875 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:36.637451 12701 raft_consensus.cc:2410] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:18:36.638942 15120 consensus_meta.cc:287] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.654s	user 0.010s	sys 0.000s
I20260709 13:18:36.640478 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.688810 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.641786 15141 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:36.641891 15136 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:36.698602 12634 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:18:36.699275 12633 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:36.699599 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:36.642765 14820 consensus_queue.cc:260] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:36.476166 15166 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.152s
I20260709 13:18:36.651167 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:36.593477 14891 consensus_queue.cc:260] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:36.654438 15192 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:36.710069 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:36.710850 12779 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:36.656982 14847 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:36.656806 15179 consensus_queue.cc:1048] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:36.658293 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:36.659435 15009 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:36.661124 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:36.718267 12702 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:36.662005 15013 consensus_queue.cc:1059] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:36.662372 15159 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:36.663265 12673 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:36.664417 15072 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.664603 15197 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:36.655540 15148 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.666976 15014 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:36.669004 15178 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:36.614424 15115 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:36.476966 15077 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:36.670964 13349 leader_election.cc:336] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.560s (SENT)
I20260709 13:18:36.754798 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:36.756191 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 85251.1us
W20260709 13:18:36.757162 13349 leader_election.cc:336] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.611s (SENT)
I20260709 13:18:36.674800 13379 raft_consensus.cc:1240] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:36.684458 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:36.698865 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:36.758510 12781 raft_consensus.cc:2393] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 5: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:36.701735 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.758884 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:36.708971 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:36.706772 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:36.713312 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:36.713218 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:36.761209 12780 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:36.709404 12778 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:36.723330 12701 raft_consensus.cc:1240] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:36.726199 12672 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:36.730261 12706 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:36.698342 14941 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.773011 12625 raft_consensus.cc:2393] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 6: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:36.698848 15136 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:36.749998 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:36.780329 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:36.781052 12708 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:36.782090 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:36.782743 12708 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:36.759029 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:36.651170 15026 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:36.763031 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:36.771481 12783 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
W20260709 13:18:36.771626 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 108452us
I20260709 13:18:36.775818 12782 raft_consensus.cc:2393] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 6: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:36.795023 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:36.776991 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.778456 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:36.793030 13349 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:18:36.782593 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 56564.3us
I20260709 13:18:36.800626 12707 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:36.801280 12672 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:36.674577 12627 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:36.802795 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.786494 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:36.804862 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:36.805799 12701 raft_consensus.cc:2393] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 6: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:36.802351 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.806633 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:36.808385 12701 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:36.807613 12625 raft_consensus.cc:2393] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 6: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:36.714685 15155 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:18:36.760486 14891 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:36.771742 14680 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.789932 14956 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:36.825294 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:36.840315 12781 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:18:36.826586 12631 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:36.840559 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:36.841904 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:36.829018 12702 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:36.771370 15192 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:36.787688 14613 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:36.772598 14820 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:36.788616 15136 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:36.794557 14479 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:36.818534 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.851840 12780 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:36.856504 12625 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:36.797248 15185 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:36.753136 15159 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.763856 15197 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.857179 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:36.770771 15072 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.857421 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.856885 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:36.820034 15116 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:36.859890 12779 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:36.863154 12780 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:36.863476 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:36.868503 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.868911 13385 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:36.868533 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:36.788218 14941 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.875886 12708 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:18:36.848376 14680 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:36.877817 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:36.878502 12780 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:36.879635 14680 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:36.880204 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:36.880834 12779 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:36.885128 12701 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:36.879570 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:36.849905 15026 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:36.891778 12634 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:36.888280 12780 raft_consensus.cc:2393] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 7: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:36.890812 12597 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:36.894891 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.896080 12625 raft_consensus.cc:2410] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:18:36.896083 12597 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:36.859705 14956 raft_consensus.cc:697] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:36.908386 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:36.865732 14939 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:36.905012 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:36.913182 12775 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:36.906613 12631 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:36.914942 13385 raft_consensus.cc:2410] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:18:36.918259 15153 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.920598 15153 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:36.908123 15013 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:36.922945 15159 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:36.924010 15014 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:36.925014 12631 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:36.926860 14941 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:36.927927 12775 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:36.927822 14956 consensus_queue.cc:237] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.908493 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.894220 15115 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:36.929986 14939 raft_consensus.cc:697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:36.930195 15115 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:36.930843 15095 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:36.931416 15095 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:36.931804 15095 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:36.931119 14939 consensus_queue.cc:237] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:36.929926 12625 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:36.932337 15115 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:36.933935 15197 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:36.935196 15077 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:36.893990 15009 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:36.872025 15116 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:36.908653 12597 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:18:36.938200 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.940553 12597 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:36.941437 12781 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:36.942531 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:36.946826 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:36.906373 15069 raft_consensus.cc:3055] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:36.859524 14613 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:36.954847 12775 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:18:36.956154 12774 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:18:36.956640 12774 pending_rounds.cc:85] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 3
I20260709 13:18:36.956930 12774 pending_rounds.cc:107] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:18:36.957491 12774 raft_consensus.cc:2889] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:36.960039 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:36.937434 15148 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:36.906375 15026 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:37.067518 12632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2021) took 1011 ms. Trace:
I20260709 13:18:37.060748 15199 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:37.202001 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 400823us
I20260709 13:18:37.202828 15199 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:37.203195 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:37.062392 12706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1563) took 1292 ms. Trace:
I20260709 13:18:37.204263 14956 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:37.204228 12706 rpcz_store.cc:276] 0709 13:18:35.769916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.770047 (+   131us) service_pool.cc:224] Handling call
0709 13:18:37.062059 (+1292012us) spinlock_profiling.cc:243] Waited 647 ms on lock 0x7b5400405454. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:37.062203 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1488977664}
I20260709 13:18:37.205042 14956 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:18:37.008291 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.003938 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:36.941708 15116 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:37.179574 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.213435 12630 raft_consensus.cc:1217] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Deduplicated request from leader. Original: 2.2->[5.3-7.4]   Dedup: 5.3->[7.4-7.4]
I20260709 13:18:36.951830 15115 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.214543 12747 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:37.214905 12630 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8. Ops: [5.3-7.4]
I20260709 13:18:37.215325 14934 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:37.215816 14934 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:37.216192 14934 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:37.216842 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:37.206174 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 407020us
I20260709 13:18:37.201315 13381 raft_consensus.cc:1240] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:37.058692 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:37.201224 12632 rpcz_store.cc:276] 0709 13:18:36.055704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.368448 (+312744us) service_pool.cc:224] Handling call
0709 13:18:36.368769 (+   321us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:36.466363 (+ 97594us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:36.466412 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:36.467183 (+   771us) log.cc:844] Serialized 52 byte log entry
0709 13:18:36.790337 (+323154us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:37.052387 (+262050us) log.cc:844] Serialized 14 byte log entry
0709 13:18:37.067216 (+ 14829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.067289 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:37.067329 (+    40us) raft_consensus.cc:1685] finished
0709 13:18:37.067339 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:37.067461 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15628,"prepare.queue_time_us":91183,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":26303,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":1656,"thread_start_us":481128,"threads_started":2,"wal-append.queue_time_us":425515}
I20260709 13:18:37.223009 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:37.224695 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:37.225330 12631 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:37.009984 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:18:37.020463 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 250358us
I20260709 13:18:37.226468 12775 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:37.226998 12672 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:37.076974 12782 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 4. Current term is 5. Ops: []
W20260709 13:18:37.148164 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.075335 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:18:37.080030 15209 log.cc:927] Time spent T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.441s	user 0.001s	sys 0.001s
I20260709 13:18:37.232489 12672 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:37.233322 12672 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:37.054992 15079 raft_consensus.cc:3055] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:36.950341 15013 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:18:37.095402 15215 log.cc:927] Time spent T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:18:36.953305 15197 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:37.219686 12747 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:37.224326 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 435150us
I20260709 13:18:37.234584 12672 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:37.228592 12783 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:18:37.228811 12598 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:37.237759 13349 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:36.958313 15069 raft_consensus.cc:740] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:37.335175 12783 pending_rounds.cc:85] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 3
I20260709 13:18:37.000412 15152 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.205689 14479 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:18:37.257972 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.241884 13380 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:37.247793 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:18:37.336653 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 109148us
I20260709 13:18:37.349463 12775 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:37.349812 13349 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:37.337651 12783 pending_rounds.cc:107] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:18:37.339953 15185 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:37.351060 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:37.351203 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:37.009595 15165 consensus_queue.cc:1059] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.331s
I20260709 13:18:37.354008 12778 raft_consensus.cc:2393] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 6: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:37.220125 14956 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:37.340986 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 106460us
I20260709 13:18:37.333650 15179 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:37.071262 14613 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:37.344194 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:37.351770 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:37.239789 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:37.350647 12783 raft_consensus.cc:2889] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:37.374960 12782 raft_consensus.cc:2393] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:37.241477 15072 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:37.227933 14941 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.234190 14680 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:18:37.338296 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 109542us
I20260709 13:18:37.382140 12705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 240) took 1545 ms. Trace:
I20260709 13:18:37.334038 15137 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:36.981264 14980 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:37.340457 12673 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:37.383245 12705 rpcz_store.cc:276] 0709 13:18:35.837183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.507768 (+670585us) service_pool.cc:224] Handling call
0709 13:18:36.508155 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:36.578453 (+ 70298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.381891 (+803438us) spinlock_profiling.cc:243] Waited 69.4 ms on lock 0x7b54001b1554. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:37.382078 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":159625600}
I20260709 13:18:36.949612 15009 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:37.359392 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:37.341912 15153 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:37.335228 15013 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:37.365592 15175 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:37.366737 15038 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:37.332044 15025 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:37.359478 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:37.227711 15190 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:37.241219 15117 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:37.354914 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:37.336277 15079 raft_consensus.cc:740] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:37.401639 15079 consensus_queue.cc:260] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:37.353420 15152 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:37.403995 15152 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:37.356307 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:37.373538 12629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1193) took 1523 ms. Trace:
I20260709 13:18:37.222116 14974 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:37.134223 15178 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:37.393712 15013 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:37.351989 15197 raft_consensus.cc:697] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:37.238679 15141 consensus_queue.cc:1059] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.132s
I20260709 13:18:37.391829 15175 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:37.387292 12598 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:37.446511 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 59306.2us
W20260709 13:18:37.386821 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 46524.1us
I20260709 13:18:37.376484 14479 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:37.392443 13349 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:37.390643 15077 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:37.454090 15077 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:37.454703 15077 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:37.395124 12778 raft_consensus.cc:2410] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:18:37.214984 15115 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:37.387254 15179 raft_consensus.cc:697] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:37.467223 15115 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:37.466797 15179 consensus_queue.cc:237] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:37.468072 15232 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:37.468503 15232 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:37.468863 15232 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:37.391115 15170 consensus_queue.cc:1059] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:18:37.399076 12703 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:37.399623 12628 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
W20260709 13:18:37.399732 15136 consensus_meta.cc:287] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.535s	user 0.011s	sys 0.000s
I20260709 13:18:37.380132 14941 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:37.400332 15026 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:37.403688 15116 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:37.470862 15136 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:37.472698 15116 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:37.472826 14941 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:37.373785 12633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56832 (request call id 2032) took 1264 ms. Trace:
I20260709 13:18:37.344149 14820 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:37.473454 12633 rpcz_store.cc:276] 0709 13:18:36.109328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.139039 (+ 29711us) service_pool.cc:224] Handling call
0709 13:18:37.373622 (+1234583us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b54002ce054. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:37.373729 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1286260480}
I20260709 13:18:37.473961 14820 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:37.474452 14820 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:37.476141 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 83778.2us
I20260709 13:18:37.476492 15232 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:37.405190 12629 rpcz_store.cc:276] 0709 13:18:35.850108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.852914 (+  2806us) service_pool.cc:224] Handling call
0709 13:18:36.131656 (+278742us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:36.172413 (+ 40757us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:36.172445 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:36.172910 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 13:18:36.382716 (+209806us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:36.382755 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.382826 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:36.425179 (+ 42353us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b4800002958. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c722dbaa1 00007f4c722d2b37 00007f4c722d29ca 00007f4c723ca26f 00007f4c723c9f37 00007f4c723de55c 00007f4c723de508 00007f4c723de488 00007f4c723de443 00007f4c723dd6ec 00007f4c71307a96
0709 13:18:37.079311 (+654132us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:18:37.373312 (+294001us) raft_consensus.cc:1685] finished
0709 13:18:37.373335 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:37.373475 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6239,"raft.queue_time_us":13301,"raft.run_cpu_time_us":2975,"raft.run_wall_time_us":252356,"spinlock_wait_cycles":6067712,"thread_start_us":209452,"threads_started":1,"wal-append.queue_time_us":346040}
I20260709 13:18:37.477314 13349 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:37.380088 14613 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:37.393489 15038 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:37.478056 15038 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:37.478307 14613 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:37.478770 14613 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:37.479148 14613 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:37.479627 13349 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:37.480511 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:37.394434 15137 raft_consensus.cc:697] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:37.473110 15136 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:37.505378 15220 log.cc:927] Time spent T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:18:37.485685 12598 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:37.395442 14680 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:37.481813 14479 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:37.561247 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:37.649775 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 201048us
W20260709 13:18:37.650753 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.406199 15190 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:37.438074 14974 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:37.394797 15159 consensus_queue.cc:1059] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
W20260709 13:18:37.613340 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 393684us
I20260709 13:18:37.486516 15165 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:37.509690 15116 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:37.414094 15117 raft_consensus.cc:697] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:37.648723 13384 raft_consensus.cc:2393] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:37.481602 15175 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:37.654397 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 168829us
W20260709 13:18:37.649770 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 302474us
I20260709 13:18:37.651818 12673 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:37.522557 15219 consensus_queue.cc:1059] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.159s
I20260709 13:18:37.658978 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:37.655025 14941 raft_consensus.cc:3055] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:37.667477 12673 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:37.652515 14479 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:37.661325 15095 consensus_meta.cc:287] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.729s	user 0.020s	sys 0.003s
I20260709 13:18:37.668578 14941 raft_consensus.cc:740] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:37.674064 12777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 261) took 1742 ms. Trace:
I20260709 13:18:37.674445 15137 consensus_queue.cc:237] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:37.675591 13376 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:37.678320 15202 log.cc:927] Time spent T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 1.396s	user 0.005s	sys 0.000s
I20260709 13:18:37.653098 15170 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:37.679409 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:37.681941 12747 leader_election.cc:400] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7.
I20260709 13:18:37.683982 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:37.658161 15148 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:37.681401 12777 rpcz_store.cc:276] 0709 13:18:35.931737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.232589 (+300852us) service_pool.cc:224] Handling call
0709 13:18:36.232931 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:36.577801 (+344870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.673818 (+1096017us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b540008a554. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:37.673999 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":791501696}
I20260709 13:18:37.660928 15165 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:37.662335 15163 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:37.690380 12747 leader_election.cc:403] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:37.690650 12634 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 2.
I20260709 13:18:37.668315 15014 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:37.686048 14941 consensus_queue.cc:260] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:37.697737 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:37.678975 14939 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:37.704095 12629 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:37.663419 15077 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:37.706840 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:37.709228 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:37.686780 15192 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.711668 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:37.722602 12633 raft_consensus.cc:2393] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 6: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:37.712788 12632 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:37.684252 15116 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:37.686388 15235 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:37.715219 12700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1254) took 1815 ms. Trace:
I20260709 13:18:37.721545 13385 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:37.700373 15165 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:37.715611 15178 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:37.714957 14939 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:37.729812 12700 rpcz_store.cc:276] 0709 13:18:35.899896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.513017 (+613121us) service_pool.cc:224] Handling call
0709 13:18:36.513430 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:36.522043 (+  8613us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:36.522072 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:36.573432 (+ 51360us) log.cc:844] Serialized 26 byte log entry
0709 13:18:37.050699 (+477267us) spinlock_profiling.cc:243] Waited 6.96 ms on lock 0x7b18004350a0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf520d1 00007f4c6d36da0f
0709 13:18:37.341389 (+290690us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:37.341445 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.341548 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:37.698406 (+356858us) raft_consensus.cc:1685] finished
0709 13:18:37.698431 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:37.715120 (+ 16689us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15049,"raft.queue_time_us":8666,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"spinlock_wait_cycles":16002304,"thread_start_us":290452,"threads_started":1,"wal-append.queue_time_us":816459}
I20260709 13:18:37.696638 14820 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:37.737675 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 65916.1us
W20260709 13:18:37.742583 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 262117us
I20260709 13:18:37.742750 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:37.746455 12629 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:37.746692 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:37.741178 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:37.747434 12631 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:37.687451 15095 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:37.749217 12630 raft_consensus.cc:2393] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 7: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:37.749437 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:37.699270 15170 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:37.721376 15014 raft_consensus.cc:697] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:37.752827 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:37.754446 13349 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:37.759066 12632 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:37.780220 12629 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:18:37.725028 14980 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:37.684114 15197 consensus_queue.cc:237] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:37.713254 15163 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:37.798635 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:37.727097 15235 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:37.799744 14934 consensus_meta.cc:287] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.583s	user 0.011s	sys 0.000s
I20260709 13:18:37.756636 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:37.727831 15232 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:37.804111 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 47315.2us
I20260709 13:18:37.732112 15192 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:37.804374 12673 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:37.805876 12632 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:18:37.820667 12747 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:37.741828 15009 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:37.751381 14939 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:37.801389 12631 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:37.802795 15178 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:37.742158 15234 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7.
I20260709 13:18:37.725335 15117 consensus_queue.cc:237] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:37.813689 15241 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:37.849521 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 46031.7us
I20260709 13:18:37.804520 14682 consensus_queue.cc:1059] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:18:37.820605 15099 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.855594 12632 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:37.858105 12631 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
W20260709 13:18:37.858669 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 45618.8us
I20260709 13:18:37.852795 12673 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:37.815665 15235 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:37.843130 14847 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:37.845782 15178 raft_consensus.cc:697] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:37.816222 14934 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:37.845979 14974 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:37.862463 15155 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:37.854238 15161 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:37.857796 14680 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:37.889823 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 69253.8us
I20260709 13:18:37.850193 15232 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:37.860088 15099 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:37.867043 15236 log.cc:927] Time spent T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:18:37.852744 15224 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:37.862087 15190 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:37.909436 14891 consensus_meta.cc:287] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 1.067s	user 0.012s	sys 0.000s
I20260709 13:18:37.857394 15241 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:37.910784 15163 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:37.932448 12747 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:37.824816 14956 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:37.959829 15192 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:18:37.960227 15163 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:37.960876 15163 raft_consensus.cc:697] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:37.961620 14956 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:37.882190 14847 raft_consensus.cc:697] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:37.961881 15163 consensus_queue.cc:237] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:37.962839 14847 consensus_queue.cc:237] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:37.883988 15155 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:37.933329 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:37.912875 15232 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:37.948901 15116 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:37.878904 15243 consensus_queue.cc:1059] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.085s
I20260709 13:18:37.952342 13383 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:37.954033 15178 consensus_queue.cc:237] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:37.955510 15241 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:37.957134 15099 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:37.965785 15155 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:37.852952 15014 consensus_queue.cc:237] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:37.969776 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 215368us
W20260709 13:18:37.974958 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 113656us
I20260709 13:18:37.944057 15077 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:37.981424 12747 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:37.943060 15095 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:37.912019 14613 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:37.992199 12747 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:37.992877 13349 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:37.994696 13349 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:37.977799 15232 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:37.980443 15242 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.997028 12747 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:37.990451 15038 consensus_meta.cc:287] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.000s
I20260709 13:18:38.005885 13379 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:37.991971 14820 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:37.993824 15026 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:38.002449 13381 raft_consensus.cc:2410] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:18:37.998021 14682 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:37.999157 15116 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:38.028630 14956 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:37.996070 15152 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.037s
W20260709 13:18:38.043478 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 46524.8us
I20260709 13:18:38.008504 15254 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:38.001948 15170 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:38.042171 15116 raft_consensus.cc:697] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:38.044821 12747 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:38.043383 14956 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.006235 15038 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:38.033198 15247 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:38.044790 15251 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:38.073278 15116 consensus_queue.cc:237] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.003309 15242 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.084376 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:38.082448 14934 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:38.085702 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:18:38.087687 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 729387us
I20260709 13:18:38.062301 14682 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:38.077226 14939 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:38.110883 12672 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:38.115129 13381 raft_consensus.cc:2376] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7. Re-sending same reply.
I20260709 13:18:38.115716 15153 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:38.116354 15153 raft_consensus.cc:697] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:38.116865 13381 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:38.118299 14613 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:38.120179 15152 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:38.079974 15254 raft_consensus.cc:697] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:38.086179 14479 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:38.122637 15254 consensus_queue.cc:237] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.124197 14479 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:38.118090 15099 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:38.091137 15247 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:38.125707 14682 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:38.127123 15247 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:38.128012 14682 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:38.128489 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:38.129057 12700 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:38.109030 15026 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:38.129938 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:38.115710 12672 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:38.129946 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:38.136058 12672 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:38.139271 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:38.139837 12706 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:38.140030 12705 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:38.141498 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:38.141808 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:38.143611 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:38.145140 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:38.142194 12705 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:38.145825 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:38.064730 15175 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.146155 12708 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:18:38.146234 12700 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:18:38.153457 12780 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 8: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:38.157356 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:38.159488 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:38.160717 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:38.161290 12777 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:38.162539 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:38.140923 15077 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:18:38.154829 15259 log.cc:927] Time spent T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.078s	user 0.004s	sys 0.000s
I20260709 13:18:38.168301 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:38.169075 13385 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:38.165597 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:38.085103 15235 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:38.178043 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:38.153383 15247 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:38.174180 12783 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
I20260709 13:18:38.187836 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:38.179185 12777 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
I20260709 13:18:38.131742 15069 consensus_queue.cc:260] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.181303 13381 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:38.182454 13377 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:38.187171 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:38.117266 15153 consensus_queue.cc:237] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:38.189847 12747 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:38.173616 14934 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:18:38.220016 12708 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:38.152691 14613 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:18:38.221825 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:38.224751 12747 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:38.235255 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:38.225086 13377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 477) took 1733 ms. Trace:
I20260709 13:18:38.225360 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:38.236742 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:38.237190 15235 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:38.238067 13380 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:18:38.228377 12625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2040) took 1727 ms. Trace:
W20260709 13:18:38.238705 15185 consensus_meta.cc:287] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.888s	user 0.012s	sys 0.000s
I20260709 13:18:38.225383 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:38.238976 12625 rpcz_store.cc:276] 0709 13:18:36.504568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.932545 (+427977us) service_pool.cc:224] Handling call
0709 13:18:37.649351 (+716806us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:37.649513 (+   162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:37.649538 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:37.649594 (+    56us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:37.649626 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.649684 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:38.228202 (+578518us) spinlock_profiling.cc:243] Waited 716 ms on lock 0x7b54000df550. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:38.228322 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1647816064}
I20260709 13:18:38.240645 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:38.231711 15175 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:38.234293 13384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51762 (request call id 496) took 1008 ms. Trace:
I20260709 13:18:38.244122 13384 rpcz_store.cc:276] 0709 13:18:37.226418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.344381 (+117963us) service_pool.cc:224] Handling call
0709 13:18:38.234031 (+889650us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b5400690854. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:38.234218 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":297037696}
W20260709 13:18:38.245020 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 113922us
I20260709 13:18:38.239202 15025 raft_consensus.cc:3055] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:38.245608 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:38.224550 15069 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:38.262912 12780 raft_consensus.cc:1275] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:18:38.264827 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:38.266435 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:38.267366 15038 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:38.225538 15255 log.cc:927] Time spent T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:18:38.283485 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:38.226363 15256 log.cc:927] Time spent T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 13:18:38.239143 15185 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:38.226230 15251 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:18:38.226783 13376 consensus_meta.cc:287] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.526s	user 0.011s	sys 0.000s
I20260709 13:18:38.285816 12672 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:38.287281 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:38.283512 13348 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:38.289664 15235 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:38.245959 15155 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.291395 12708 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:38.267848 15025 raft_consensus.cc:740] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:38.292332 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:38.236743 13377 rpcz_store.cc:276] 0709 13:18:36.491627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.507322 (+ 15695us) service_pool.cc:224] Handling call
0709 13:18:38.182033 (+1674711us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b5800095198. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c7238bfc0 00007f4c72376929 00007f4c723763d2 00007f4c6b8edbd4
0709 13:18:38.182267 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:38.222829 (+ 40562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:38.225018 (+  2189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54577024}
I20260709 13:18:38.186921 15261 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.117s
I20260709 13:18:38.284683 15178 consensus_queue.cc:1059] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:18:38.243680 14682 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.297729 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:38.273859 14934 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:38.291930 13378 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:38.259888 15077 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:38.315725 15077 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:38.245478 15265 consensus_queue.cc:1059] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.066s
I20260709 13:18:38.318148 12709 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:38.317269 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:18:38.304082 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 57483.9us
I20260709 13:18:38.304106 13348 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:38.305593 12778 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:38.319556 12672 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:38.300102 15222 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.008s	sys 0.017s
I20260709 13:18:38.311612 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:38.335253 12780 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:38.301783 15185 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:38.311514 14682 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:38.307598 15253 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:38.336148 15178 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:38.312721 15232 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.346238 15178 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.346700 15178 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:38.347479 14682 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:38.347642 15232 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:38.301741 15038 consensus_queue.cc:1059] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:18:38.313599 15155 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:38.314373 14939 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:38.314478 12776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 1938) took 1079 ms. Trace:
W20260709 13:18:38.314837 15165 consensus_meta.cc:287] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.563s	user 0.008s	sys 0.003s
I20260709 13:18:38.349902 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:38.350250 15165 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:38.350889 14956 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:38.309016 15235 raft_consensus.cc:697] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:38.303273 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:38.300560 13376 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:38.334040 12626 raft_consensus.cc:606] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Received request to transfer leadership to TS 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:38.336447 15222 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.008s	sys 0.018s
W20260709 13:18:38.344774 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 48788.1us
I20260709 13:18:38.344862 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:38.353590 12626 raft_consensus.cc:614] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:38.349272 15077 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:38.352705 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:38.354275 12627 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:38.354735 12775 raft_consensus.cc:2393] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:38.333868 15136 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:38.349731 12776 rpcz_store.cc:276] 0709 13:18:37.235532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.235641 (+   109us) service_pool.cc:224] Handling call
0709 13:18:37.342248 (+106607us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:37.649287 (+307039us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:37.649319 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:37.662120 (+ 12801us) log.cc:844] Serialized 26 byte log entry
0709 13:18:37.982998 (+320878us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:37.995473 (+ 12475us) log.cc:844] Serialized 14 byte log entry
0709 13:18:38.077914 (+ 82441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:38.077995 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:38.314264 (+236269us) raft_consensus.cc:1685] finished
0709 13:18:38.314283 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:38.314404 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6809,"prepare.queue_time_us":12211,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":1541,"raft.queue_time_us":71852,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":3009,"thread_start_us":566448,"threads_started":3,"wal-append.queue_time_us":370494}
I20260709 13:18:38.356662 13348 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:38.356920 14939 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:38.334569 14934 raft_consensus.cc:697] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:38.356264 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:38.358052 15013 consensus_meta.cc:287] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.874s	user 0.012s	sys 0.000s
I20260709 13:18:38.358750 15266 raft_consensus.cc:3055] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:38.361825 12746 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:38.367950 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:38.368417 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:38.369877 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:38.368861 12709 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:38.375260 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:38.373234 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:38.374354 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:38.379221 12709 raft_consensus.cc:2393] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 6: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
W20260709 13:18:38.381381 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 85051.2us
I20260709 13:18:38.382572 12706 raft_consensus.cc:2393] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 4: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:38.383970 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:38.385900 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:38.386134 13348 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:38.388491 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:38.391485 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:38.392165 12780 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
I20260709 13:18:38.393230 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:38.394078 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:38.394686 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:38.395190 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:38.395991 12782 raft_consensus.cc:2393] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 6: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:38.365986 15185 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:38.367753 15232 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:38.392025 15277 log.cc:927] Time spent T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:18:38.397344 12704 raft_consensus.cc:2393] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 7: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:38.395278 12775 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:38.393291 15199 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:38.399217 15176 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:38.397395 12701 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 7. Current term is 8. Ops: [5.7-7.8]
I20260709 13:18:38.399943 13348 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:38.383936 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:38.381716 15152 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:38.365887 15266 raft_consensus.cc:740] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:38.413110 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:38.402112 15038 consensus_queue.cc:1059] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:18:38.382247 14613 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.405750 15141 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.374408 14939 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:38.407612 15165 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:38.367164 15013 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:38.427940 15013 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:38.410089 15176 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.428606 15176 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:38.403291 14934 consensus_queue.cc:237] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.420246 15152 raft_consensus.cc:697] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:38.430895 15152 consensus_queue.cc:237] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.433804 12634 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:18:38.414951 15185 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:38.434072 15266 consensus_queue.cc:260] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.371845 15136 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:18:38.435180 15266 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:38.423974 14613 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.436323 12707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1258) took 2382 ms. Trace:
I20260709 13:18:38.436694 12707 rpcz_store.cc:276] 0709 13:18:36.053869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.522243 (+468374us) service_pool.cc:224] Handling call
0709 13:18:36.577598 (+ 55355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:36.801202 (+223604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:38.436086 (+1634884us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b5400248bd4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:38.436252 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":512389504}
I20260709 13:18:38.437289 14613 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:38.438872 12781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 1937) took 1204 ms. Trace:
I20260709 13:18:38.440623 12708 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:18:38.443073 13380 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:38.439348 12781 rpcz_store.cc:276] 0709 13:18:37.234902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.235254 (+   352us) service_pool.cc:224] Handling call
0709 13:18:37.344683 (+109429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:37.667171 (+322488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:37.667220 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:37.667294 (+    74us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:37.667344 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:38.438578 (+771234us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b5400089b54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:38.438637 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:38.438793 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":741091200}
I20260709 13:18:38.410133 15265 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:18:38.409454 15232 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:38.483363 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 819450us
I20260709 13:18:38.449198 15281 raft_consensus.cc:3055] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:38.548475 15239 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:38.441746 15141 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.549043 15239 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.550998 15239 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:38.551251 15141 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:38.553150 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:38.554016 12625 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:38.556596 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:18:38.558650 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:38.559190 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.539944 15115 raft_consensus.cc:3055] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:18:38.541368 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.403823 15235 consensus_queue.cc:237] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:38.567025 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:38.568109 12632 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:38.569190 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:38.540503 12598 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:38.565371 12629 raft_consensus.cc:1275] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:18:38.569831 12632 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:38.574465 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:38.575500 12631 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 6: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:38.542997 15281 raft_consensus.cc:740] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:38.583246 12632 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:38.547109 15072 raft_consensus.cc:3055] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:18:38.606704 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 66320.1us
I20260709 13:18:38.611975 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:38.572283 15095 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:38.573809 15115 raft_consensus.cc:740] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:38.618939 12597 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:38.575973 14820 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.590698 15072 raft_consensus.cc:740] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:38.611819 15013 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:38.651808 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:38.652559 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:38.656514 12629 raft_consensus.cc:2410] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:18:38.652681 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:38.657205 12632 raft_consensus.cc:2393] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 7: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
W20260709 13:18:38.652892 12628 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39590 (request call id 278) took 1709 ms (client timeout 1987 ms). Trace:
I20260709 13:18:38.658087 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:38.620776 15279 log.cc:927] Time spent T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:18:38.653214 12625 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:38.663194 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:38.664047 12625 raft_consensus.cc:2410] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:18:38.663959 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:18:38.658607 13349 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.65:43603 timed out after 1.987s (SENT) [suppressed 15 similar messages]
I20260709 13:18:38.664924 12629 raft_consensus.cc:2393] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 6: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:38.659276 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:38.658327 12628 rpcz_store.cc:269] 0709 13:18:36.943682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.003698 (+ 60016us) service_pool.cc:224] Handling call
0709 13:18:38.652671 (+1648973us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b54002eaed4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:38.652805 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":427678592}
I20260709 13:18:38.661654 12703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1262) took 2564 ms. Trace:
I20260709 13:18:38.654407 12630 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:38.665421 12632 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:38.590126 15178 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.665841 12703 rpcz_store.cc:276] 0709 13:18:36.086870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.523534 (+436664us) service_pool.cc:224] Handling call
0709 13:18:36.523881 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:37.470065 (+946184us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:38.651235 (+1181170us) spinlock_profiling.cc:243] Waited 875 ms on lock 0x7b54001cd9d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:38.651471 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2012283776}
I20260709 13:18:38.694195 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:38.685871 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:38.697412 12631 raft_consensus.cc:2393] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 7: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:38.700273 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:38.700585 12632 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:18:38.701980 12632 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:38.704905 12625 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:38.702905 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:38.700219 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:38.711997 12630 raft_consensus.cc:2368] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7.
I20260709 13:18:38.714027 15072 consensus_queue.cc:260] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.715131 15072 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:38.715651 15115 consensus_queue.cc:260] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.716862 15115 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:38.630741 15159 consensus_queue.cc:1059] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
W20260709 13:18:38.715248 15280 log.cc:927] Time spent T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 13:18:38.720029 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:38.720863 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:38.667377 15120 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:38.722714 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:38.667734 15199 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:38.626456 15095 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:38.732859 15095 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:38.623567 15176 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.725461 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:38.730248 15120 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.712330 15161 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:38.729454 15278 log.cc:927] Time spent T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.369s	user 0.003s	sys 0.001s
I20260709 13:18:38.739708 15161 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:38.741467 15284 log.cc:927] Time spent T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:18:38.742609 15161 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:38.742766 13383 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51768 (request call id 428) took 2159 ms (client timeout 1722 ms). Trace:
I20260709 13:18:38.732312 15281 consensus_queue.cc:260] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.743628 15281 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:38.744575 12627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56838 (request call id 1220) took 1977 ms (client timeout 2379 ms). Trace:
I20260709 13:18:38.739670 15120 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:38.745069 12627 rpcz_store.cc:269] 0709 13:18:36.767404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.802518 (+ 35114us) service_pool.cc:224] Handling call
0709 13:18:38.744393 (+1941875us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b54002feed4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:38.744511 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3566108928}
I20260709 13:18:38.751704 12625 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
W20260709 13:18:38.743275 13383 rpcz_store.cc:269] 0709 13:18:36.583667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.620772 (+ 37105us) service_pool.cc:224] Handling call
0709 13:18:38.742520 (+2121748us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b540047f254. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:38.742630 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3028397824}
I20260709 13:18:38.736960 14941 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:38.753715 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:38.754504 13384 raft_consensus.cc:2368] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 5. Current term is 6.
I20260709 13:18:38.755290 12626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:33786 (request call id 20) took 2685 ms. Trace:
I20260709 13:18:38.755712 12626 rpcz_store.cc:276] 0709 13:18:36.070028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.366058 (+296030us) service_pool.cc:224] Handling call
0709 13:18:38.755122 (+2389064us) spinlock_profiling.cc:243] Waited 1.97 s on lock 0x7b54002feed4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3ac853 00007f4c7237a60a 00007f4c6b9007d4 00007f4c6b900762 00007f4c6b900691 00007f4c6b90060c 00007f4c6b8ff8c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:38.755235 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4523832576}
I20260709 13:18:38.756270 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:38.757488 13379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 431) took 1037 ms. Trace:
I20260709 13:18:38.757956 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:38.757875 13379 rpcz_store.cc:276] 0709 13:18:37.719910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.720064 (+   154us) service_pool.cc:224] Handling call
0709 13:18:37.720388 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:38.006340 (+285952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:38.757283 (+750943us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b540026cad4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:38.757435 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":656128384}
I20260709 13:18:38.758579 13384 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:38.777806 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:38.723875 15013 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:18:38.778692 13384 raft_consensus.cc:2368] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8.
I20260709 13:18:38.786523 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:38.787391 12626 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:38.790938 13383 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:38.791325 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:38.792454 13384 raft_consensus.cc:2393] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:38.795463 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:38.795888 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:38.657466 14820 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.806324 15199 raft_consensus.cc:697] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:38.814656 15199 consensus_queue.cc:237] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:38.817752 15095 raft_consensus.cc:3055] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:38.818222 15095 raft_consensus.cc:740] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:38.818991 15095 consensus_queue.cc:260] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:38.820386 15095 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:38.822395 15294 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:38.823104 15294 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.823833 15175 raft_consensus.cc:3055] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:38.824221 15175 raft_consensus.cc:740] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:38.804890 15290 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.825747 15294 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:38.826067 15293 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:38.825834 15290 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:38.826594 15293 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.828123 15290 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:38.830116 12627 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7. Ops: []
W20260709 13:18:38.832178 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 442482us
I20260709 13:18:38.834951 12672 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:38.838665 14479 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:38.839120 14479 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.839510 14479 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:38.840144 12626 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:18:38.825083 15175 consensus_queue.cc:260] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.836367 15183 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.845088 12625 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:38.855006 15232 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:38.865731 15292 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.866844 14479 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.869369 14479 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:38.872711 14820 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:38.850322 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 61356.3us
I20260709 13:18:38.846410 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:38.874387 12597 leader_election.cc:400] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 5. Current term is 6.
I20260709 13:18:38.874802 12597 leader_election.cc:403] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:38.866514 15292 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:38.861115 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.879750 12597 leader_election.cc:400] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8.
I20260709 13:18:38.880673 12597 leader_election.cc:403] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:38.863137 15175 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:38.900993 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:38.884791 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:38.886793 12631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:18:38.904006 12631 raft_consensus.cc:606] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Received request to transfer leadership to TS 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:38.904291 12631 raft_consensus.cc:614] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:38.878736 14820 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:18:38.884213 12672 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:38.880201 14941 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:38.903168 13380 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:38.907147 12597 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:38.862943 15183 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.883368 15296 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:18:38.915278 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:38.915778 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:38.909269 15222 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:38.919399 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:38.920256 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:38.922823 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:38.905237 15293 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:38.923427 12701 raft_consensus.cc:2376] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6. Re-sending same reply.
I20260709 13:18:38.920537 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:38.924597 12700 raft_consensus.cc:2410] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:18:38.926812 15292 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:38.883049 15166 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:18:38.921480 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:38.910333 15170 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:38.928479 12705 raft_consensus.cc:2393] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 6: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:38.929883 12703 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 6: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:38.930626 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:38.931870 12708 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:38.933022 12700 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:38.930864 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:38.934165 12701 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:38.935328 12631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:18:38.935904 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:38.935914 12631 raft_consensus.cc:606] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Received request to transfer leadership to TS 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:38.942528 12633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 279) took 1194 ms. Trace:
I20260709 13:18:38.942759 12631 raft_consensus.cc:614] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:38.946303 15240 raft_consensus.cc:3055] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:38.946792 15240 raft_consensus.cc:740] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:38.946961 15183 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:38.947433 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:18:38.947711 15240 consensus_queue.cc:260] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:38.904788 15199 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:18:38.943025 12633 rpcz_store.cc:276] 0709 13:18:37.747967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.748495 (+   528us) service_pool.cc:224] Handling call
0709 13:18:37.748775 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:37.811745 (+ 62970us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:37.820132 (+  8387us) log.cc:844] Serialized 14 byte log entry
0709 13:18:38.666465 (+846333us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:38.666522 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:38.666571 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:38.942235 (+275664us) spinlock_profiling.cc:243] Waited 62.7 ms on lock 0x7b54000ddc54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:38.942306 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:38.942465 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":570716,"prepare.queue_time_us":10945,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":37,"raft.queue_time_us":586051,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":10759,"spinlock_wait_cycles":144201600,"thread_start_us":275144,"threads_started":2,"wal-append.queue_time_us":212226}
I20260709 13:18:38.952023 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:38.922519 15222 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:38.952296 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:38.954866 12703 raft_consensus.cc:1240] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:38.934718 15170 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.956858 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:38.956632 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:38.956682 12703 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:38.949041 15240 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:38.958613 12705 raft_consensus.cc:2393] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 7: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:38.958535 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:38.958760 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:38.959352 12703 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 3.
I20260709 13:18:38.960258 12700 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:38.960637 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:38.961526 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:38.962157 12700 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 3.
I20260709 13:18:38.963033 12672 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:38.963840 15166 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:38.973956 12598 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:38.955425 15234 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.881s	user 0.015s	sys 0.000s
I20260709 13:18:38.969329 14820 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 5. Current term is 6.
I20260709 13:18:38.978804 12598 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:38.956101 15222 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:38.982180 12598 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:38.982682 12701 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:38.987648 12598 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:38.970057 15283 log.cc:927] Time spent T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.313s	user 0.001s	sys 0.002s
I20260709 13:18:38.958297 15170 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:38.989108 12598 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:38.970377 15296 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8.
I20260709 13:18:38.979759 15148 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:38.969491 15025 consensus_queue.cc:260] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:39.010815 15025 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:39.011471 15242 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:39.014674 13376 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51762 (request call id 494) took 1789 ms (client timeout 1551 ms). Trace:
W20260709 13:18:39.015169 13376 rpcz_store.cc:269] 0709 13:18:37.225424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.227825 (+  2401us) service_pool.cc:224] Handling call
0709 13:18:39.014413 (+1786588us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b540051fc54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:39.014573 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":620882176}
I20260709 13:18:38.986058 15292 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:18:39.023171 12779 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.140s	user 0.047s	sys 0.051s
W20260709 13:18:38.982514 15234 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.908s	user 0.016s	sys 0.000s
I20260709 13:18:38.990284 15293 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:38.994256 15199 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:39.018967 13381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 439) took 1219 ms. Trace:
I20260709 13:18:39.020793 13385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 436) took 1292 ms. Trace:
I20260709 13:18:39.021988 13378 raft_consensus.cc:1240] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:39.009614 14820 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:39.026101 13385 rpcz_store.cc:276] 0709 13:18:37.728497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.746419 (+ 17922us) service_pool.cc:224] Handling call
0709 13:18:37.746695 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:38.177044 (+430349us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:39.020570 (+843526us) spinlock_profiling.cc:243] Waited 422 ms on lock 0x7b54000596d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:39.020729 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":970869632}
W20260709 13:18:39.027606 12779 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.144s	user 0.048s	sys 0.051s
I20260709 13:18:39.029541 12598 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:39.030009 15137 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.030553 15137 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:39.031853 15293 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:39.032320 15199 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:39.032912 15199 raft_consensus.cc:697] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:39.033818 12701 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:39.033854 15199 consensus_queue.cc:237] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:39.025871 13381 rpcz_store.cc:276] 0709 13:18:37.800117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.152185 (+352068us) service_pool.cc:224] Handling call
0709 13:18:38.152485 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:38.240637 (+ 88152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:39.018707 (+778070us) spinlock_profiling.cc:243] Waited 28.6 ms on lock 0x7b54004bfd54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:39.018908 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65725696}
I20260709 13:18:39.027568 14820 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.046602 15222 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:39.048362 15178 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:39.049099 12707 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:18:39.049470 14820 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:39.039477 15292 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
W20260709 13:18:39.050657 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.052182 15222 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:39.052981 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:39.055734 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:39.062378 12632 raft_consensus.cc:1275] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:39.062575 13382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 478) took 2570 ms. Trace:
I20260709 13:18:39.063449 12628 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:39.063598 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:39.054193 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:39.064252 12632 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:18:39.062387 12627 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:39.066501 15137 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:39.071043 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 459120us
I20260709 13:18:39.055176 12631 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:39.073837 14934 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:39.075055 15170 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:39.075531 15197 consensus_queue.cc:1175] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "522fd2ba0e9f402691b5ef29af8579cd" responder_term: 7 status { last_received { term: 6 index: 8 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:18:39.079697 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:39.080322 12774 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:39.055056 15176 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
W20260709 13:18:39.084725 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:39.085210 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.069769 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:39.086964 12700 raft_consensus.cc:2393] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 8: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:39.062690 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:39.087471 12630 raft_consensus.cc:2376] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6. Re-sending same reply.
I20260709 13:18:39.069923 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:39.088160 12629 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:39.089440 12776 raft_consensus.cc:1240] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:39.089848 12706 raft_consensus.cc:1275] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:39.091579 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:39.092320 12776 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:39.093080 12774 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:39.094203 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:39.094378 12630 raft_consensus.cc:1275] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:18:39.094700 12630 pending_rounds.cc:85] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 3
I20260709 13:18:39.095007 12630 pending_rounds.cc:107] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:18:39.095281 12630 raft_consensus.cc:2889] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:39.094801 13385 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 3.
I20260709 13:18:39.096388 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:39.096568 12777 raft_consensus.cc:1275] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:18:39.097020 13380 raft_consensus.cc:2376] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7. Re-sending same reply.
I20260709 13:18:39.097605 12634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2047) took 1396 ms. Trace:
I20260709 13:18:39.089349 12626 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:18:39.097965 12634 rpcz_store.cc:276] 0709 13:18:37.701270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:37.701795 (+   525us) service_pool.cc:224] Handling call
0709 13:18:37.702195 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:38.434249 (+732054us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:39.097365 (+663116us) spinlock_profiling.cc:243] Waited 731 ms on lock 0x7b540004a1d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:39.097536 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1681976704}
I20260709 13:18:39.102262 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:39.123931 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:39.124739 12781 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:39.106501 13349 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:39.107728 12707 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:39.108434 12703 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:39.127576 13349 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:39.128727 13349 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:39.106035 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:39.133100 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:39.133685 13384 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:39.118705 12629 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:39.064792 14820 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:39.137739 12778 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:39.135037 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:39.141788 12779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 1935) took 2434 ms. Trace:
I20260709 13:18:39.062934 13382 rpcz_store.cc:276] 0709 13:18:36.492048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.494606 (+  2558us) service_pool.cc:224] Handling call
0709 13:18:37.518470 (+1023864us) spinlock_profiling.cc:243] Waited 25.8 ms on lock 0x7b5800095198. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c7238bfc0 00007f4c72376929 00007f4c723763d2 00007f4c6b8edbd4
0709 13:18:37.649837 (+131367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:37.672422 (+ 22585us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:37.799294 (+126872us) log.cc:844] Serialized 14 byte log entry
0709 13:18:38.640992 (+841698us) spinlock_profiling.cc:243] Waited 4.94 ms on lock 0x7b180048cca0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6cf5a4bc 00007f4c6a2a848b 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c2455 00007f4c6d3aa060 00007f4c6d3bc716 00007f4c6d3cd287
0709 13:18:38.658855 (+ 17863us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:38.658934 (+    79us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:38.658999 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:39.062324 (+403325us) spinlock_profiling.cc:243] Waited 22.3 ms on lock 0x7b5400498c54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:39.062368 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:39.062490 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":208231,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":1541,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":7787,"spinlock_wait_cycles":122013440,"thread_start_us":233043,"threads_started":3,"wal-append.queue_time_us":70287,"wal-append.run_cpu_time_us":1313,"wal-append.run_wall_time_us":1039629}
I20260709 13:18:39.138792 15178 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:39.104705 13378 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:18:39.156461 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.162142 13349 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:39.162034 15116 consensus_queue.cc:1048] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:39.163043 15175 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:39.163431 15175 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:39.163777 15175 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:39.166481 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:39.167819 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 74147.4us
I20260709 13:18:39.171540 12672 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:39.172974 15254 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:39.173413 15254 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.173794 15254 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:39.099912 14613 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.177851 13349 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:39.178781 12781 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:39.180171 12626 raft_consensus.cc:1240] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:39.181543 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:39.184418 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:39.185036 12631 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:39.185458 14479 consensus_queue.cc:1059] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:39.185921 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:39.186690 12627 raft_consensus.cc:2393] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:39.185467 12673 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:39.187502 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:39.192195 12700 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:39.192404 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:39.194015 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:39.194579 12703 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:39.194916 13349 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:39.191752 12672 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:39.199230 13384 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:39.201176 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:39.202023 12628 raft_consensus.cc:2393] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 4: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:39.132954 15242 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:39.132856 15176 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:39.206322 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:39.207005 12707 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:39.201176 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:39.246711 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:39.237296 12625 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:39.247330 12776 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:39.261711 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 188274us
W20260709 13:18:39.262812 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 71179.6us
I20260709 13:18:39.259262 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:39.262910 14479 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:18:39.263335 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.259610 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:39.263777 14479 raft_consensus.cc:697] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:39.269480 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:39.270644 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:39.271732 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:39.272524 12626 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:39.273900 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:39.274885 12782 raft_consensus.cc:2393] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 5: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:39.277593 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:39.281448 15265 consensus_queue.cc:1059] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:39.281901 14479 consensus_queue.cc:237] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:39.243575 15232 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:18:39.207899 15166 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.198874 14613 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:39.284946 15166 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:39.286645 14613 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:39.286939 15038 consensus_queue.cc:1059] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:39.287567 15166 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:39.287786 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:39.243589 15009 consensus_queue.cc:1059] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:18:39.288390 12774 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:39.288822 15009 raft_consensus.cc:3055] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:39.289165 15009 raft_consensus.cc:740] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:39.289390 15266 raft_consensus.cc:3055] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:39.289825 15266 raft_consensus.cc:740] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:39.281697 15183 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:39.290032 15009 consensus_queue.cc:260] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:39.291054 15183 raft_consensus.cc:697] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:39.291419 15009 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:39.292023 15183 consensus_queue.cc:237] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:39.293326 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:39.293979 12783 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:39.285198 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:39.201846 15178 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:18:39.262483 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 77226.6us
I20260709 13:18:39.304445 12673 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:39.300184 15265 raft_consensus.cc:3055] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:39.287873 15038 consensus_queue.cc:1059] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 13:18:39.303589 12747 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:39.308480 12778 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 6: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:39.333884 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:39.334867 12778 raft_consensus.cc:2393] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 6: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:39.313412 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:39.198771 15252 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:39.201759 15185 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:39.344041 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:39.275480 15099 raft_consensus.cc:3055] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:39.339607 12776 raft_consensus.cc:2410] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:18:39.358325 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:39.359346 12629 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:39.360493 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:39.361438 12782 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:39.324437 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:39.245642 15294 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:39.347796 12673 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:39.349757 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:39.402132 12777 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:39.402539 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 54889.3us
I20260709 13:18:39.202309 15247 consensus_queue.cc:1048] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.128s
I20260709 13:18:39.352919 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:39.389330 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:39.394747 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.403978 12673 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:39.202540 15235 consensus_queue.cc:1059] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.135s
I20260709 13:18:39.322017 15176 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:18:39.424754 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 121236us
I20260709 13:18:39.331208 14745 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:39.250105 15281 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.109s
I20260709 13:18:39.321632 15265 raft_consensus.cc:740] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:39.246124 15137 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.105s
W20260709 13:18:39.368958 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 167913us
I20260709 13:18:39.350528 15252 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.312758 15232 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:39.440564 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:39.442436 12774 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:39.462803 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:39.463383 12707 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:39.463605 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:39.442957 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:39.244000 15298 consensus_queue.cc:1048] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.169s
I20260709 13:18:39.244175 15237 consensus_queue.cc:1059] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.160s
I20260709 13:18:39.358526 15099 raft_consensus.cc:740] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:39.451527 13349 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:39.296928 15175 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:39.457262 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:39.501998 12778 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:39.473644 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:39.504132 12778 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 13:18:39.504012 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 52586.9us
I20260709 13:18:39.464704 12782 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:39.469352 12776 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:39.450120 12626 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:39.356557 14941 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:39.510761 13349 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:18:39.519043 15307 log.cc:927] Time spent T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.217s	user 0.000s	sys 0.000s
I20260709 13:18:39.519697 15175 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:39.521176 12708 raft_consensus.cc:1275] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:39.444458 15176 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:39.528985 12777 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:39.528537 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:39.532898 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:39.533505 13384 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:39.472146 15027 raft_consensus.cc:3055] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:39.472371 15152 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:18:39.534704 15027 raft_consensus.cc:740] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:39.529469 15311 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:39.538204 15311 raft_consensus.cc:697] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 6 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:39.539768 12707 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:39.441426 15251 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:39.521303 12672 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:39.552263 12672 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:39.431866 15136 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:39.556864 15222 raft_consensus.cc:2764] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:18:39.558080 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:39.558809 12629 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:39.459167 15014 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:39.464835 15252 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:39.564653 12672 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:39.547667 13349 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:39.521914 15234 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:39.434173 15185 raft_consensus.cc:697] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:18:39.545217 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 71657.7us
I20260709 13:18:39.545481 12627 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:39.547426 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:39.589413 12778 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:39.527511 12783 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:18:39.522509 15237 consensus_queue.cc:1243] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 522fd2ba0e9f402691b5ef29af8579cd log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:18:39.595554 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:39.596426 12708 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:39.543717 15176 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:39.612287 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:39.603479 13349 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:39.435701 15294 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:39.607635 12777 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:39.608243 13384 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:39.541380 15178 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:18:39.562883 15280 log.cc:927] Time spent T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:18:39.611106 12597 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:39.617103 13349 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:39.563817 15038 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:39.468811 15166 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.130s
I20260709 13:18:39.619948 13379 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:18:39.572671 14980 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:39.623589 12597 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:39.506562 15232 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:39.541201 15242 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 13:18:39.601059 15222 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:39.634444 12597 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:39.636341 12708 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:39.589497 15298 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:18:39.633471 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 200427us
I20260709 13:18:39.611872 15170 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:39.576331 15014 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:39.651109 12673 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:39.653043 12673 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:39.631466 15293 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:39.583877 15175 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:18:39.636802 14613 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:39.631629 15312 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:39.677762 15312 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.678205 15312 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:39.638752 15197 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:39.636934 15178 raft_consensus.cc:697] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:39.643371 15298 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.645798 15038 raft_consensus.cc:697] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:39.655644 15222 raft_consensus.cc:697] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:39.641144 14980 raft_consensus.cc:697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:39.616593 15254 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:39.658149 15136 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:39.654227 15014 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:39.718977 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 130730us
I20260709 13:18:39.729153 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:39.683108 14820 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:39.683408 15137 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:39.670478 15170 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:39.539201 15311 consensus_queue.cc:237] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:39.626122 15099 consensus_queue.cc:260] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:39.734670 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.694841 15242 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.697970 15298 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:39.707353 14613 raft_consensus.cc:697] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:39.570912 15265 consensus_queue.cc:260] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:39.762753 13385 raft_consensus.cc:2393] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 4: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:39.763374 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:39.773797 12747 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:39.758867 14745 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:39.696815 15185 consensus_queue.cc:237] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:39.767781 15242 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:39.769626 15247 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:39.790668 15222 consensus_queue.cc:237] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:39.766544 15136 raft_consensus.cc:697] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:39.797729 15178 consensus_queue.cc:237] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:39.798115 14980 consensus_queue.cc:237] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:39.914806 12672 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:39.799602 15313 log.cc:927] Time spent T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:18:39.916437 15141 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:39.916844 15141 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.917167 15141 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:39.803432 15176 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:39.921691 15176 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:39.922639 15077 consensus_queue.cc:1059] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:39.924036 15077 raft_consensus.cc:3055] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:39.924383 15077 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:18:39.927553 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 274604us
W20260709 13:18:39.928377 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.930398 15322 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.930872 15322 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:39.933092 15322 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:39.804927 15009 consensus_meta.cc:287] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.513s	user 0.008s	sys 0.000s
I20260709 13:18:39.806250 15254 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:39.806355 15294 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:39.806915 15038 consensus_queue.cc:237] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:39.807993 15316 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:18:39.934420 15239 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.002s	sys 0.011s
I20260709 13:18:39.809967 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:39.935039 15316 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:39.810288 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:39.813783 15232 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:39.935457 12706 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:39.935408 15322 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:39.936189 12708 raft_consensus.cc:2393] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 4: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:39.820318 15252 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:39.938174 15316 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:39.938814 15252 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:39.826182 15235 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:18:39.827543 15240 consensus_meta.cc:287] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.853s	user 0.011s	sys 0.000s
I20260709 13:18:39.827935 15099 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:39.940761 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 167079us
I20260709 13:18:39.942255 12747 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:39.935093 15239 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.002s	sys 0.011s
I20260709 13:18:39.828464 15265 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:39.831207 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:39.807950 14613 consensus_queue.cc:237] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:39.776413 15320 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:39.833688 14479 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:39.836841 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 202454us
I20260709 13:18:39.954727 14479 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:39.958500 14479 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:39.974529 12597 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:39.976156 12597 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:39.977548 12597 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:39.881368 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:39.991720 12597 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:39.993516 12597 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:39.858567 15136 consensus_queue.cc:237] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:39.831199 14934 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 13:18:39.836544 15312 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:39.878747 15242 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:39.863634 15247 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:40.014132 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:18:40.014894 12528 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40528 (request call id 135) took 12954 ms (client timeout 15000 ms). Trace:
I20260709 13:18:40.015317 12631 raft_consensus.cc:2393] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 4: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:39.849207 15079 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:39.864552 15027 consensus_queue.cc:260] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:39.996011 15326 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:39.885190 15263 raft_consensus.cc:3055] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:40.034433 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:39.906946 15266 consensus_queue.cc:260] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:39.840457 15014 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:39.848146 15298 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:39.948567 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:40.026512 14820 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:39.953311 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:39.950824 15235 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:39.954173 15322 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.958338 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:39.925168 15077 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:39.955998 14847 consensus_queue.cc:1059] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:18:39.983940 15009 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:39.986698 15240 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:39.984129 15324 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:39.997190 12705 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:39.852870 12703 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:40.016793 12597 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:40.019637 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 91265.8us
I20260709 13:18:40.071177 12673 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:39.142160 12779 rpcz_store.cc:276] 0709 13:18:36.707865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.881431 (+173566us) service_pool.cc:224] Handling call
0709 13:18:36.881809 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:36.882198 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:36.882232 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:36.882917 (+   685us) log.cc:844] Serialized 26 byte log entry
0709 13:18:39.028035 (+2145118us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:39.031787 (+  3752us) log.cc:844] Serialized 14 byte log entry
0709 13:18:39.141461 (+109674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:39.141544 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:39.141612 (+    68us) raft_consensus.cc:1685] finished
0709 13:18:39.141623 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:39.141734 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36508,"prepare.queue_time_us":3592,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":43217,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":13461,"thread_start_us":2217489,"threads_started":2,"wal-append.queue_time_us":501603,"wal-append.run_cpu_time_us":1676,"wal-append.run_wall_time_us":1009551}
I20260709 13:18:40.030679 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:40.074586 12634 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:40.033826 13380 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:18:39.904446 15137 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:40.014640 15296 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.952464 12628 raft_consensus.cc:2410] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:18:40.050420 12708 raft_consensus.cc:2393] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 8: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:40.047482 12629 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:18:40.048235 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:40.063122 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 109877us
I20260709 13:18:40.081019 12631 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:40.081435 12747 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:40.082679 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 66050.9us
I20260709 13:18:40.083627 12597 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:40.034807 15263 raft_consensus.cc:740] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:40.068519 13384 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:18:40.077724 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:40.096552 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:40.065994 15251 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:40.105473 12747 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:40.068351 15266 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:40.067700 15027 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:40.015336 12528 rpcz_store.cc:269] 0709 13:18:27.060950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.061197 (+   247us) service_pool.cc:224] Handling call
0709 13:18:40.014827 (+12953630us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:36.336867 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:36.338370 (+  1503us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:36.338446 (+    76us) write_op.cc:435] Acquired schema lock
0709 13:18:36.338468 (+    22us) tablet.cc:662] Decoding operations
0709 13:18:36.339905 (+  1437us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:36.339996 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:36.340028 (+    32us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:18:36.397946 (+ 57918us) tablet.cc:701] Row locks acquired
0709 13:18:36.398011 (+    65us) write_op.cc:260] PREPARE: finished
0709 13:18:36.398220 (+   209us) write_op.cc:270] Start()
0709 13:18:36.398399 (+   179us) write_op.cc:276] Timestamp: P: 1783603116398149 usec, L: 0
0709 13:18:36.398425 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:18:36.399173 (+   748us) log.cc:844] Serialized 5218 byte log entry
0709 13:18:37.400594 (+1001421us) op_driver.cc:464] REPLICATION: finished
0709 13:18:37.685837 (+285243us) write_op.cc:301] APPLY: starting
0709 13:18:37.685958 (+   121us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:37.686282 (+   324us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:37.688632 (+  2350us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:37.867375 (+178743us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:37.867857 (+   482us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 13:18:37.867916 (+    59us) write_op.cc:312] APPLY: finished
0709 13:18:37.868392 (+   476us) log.cc:844] Serialized 95 byte log entry
0709 13:18:37.868625 (+   233us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:37.868800 (+   175us) write_op.cc:454] Released schema lock
0709 13:18:37.868895 (+    95us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:37.869391 (+   496us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:37.869429 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2626527,"reported_tablets":24,"child_traces":[["op",{"apply.queue_time_us":284669,"apply.run_cpu_time_us":6107,"apply.run_wall_time_us":183680,"num_ops":13,"prepare.queue_time_us":709997,"prepare.run_cpu_time_us":64646,"prepare.run_wall_time_us":385525,"raft.queue_time_us":510030,"raft.run_cpu_time_us":91236,"raft.run_wall_time_us":261619,"replication_time_us":1001971,"thread_start_us":1136918,"threads_started":4,"wal-append.queue_time_us":456689,"wal-append.run_cpu_time_us":11977,"wal-append.run_wall_time_us":2052134}]]}
I20260709 13:18:40.070279 15240 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:40.108170 12597 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:40.103868 12673 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:40.111249 12747 leader_election.cc:400] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7.
I20260709 13:18:40.112155 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:40.070801 13377 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:40.117415 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:40.063129 15322 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:40.098744 15292 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:40.101768 15185 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:40.091050 15328 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:40.109938 15077 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:40.103832 15326 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:40.101466 15281 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:40.128623 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:40.108632 15294 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:40.102476 15199 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:40.124958 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:40.135541 12746 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:40.112114 15038 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:18:40.120276 12673 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:40.112936 15324 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:40.091312 15296 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:40.140570 12598 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:40.120359 15170 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.139726 13349 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:40.202096 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:40.284514 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 144047us
I20260709 13:18:40.254320 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:40.288041 12598 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:40.284515 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 149055us
I20260709 13:18:40.112725 15240 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:40.117712 15239 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:40.120623 14847 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:40.123204 15222 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:40.219736 12774 raft_consensus.cc:1275] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:18:40.126586 15251 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:40.238224 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:18:40.286617 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 161714us
I20260709 13:18:40.384339 12747 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:40.285741 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 165540us
I20260709 13:18:40.370491 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:40.386411 12776 raft_consensus.cc:2410] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:18:40.124220 15235 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:40.371254 12781 raft_consensus.cc:3055] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:40.387089 12781 raft_consensus.cc:740] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:40.153492 12775 raft_consensus.cc:1275] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:18:40.239555 15335 log.cc:927] Time spent T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 13:18:40.382864 12778 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:40.130460 15185 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.391119 12673 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:40.204912 15026 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.405606 12747 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:40.237142 15014 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:40.222226 15312 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:40.388666 12781 consensus_queue.cc:260] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:40.404654 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 264997us
I20260709 13:18:40.433507 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:40.369774 15329 log.cc:927] Time spent T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.267s	user 0.000s	sys 0.002s
I20260709 13:18:40.151592 15328 raft_consensus.cc:697] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:40.152444 15326 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:40.292910 15320 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:40.286985 15247 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:40.195283 15009 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:40.292644 15281 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:40.431972 12782 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:18:40.132659 15197 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.277s
I20260709 13:18:40.371810 15239 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.382380 15316 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:40.171118 15292 consensus_queue.cc:1048] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.265s
W20260709 13:18:40.456079 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 168119us
I20260709 13:18:40.417737 15293 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:40.454118 15311 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:40.461225 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:40.421618 14934 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:40.417546 15175 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:40.421425 14939 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:40.320168 15170 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:40.293160 15199 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:40.418206 15294 raft_consensus.cc:697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:40.378968 14745 consensus_queue.cc:1059] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 13:18:40.151973 15263 consensus_queue.cc:260] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:40.452850 15298 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:40.417814 15185 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:40.459852 15320 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.509848 15320 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:40.510140 15332 log.cc:927] Time spent T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.223s	user 0.000s	sys 0.000s
W20260709 13:18:40.510703 15339 log.cc:927] Time spent T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:18:40.511657 13376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51762 (request call id 505) took 1416 ms. Trace:
W20260709 13:18:40.512661 15348 log.cc:927] Time spent T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:18:40.421591 14847 consensus_queue.cc:1059] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:18:40.459671 15326 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:40.495549 15099 consensus_meta.cc:287] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.555s	user 0.006s	sys 0.006s
I20260709 13:18:40.391760 15324 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:40.494441 15243 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.512079 13376 rpcz_store.cc:276] 0709 13:18:39.095466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.097935 (+  2469us) service_pool.cc:224] Handling call
0709 13:18:39.104501 (+  6566us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:39.189232 (+ 84731us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:39.198296 (+  9064us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:39.373262 (+174966us) log.cc:844] Serialized 26 byte log entry
0709 13:18:40.037961 (+664699us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:40.047973 (+ 10012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.053375 (+  5402us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:40.511412 (+458037us) raft_consensus.cc:1685] finished
0709 13:18:40.511443 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:40.511597 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20647,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":84,"thread_start_us":446007,"threads_started":1,"wal-append.queue_time_us":667463}
I20260709 13:18:40.472060 15351 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:40.466768 15239 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:40.476262 15316 raft_consensus.cc:697] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:40.481305 14934 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.548889 14934 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:40.466086 15009 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:40.555492 15009 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:40.431815 15026 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:40.533607 15356 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:40.557734 15324 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:40.557629 15356 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:40.558300 15338 log.cc:927] Time spent T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:18:40.558830 15026 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:40.560058 15356 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:40.560211 13381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51762 (request call id 503) took 1467 ms. Trace:
W20260709 13:18:40.560051 15343 log.cc:927] Time spent T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.331s	user 0.000s	sys 0.001s
I20260709 13:18:40.560688 13381 rpcz_store.cc:276] 0709 13:18:39.092499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.123106 (+ 30607us) service_pool.cc:224] Handling call
0709 13:18:39.202943 (+ 79837us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:39.606578 (+403635us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:39.840003 (+233425us) log.cc:844] Serialized 14 byte log entry
0709 13:18:40.104210 (+264207us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:40.132379 (+ 28169us) log.cc:844] Serialized 26 byte log entry
0709 13:18:40.152331 (+ 19952us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:40.168030 (+ 15699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.201469 (+ 33439us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:40.559942 (+358473us) raft_consensus.cc:1685] finished
0709 13:18:40.559975 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:40.560126 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":321454,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":54028,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":22052,"thread_start_us":683288,"threads_started":3,"wal-append.queue_time_us":400551}
W20260709 13:18:40.534127 15336 log.cc:927] Time spent T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.300s	user 0.000s	sys 0.000s
I20260709 13:18:40.534502 15263 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:40.545713 15243 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:40.567977 15243 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:40.538149 15342 log.cc:927] Time spent T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 13:18:40.490296 15322 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:40.487646 15175 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:40.569375 15175 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:40.574630 12628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56832 (request call id 2079) took 1294 ms. Trace:
I20260709 13:18:40.575076 12628 rpcz_store.cc:276] 0709 13:18:39.280566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.280814 (+   248us) service_pool.cc:224] Handling call
0709 13:18:40.574415 (+1293601us) spinlock_profiling.cc:243] Waited 615 ms on lock 0x7b54000dcd54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:40.574561 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1414292864}
I20260709 13:18:40.577359 15251 raft_consensus.cc:3055] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:40.577829 15251 raft_consensus.cc:740] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:40.578630 15251 consensus_queue.cc:260] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:40.579931 15251 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:40.581339 15294 consensus_queue.cc:237] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:40.582333 15349 log.cc:927] Time spent T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:18:40.582662 15185 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:40.583940 14939 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:40.584157 15199 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:40.585289 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:40.543010 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 97574.2us
I20260709 13:18:40.586740 12781 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:40.519935 15281 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:40.589172 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:40.597446 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:40.598528 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:40.599665 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:40.613101 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:40.614151 12634 raft_consensus.cc:2393] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 7: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:40.614984 12708 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:40.616014 12625 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:40.615084 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:40.619083 12634 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:40.619491 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:40.623628 12631 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:40.602687 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:40.614418 12628 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:40.630309 12625 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:18:40.631958 12625 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:40.632262 12781 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:40.633358 12781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 281) took 1349 ms. Trace:
W20260709 13:18:40.641033 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 54864.6us
I20260709 13:18:40.638903 12778 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:18:40.577157 15240 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:40.646024 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 240550us
I20260709 13:18:40.584484 14939 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:40.646076 12634 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
W20260709 13:18:40.636576 15333 log.cc:927] Time spent T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.408s	user 0.000s	sys 0.001s
I20260709 13:18:40.584342 14745 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:40.652030 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:40.659865 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:40.639655 15337 log.cc:927] Time spent T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.398s	user 0.000s	sys 0.001s
W20260709 13:18:40.651633 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 246568us
I20260709 13:18:40.636138 12781 rpcz_store.cc:276] 0709 13:18:39.283850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.363159 (+ 79309us) service_pool.cc:224] Handling call
0709 13:18:39.363531 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:40.294835 (+931304us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b54001fb1a0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d329dc0 00007f4c6d3b41c0 00007f4c6d3b5231 00007f4c6d3b1d8f 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c
0709 13:18:40.586348 (+291513us) spinlock_profiling.cc:243] Waited 84.5 ms on lock 0x7b58007d75d0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d30e0aa 00007f4c6d390447 00007f4c6d3a8fe7 00007f4c6d3aa7de 00007f4c6d3b4b49 00007f4c6d3b5416 00007f4c6d3b1d8f 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4
0709 13:18:40.632649 (+ 46301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.633172 (+   523us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":738631808}
I20260709 13:18:40.635630 15356 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.568760 14934 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:40.617796 15014 consensus_queue.cc:1048] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:18:40.649591 15293 raft_consensus.cc:697] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:40.666144 14479 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:18:40.653779 15282 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
W20260709 13:18:40.663257 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.643301 15351 raft_consensus.cc:697] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:40.642683 15355 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:40.670135 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:40.658982 12708 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:18:40.666198 15296 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:40.646158 15222 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:40.657711 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:40.652782 15099 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.677313 15026 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:18:40.680404 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 210356us
W20260709 13:18:40.683748 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.644042 15243 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
W20260709 13:18:40.675488 15266 consensus_meta.cc:287] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.563s	user 0.008s	sys 0.003s
I20260709 13:18:40.672222 12627 raft_consensus.cc:2410] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:18:40.719347 12703 raft_consensus.cc:1275] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:18:40.721365 12708 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:40.733063 12705 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8. Ops: [6.7-7.8]
I20260709 13:18:40.545030 15320 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:40.734639 12707 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:40.672413 15356 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:40.668915 15199 consensus_queue.cc:1048] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 13:18:40.665251 14745 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:40.673188 15324 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 13:18:40.686065 14847 consensus_queue.cc:1048] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 13:18:40.584754 15328 consensus_queue.cc:237] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:40.734650 12634 raft_consensus.cc:2410] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:18:40.764976 12704 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7. Ops: []
W20260709 13:18:40.757442 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 99398.7us
I20260709 13:18:40.774900 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:40.775621 12625 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:18:40.791925 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:40.792130 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:40.794005 13349 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:40.794402 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:40.795856 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:40.796423 12776 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:18:40.682905 14613 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:40.791119 12701 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
W20260709 13:18:40.735287 15354 log.cc:927] Time spent T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:18:40.811708 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:40.812711 12782 raft_consensus.cc:2393] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:40.813288 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:40.733959 15355 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:40.815876 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:40.816184 14745 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:40.816831 15353 log.cc:927] Time spent T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.282s	user 0.002s	sys 0.002s
I20260709 13:18:40.816964 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:40.817430 15355 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:40.755901 15222 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:40.818339 14956 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:40.818636 15352 log.cc:927] Time spent T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:18:40.818897 14956 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:40.732409 15183 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:40.820205 15222 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:40.820016 15183 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:40.821009 14956 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:40.821523 15364 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:40.821708 15025 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.822175 15183 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:40.817912 14613 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:40.822439 15025 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:40.764223 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:40.823917 15320 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:40.824368 14613 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:40.824508 15025 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:40.825098 15025 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.825253 13381 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:40.832424 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:40.856726 12625 raft_consensus.cc:1275] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:18:40.877142 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:40.881013 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:40.884692 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 75622.6us
I20260709 13:18:40.884840 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:40.758868 15362 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:40.884537 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:40.793720 15170 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:40.890429 12704 raft_consensus.cc:2393] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 6: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:40.891585 12782 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:40.892690 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:40.893078 12634 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:40.805456 15178 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:40.896562 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:40.897749 12634 raft_consensus.cc:2393] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:40.897891 12780 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:18:40.899711 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 296385us
I20260709 13:18:40.901768 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:40.824594 15320 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.794910 15099 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:40.909893 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 73157us
I20260709 13:18:40.825541 15025 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:40.906201 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 110432us
I20260709 13:18:40.909098 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:40.888790 15355 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.925005 12597 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:40.878691 15266 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:40.793257 15293 consensus_queue.cc:237] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:40.923230 12673 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:40.892359 15199 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.753664 15316 consensus_queue.cc:237] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:40.817915 15365 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.088s
I20260709 13:18:40.912127 12780 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:40.934280 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:40.946178 15077 consensus_meta.cc:287] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.810s	user 0.011s	sys 0.000s
I20260709 13:18:40.946627 15038 raft_consensus.cc:3055] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:18:40.946979 15079 consensus_meta.cc:287] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.917s	user 0.008s	sys 0.004s
I20260709 13:18:40.948287 12673 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:40.948894 15009 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:40.949082 15247 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:40.949594 15243 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:18:40.950464 15265 consensus_meta.cc:287] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.998s	user 0.012s	sys 0.000s
I20260709 13:18:40.955906 15185 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:40.960602 15366 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:18:40.964439 15240 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:40.906126 15178 raft_consensus.cc:697] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:40.933225 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:40.990864 15232 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:40.922312 15197 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.971349 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:40.972419 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:40.973088 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:40.973148 14939 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:40.900614 15362 raft_consensus.cc:697] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:40.983311 13348 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:40.984086 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:40.943562 15266 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:40.992457 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:40.994316 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:40.995085 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:40.977337 15141 consensus_meta.cc:287] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 1.060s	user 0.010s	sys 0.003s
I20260709 13:18:41.007736 12633 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
W20260709 13:18:41.011937 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 63729.6us
I20260709 13:18:41.012452 14479 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:41.018222 15185 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.879295 15364 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:40.995500 15354 log.cc:927] Time spent T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:18:40.966849 15175 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:40.911375 15320 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:40.989142 15038 raft_consensus.cc:740] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:41.004217 15009 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.933755 15355 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:41.008344 14613 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.127264 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:41.149528 15008 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:40.985761 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 45389.3us
I20260709 13:18:41.139465 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:41.175205 12704 raft_consensus.cc:2393] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 7: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
W20260709 13:18:41.130640 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 147406us
I20260709 13:18:41.130678 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:40.975451 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:40.995247 15026 consensus_queue.cc:1059] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.128s
I20260709 13:18:40.985177 15199 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:40.878675 15351 consensus_queue.cc:237] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:41.074084 15265 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.155090 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:41.159236 12782 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:41.123396 15079 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:41.129452 15014 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.128598 15232 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:40.984647 15312 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:18:41.132016 15197 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:41.165450 15009 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:41.197286 12634 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:41.193321 13382 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:41.223793 15383 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:18:41.224781 15374 log.cc:927] Time spent T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:18:41.157543 15185 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:41.170900 14613 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:41.152119 15141 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:41.236719 15311 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.243079 15199 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:41.246721 15373 log.cc:927] Time spent T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:18:41.248651 15320 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:41.254511 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:41.262288 15197 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:41.267854 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:41.268671 12782 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:18:41.228437 15242 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.270859 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:18:41.299634 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.180343 15008 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:41.239663 15383 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:18:41.263316 15099 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:41.310978 15371 log.cc:927] Time spent T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 13:18:41.228029 15014 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:41.222836 15079 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:41.259565 15365 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:41.322810 15242 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:41.181452 15178 consensus_queue.cc:237] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:41.277650 15364 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:41.217964 15265 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:41.318863 15199 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:18:41.348290 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.357805 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:41.358886 12780 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 6: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
W20260709 13:18:41.477813 12600 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.11.63.126:38321, user_credentials={real_user=slave}} blocked reactor thread for 123101us
W20260709 13:18:41.490096 15239 consensus_meta.cc:287] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.876s	user 0.009s	sys 0.002s
I20260709 13:18:41.490731 15239 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:41.490242 15381 log.cc:927] Time spent T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:18:41.497247 12629 raft_consensus.cc:1240] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8. Ops: []
W20260709 13:18:41.495133 15385 log.cc:927] Time spent T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
I20260709 13:18:41.495631 15266 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:41.496239 15239 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:41.273376 15311 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:41.362588 15199 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.275556 15038 consensus_queue.cc:260] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:41.260105 15362 consensus_queue.cc:237] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:41.350692 15312 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.157s
I20260709 13:18:41.522151 15175 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:41.525089 12659 heartbeater.cc:646] Failed to heartbeat to 127.11.63.126:38321 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.11.63.126:38321 timed out after 15.000s (SENT)
I20260709 13:18:41.530145 15232 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:41.530475 14820 consensus_meta.cc:287] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 1.470s	user 0.013s	sys 0.000s
I20260709 13:18:41.533218 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:41.544087 12632 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:41.534500 15263 consensus_meta.cc:287] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.970s	user 0.011s	sys 0.000s
W20260709 13:18:41.539331 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 354083us
I20260709 13:18:41.477489 15242 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:41.341881 15365 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:41.514935 15199 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:41.595568 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:41.502857 15170 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.003s	sys 0.013s
I20260709 13:18:41.566659 12781 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:41.569092 15386 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:41.570708 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:41.571980 13376 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:41.540959 15038 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:41.574707 15265 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:41.584743 15247 consensus_meta.cc:287] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.577s	user 0.008s	sys 0.000s
I20260709 13:18:41.536283 15356 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:41.566991 15232 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
W20260709 13:18:41.602403 15170 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.009s	sys 0.013s
I20260709 13:18:41.619077 12597 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:41.563932 14820 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:18:41.619870 15392 log.cc:927] Time spent T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:18:41.620452 14820 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:41.620844 14820 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.621165 14820 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:41.621943 15251 consensus_meta.cc:287] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 1.042s	user 0.015s	sys 0.000s
I20260709 13:18:41.622809 12777 raft_consensus.cc:2410] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:18:41.623541 15240 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.624111 15240 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:41.543706 15383 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:41.606667 15141 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:41.610816 15185 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:41.625880 15383 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:41.626528 15185 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:41.627096 15185 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:41.628264 15383 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:41.629032 12626 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:41.564543 12775 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:41.596954 13380 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:41.565402 15355 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:41.603173 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:41.634517 13379 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:41.635811 15311 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:41.636466 15311 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:41.613428 15386 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:41.640062 15386 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:41.641196 15014 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:41.641800 15360 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.642201 13379 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:41.587568 12632 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:41.637938 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:41.642329 15360 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:41.598363 15326 consensus_meta.cc:287] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 1.059s	user 0.012s	sys 0.000s
I20260709 13:18:41.647154 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:41.648445 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:41.650471 12632 raft_consensus.cc:2376] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6. Re-sending same reply.
I20260709 13:18:41.652045 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:41.652670 12632 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
I20260709 13:18:41.653420 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:41.653463 12626 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:18:41.654441 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:41.636996 15311 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:41.611101 15247 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:41.658115 13349 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:41.629437 15185 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:41.645464 15360 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:41.680538 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:41.668489 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:41.655390 15079 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:41.694697 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:41.697307 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:41.666677 15312 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:41.666455 15383 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:41.701372 15009 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:41.677105 15386 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:41.690061 15355 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.655645 15326 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:41.722061 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:41.711504 15386 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:41.748656 15386 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:41.750566 15240 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:41.753875 15356 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.754503 15356 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:41.712644 15312 raft_consensus.cc:697] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 7 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:41.760301 12631 raft_consensus.cc:1240] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:18:41.762459 15311 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:41.765285 15386 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:41.743512 15355 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:41.772339 12628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2095) took 1133 ms. Trace:
I20260709 13:18:41.797528 12628 rpcz_store.cc:276] 0709 13:18:40.639432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.643649 (+  4217us) service_pool.cc:224] Handling call
0709 13:18:40.650025 (+  6376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:40.832781 (+182756us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:40.832808 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:40.832845 (+    37us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:40.832874 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:41.772095 (+939221us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b54002db4d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:41.772141 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:41.772276 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":419879168}
I20260709 13:18:41.776096 15025 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:41.777940 15362 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.803726 15362 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:41.778286 15009 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:41.717350 15079 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.809672 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:41.720963 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:41.792027 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:41.812232 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:41.796319 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:41.813059 12774 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:41.797966 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:41.823518 15362 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:41.773447 15199 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:41.834712 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:41.837338 12628 raft_consensus.cc:2410] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:18:41.845155 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:41.845978 12628 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:41.824939 15356 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:41.828220 15355 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:41.825528 13376 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:41.843657 15362 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.855939 15362 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:41.859210 15362 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:41.820178 15079 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:41.864970 15355 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:41.865645 15355 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:41.864233 12628 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:41.867986 15356 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.871187 15356 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:41.872412 15199 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:41.873267 15356 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:41.848238 15038 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:41.878731 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 906297us
I20260709 13:18:41.853971 15025 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.867807 15137 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.867873 15355 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:41.881927 15136 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:41.885008 15362 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.848213 15312 consensus_queue.cc:237] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:41.873759 14820 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:41.893461 15038 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:41.948230 15326 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:41.899480 15243 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:18:41.942034 15362 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:41.898103 15025 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:41.951001 14479 raft_consensus.cc:697] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:41.908964 15137 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:41.980710 14479 consensus_queue.cc:237] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:41.978569 15386 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:41.981907 15141 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:18:42.008060 14820 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:42.009080 15079 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:42.001341 15402 log.cc:927] Time spent T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:18:42.009450 15137 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:42.012732 15038 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:42.012914 15027 raft_consensus.cc:3055] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:42.013531 15027 raft_consensus.cc:740] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:42.008076 15362 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:42.014761 15027 consensus_queue.cc:260] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.026278 15025 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:42.027120 15027 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:41.967985 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:42.069711 12628 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
W20260709 13:18:42.074551 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 930892us
I20260709 13:18:41.970391 12632 raft_consensus.cc:2393] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 8: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:41.971724 12672 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:42.082876 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 111109us
W20260709 13:18:42.079717 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.088363 12673 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:42.080858 12783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1718) took 1153 ms. Trace:
I20260709 13:18:42.073477 12628 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:42.096628 12628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1243) took 1211 ms. Trace:
I20260709 13:18:42.091804 15141 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:42.097203 12628 rpcz_store.cc:276] 0709 13:18:40.885629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.072130 (+1186501us) service_pool.cc:224] Handling call
0709 13:18:42.073120 (+   990us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:42.093496 (+ 20376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.096480 (+  2984us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:42.103609 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:42.107761 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:42.108814 12703 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:42.109053 12700 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:18:42.110807 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:42.112479 12700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1309) took 1393 ms. Trace:
I20260709 13:18:42.113058 12700 rpcz_store.cc:276] 0709 13:18:40.719216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.719515 (+   299us) service_pool.cc:224] Handling call
0709 13:18:42.108017 (+1388502us) spinlock_profiling.cc:243] Waited 5.72 ms on lock 0x7b5800002798. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28deee 0000000000375939 00007f4c723011e0 00007f4c723ca0c4 00007f4c723ca131 00007f4c7238bfc0 00007f4c72376929 00007f4c723763d2 00007f4c6b8edbd4
0709 13:18:42.108354 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:42.109543 (+  1189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.112361 (+  2818us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13145856}
I20260709 13:18:42.092922 12783 rpcz_store.cc:276] 0709 13:18:40.927658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.131224 (+203566us) service_pool.cc:224] Handling call
0709 13:18:41.131514 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:41.193520 (+ 62006us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:41.193550 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:41.218321 (+ 24771us) log.cc:844] Serialized 26 byte log entry
0709 13:18:41.505662 (+287341us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:41.505718 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:41.996483 (+490765us) spinlock_profiling.cc:243] Waited 49.1 ms on lock 0x7b540019d2d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:41.996535 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:41.996581 (+    46us) raft_consensus.cc:1685] finished
0709 13:18:41.996593 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:42.080468 (+ 83875us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12421,"raft.queue_time_us":40968,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":24414,"spinlock_wait_cycles":112869248,"thread_start_us":242188,"threads_started":1,"wal-append.queue_time_us":438468}
I20260709 13:18:42.122102 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:42.123193 12706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1316) took 1387 ms. Trace:
I20260709 13:18:42.123605 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:42.123692 12706 rpcz_store.cc:276] 0709 13:18:40.735699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.790856 (+ 55157us) service_pool.cc:224] Handling call
0709 13:18:40.808505 (+ 17649us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:40.839531 (+ 31026us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:40.839562 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:40.839602 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:40.839635 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.122646 (+1283011us) spinlock_profiling.cc:243] Waited 30.6 ms on lock 0x7b54001b0654. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:42.122748 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:42.122986 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":70334848}
I20260709 13:18:42.124594 12704 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:18:42.126027 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:42.128948 15025 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:42.129431 12705 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:42.130514 15025 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.131520 15362 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:42.131978 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:42.132974 12701 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:42.132542 15362 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.134267 15025 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:42.134788 15141 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:42.135738 15141 raft_consensus.cc:697] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:42.135807 15362 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:42.139632 13382 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:18:42.134307 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:42.143131 15293 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.148557 15362 consensus_queue.cc:1059] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:42.149094 15293 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.150173 15362 raft_consensus.cc:3055] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:42.150951 15362 raft_consensus.cc:740] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:42.152235 15293 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:42.152102 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:42.152606 15362 consensus_queue.cc:260] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.154632 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:42.155601 12709 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 4.
I20260709 13:18:42.141660 15141 consensus_queue.cc:237] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.156093 15362 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:42.157972 12709 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:42.158210 15235 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.158455 15038 consensus_queue.cc:1059] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:42.159241 15235 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.160514 15265 raft_consensus.cc:3055] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:42.161067 15265 raft_consensus.cc:740] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:42.162111 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:42.162438 15265 consensus_queue.cc:260] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.163167 15235 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:42.163746 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 75829.2us
I20260709 13:18:42.164621 15265 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:42.170517 12626 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39590 (request call id 307) took 1727 ms (client timeout 2190 ms). Trace:
W20260709 13:18:42.173149 12627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56832 (request call id 2089) took 1578 ms (client timeout 1762 ms). Trace:
I20260709 13:18:42.152613 14613 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:42.177546 12627 rpcz_store.cc:269] 0709 13:18:40.594468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.602385 (+  7917us) service_pool.cc:224] Handling call
0709 13:18:42.172613 (+1570228us) spinlock_profiling.cc:243] Waited 47.2 ms on lock 0x7b540002c954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:42.172908 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108560640}
I20260709 13:18:42.178542 12630 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:18:42.178890 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:42.174639 12625 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:42.179880 12709 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:42.181043 12630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2070) took 3001 ms. Trace:
W20260709 13:18:42.174732 12633 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39590 (request call id 288) took 3122 ms (client timeout 1502 ms). Trace:
I20260709 13:18:42.186144 12630 rpcz_store.cc:276] 0709 13:18:39.179680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.180934 (+  1254us) service_pool.cc:224] Handling call
0709 13:18:42.177750 (+2996816us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:42.179041 (+  1291us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.180721 (+  1680us) spinlock_profiling.cc:243] Waited 3 s on lock 0x7b5400159fd0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:42.180951 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6891188096}
I20260709 13:18:42.176855 15222 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.181346 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:42.182662 12631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2096) took 1534 ms. Trace:
I20260709 13:18:42.183861 12634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56838 (request call id 1249) took 1296 ms. Trace:
W20260709 13:18:42.171289 12626 rpcz_store.cc:269] 0709 13:18:40.442366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.442628 (+   262us) service_pool.cc:224] Handling call
0709 13:18:42.169835 (+1727207us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540004bad4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:42.170090 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2648174336}
W20260709 13:18:42.186357 12633 rpcz_store.cc:269] 0709 13:18:39.051717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.053964 (+  2247us) service_pool.cc:224] Handling call
0709 13:18:42.174235 (+3120271us) spinlock_profiling.cc:243] Waited 1.94 s on lock 0x7b5400159fd4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:42.174451 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4469087616}
I20260709 13:18:42.189579 15222 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.190828 12634 rpcz_store.cc:276] 0709 13:18:40.887238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.905971 (+ 18733us) service_pool.cc:224] Handling call
0709 13:18:42.183414 (+1277443us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b54002ecf54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:42.183607 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":631234304}
I20260709 13:18:42.190196 12631 rpcz_store.cc:276] 0709 13:18:40.648263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.649229 (+   966us) service_pool.cc:224] Handling call
0709 13:18:40.659381 (+ 10152us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:41.786355 (+1126974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.182057 (+395702us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540004abd4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:42.182438 (+   381us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2531304832}
I20260709 13:18:42.194682 15222 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:42.189746 12629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2086) took 1583 ms. Trace:
W20260709 13:18:42.195912 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.195864 12629 rpcz_store.cc:276] 0709 13:18:40.593722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.596391 (+  2669us) service_pool.cc:224] Handling call
0709 13:18:40.596711 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:41.497698 (+900987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.177061 (+679363us) spinlock_profiling.cc:243] Waited 900 ms on lock 0x7b540027a454. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:42.177442 (+   381us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2070620416}
W20260709 13:18:42.196691 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:42.197135 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.199692 15324 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:42.186100 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:42.201292 12628 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:42.203224 14820 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:42.201319 15079 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:42.207866 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:42.208654 12700 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:42.208179 15079 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.209473 14820 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.210950 15079 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:42.200820 15358 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.212124 15358 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.214740 15356 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:42.215250 15358 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:42.200103 12629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1248) took 1312 ms. Trace:
I20260709 13:18:42.208076 15326 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:42.201527 12626 raft_consensus.cc:1240] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:42.204219 12706 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:18:42.216815 12628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1244) took 1320 ms. Trace:
I20260709 13:18:42.217487 12628 rpcz_store.cc:276] 0709 13:18:40.885878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.199959 (+1314081us) service_pool.cc:224] Handling call
0709 13:18:42.200823 (+   864us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:42.205599 (+  4776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.205872 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:42.216554 12629 rpcz_store.cc:276] 0709 13:18:40.886917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.197888 (+1310971us) service_pool.cc:224] Handling call
0709 13:18:42.198452 (+   564us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:42.198939 (+   487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:42.198980 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:42.199071 (+    91us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:42.199158 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.199247 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:42.199707 (+   460us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:42.220021 12626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 308) took 1306 ms. Trace:
I20260709 13:18:42.220384 12626 rpcz_store.cc:276] 0709 13:18:40.913291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.200346 (+1287055us) service_pool.cc:224] Handling call
0709 13:18:42.201220 (+   874us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:42.216772 (+ 15552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.219965 (+  3193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:42.201980 12703 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:42.221774 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1620) took 1386 ms. Trace:
W20260709 13:18:42.222427 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.219820 15079 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:42.215708 15356 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:42.226626 15079 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.228878 15079 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:42.230026 15326 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:42.231405 15355 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:42.228714 15356 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:42.233394 15355 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:42.236266 15355 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:42.236717 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:42.237656 12626 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:42.238781 15326 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:42.238976 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:42.239822 13349 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:42.240164 15324 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.242184 15365 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:42.242756 15027 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:42.243574 15324 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:42.243667 12746 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:42.244674 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:42.244899 15025 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:42.200184 15293 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.198770 15292 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.197908 15197 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.250685 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:42.212074 14820 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:42.222196 12702 rpcz_store.cc:276] 0709 13:18:40.835462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.141842 (+306380us) service_pool.cc:224] Handling call
0709 13:18:41.142262 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:41.971050 (+828788us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:41.971085 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:41.973400 (+  2315us) log.cc:844] Serialized 26 byte log entry
0709 13:18:42.096113 (+122713us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:42.096231 (+   118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.096393 (+   162us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:42.216324 (+119931us) raft_consensus.cc:1685] finished
0709 13:18:42.216359 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:42.221539 (+  5180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":739675,"raft.queue_time_us":752043,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":9136,"thread_start_us":88555,"threads_started":2,"wal-append.queue_time_us":128673}
I20260709 13:18:42.256718 15292 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:42.257344 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:42.243613 15027 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.254676 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:42.249025 15358 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:42.263760 15298 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:42.264416 15298 raft_consensus.cc:697] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:42.265780 15298 consensus_queue.cc:237] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:42.267163 15027 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:42.268209 15197 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.268812 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:42.270462 12778 raft_consensus.cc:2410] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 8 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 2.
I20260709 13:18:42.270673 15197 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:42.261013 12598 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:42.271664 12626 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:42.270951 12630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1247) took 1384 ms. Trace:
I20260709 13:18:42.263181 12673 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:42.272648 12630 rpcz_store.cc:276] 0709 13:18:40.886671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.269857 (+1383186us) service_pool.cc:224] Handling call
0709 13:18:42.270317 (+   460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:42.270600 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:42.270633 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:42.270679 (+    46us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:42.270714 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.270772 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:42.270895 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:42.278831 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:42.281077 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:42.284240 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:42.286302 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:42.272295 15027 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.300611 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:42.268740 15025 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:42.290457 14934 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:42.292047 15356 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:42.293179 15251 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:18:42.296344 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 63849.8us
I20260709 13:18:42.297830 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:42.287228 12628 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:42.286928 12629 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:42.308720 14479 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:42.306201 15025 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.309070 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:42.309154 14820 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:42.309463 14613 consensus_queue.cc:1059] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:42.300333 15027 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.310518 14613 raft_consensus.cc:3055] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:42.310899 14613 raft_consensus.cc:740] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:42.311399 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:42.312083 12776 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:42.312460 15027 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:42.312729 12775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36488 (request call id 1720) took 1336 ms. Trace:
I20260709 13:18:42.312093 14613 consensus_queue.cc:260] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:42.313292 12775 rpcz_store.cc:276] 0709 13:18:40.976387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.059115 (+ 82728us) service_pool.cc:224] Handling call
0709 13:18:42.312429 (+1253314us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b540011c954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:42.312558 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":725618432}
W20260709 13:18:42.302211 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.282269 12630 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:42.261332 15293 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.309787 12629 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:42.304112 15358 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.317231 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:42.318255 12709 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:42.314455 14613 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:42.300897 15252 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.322777 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:42.323976 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:42.325323 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:42.321202 15027 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:42.335868 14934 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:42.347498 14934 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.348819 15235 consensus_queue.cc:1048] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:42.335726 15356 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.336051 15251 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.353149 15292 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:42.353144 15251 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:42.354943 12633 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:42.355697 12633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1253) took 1446 ms. Trace:
I20260709 13:18:42.355842 12630 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:42.356379 15358 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:42.357762 15025 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:42.359733 15293 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:42.361605 14934 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:42.364575 12709 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:42.366182 12707 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:42.367594 13384 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:18:42.368067 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:42.368330 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:42.368793 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:42.370007 12776 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 6: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:42.368759 13382 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:42.371414 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:42.372314 12702 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:42.377898 12777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54346 (request call id 291) took 1565 ms. Trace:
I20260709 13:18:42.377940 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:42.378552 12777 rpcz_store.cc:276] 0709 13:18:40.812655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.816742 (+  4087us) service_pool.cc:224] Handling call
0709 13:18:42.377430 (+1560688us) spinlock_profiling.cc:243] Waited 805 ms on lock 0x7b54000b04d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:42.377693 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1851242752}
I20260709 13:18:42.380271 15027 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.380729 13382 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:42.382967 14934 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:42.339818 15099 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.383970 15099 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.385826 12626 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:42.343936 12779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 1995) took 1510 ms. Trace:
I20260709 13:18:42.386439 12626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 311) took 1472 ms. Trace:
I20260709 13:18:42.386821 12626 rpcz_store.cc:276] 0709 13:18:40.914286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.384871 (+1470585us) service_pool.cc:224] Handling call
0709 13:18:42.385229 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:42.386200 (+   971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.386388 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:42.386706 12779 rpcz_store.cc:276] 0709 13:18:40.833128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.834282 (+  1154us) service_pool.cc:224] Handling call
0709 13:18:40.834594 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:41.169957 (+335363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:41.169985 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:41.170021 (+    36us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:41.170048 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.343445 (+1173397us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b54002cbd54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:42.343515 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:42.343730 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":770856192}
I20260709 13:18:42.383303 12631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1251) took 1495 ms. Trace:
I20260709 13:18:42.388393 12631 rpcz_store.cc:276] 0709 13:18:40.888334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.382298 (+1493964us) service_pool.cc:224] Handling call
0709 13:18:42.382682 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:42.382935 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:42.382961 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:42.383009 (+    48us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:42.383049 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.383109 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:42.383228 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:42.390103 12781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36488 (request call id 1712) took 1554 ms. Trace:
I20260709 13:18:42.390244 12631 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:42.390847 12631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 312) took 1476 ms. Trace:
I20260709 13:18:42.390544 12781 rpcz_store.cc:276] 0709 13:18:40.791581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.791711 (+   130us) service_pool.cc:224] Handling call
0709 13:18:42.345240 (+1553529us) spinlock_profiling.cc:243] Waited 774 ms on lock 0x7b540044bbd4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:42.345415 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1780159872}
I20260709 13:18:42.365314 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:42.391712 12631 rpcz_store.cc:276] 0709 13:18:40.914583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.389465 (+1474882us) service_pool.cc:224] Handling call
0709 13:18:42.389893 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:42.390604 (+   711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.390794 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:42.392577 12634 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:18:42.393326 12634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1250) took 1505 ms. Trace:
I20260709 13:18:42.393746 12634 rpcz_store.cc:276] 0709 13:18:40.887697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.353373 (+1465676us) service_pool.cc:224] Handling call
0709 13:18:42.392295 (+ 38922us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:42.393043 (+   748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.393252 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:42.394661 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.356086 12633 rpcz_store.cc:276] 0709 13:18:40.908893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.354073 (+1445180us) service_pool.cc:224] Handling call
0709 13:18:42.354710 (+   637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:42.355436 (+   726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.355633 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:42.337954 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 122042us
I20260709 13:18:42.368963 15008 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:42.407927 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:42.408710 12781 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:42.408746 15038 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:42.407958 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:42.409554 15038 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:42.404242 12625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1241) took 1513 ms. Trace:
I20260709 13:18:42.411118 12625 rpcz_store.cc:276] 0709 13:18:40.885044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.889140 (+  4096us) service_pool.cc:224] Handling call
0709 13:18:42.174005 (+1284865us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:42.180116 (+  6111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.397820 (+217704us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540004bad0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:42.397944 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2953740544}
I20260709 13:18:42.412688 15038 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:42.414135 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 167647us
I20260709 13:18:42.417135 15222 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:42.420688 12629 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:42.421304 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:42.422148 12702 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:18:42.422592 12629 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:42.417272 12673 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:42.419567 15222 consensus_queue.cc:1048] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:42.426461 12632 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:42.396129 12527 catalog_manager.cc:5697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 4 to 5, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 5 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:18:42.431596 12632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 310) took 1517 ms. Trace:
I20260709 13:18:42.386468 15099 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:42.432086 12632 rpcz_store.cc:276] 0709 13:18:40.913985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.381121 (+1467136us) service_pool.cc:224] Handling call
0709 13:18:42.381512 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:42.427142 (+ 45630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.431509 (+  4367us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:42.457056 12598 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:42.460628 12628 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:42.465778 12627 raft_consensus.cc:1240] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:42.427840 15356 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:42.478118 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 61028.4us
I20260709 13:18:42.469652 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:42.485962 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:42.486483 12781 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:18:42.487155 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:42.496109 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.506662 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:42.508087 12781 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:42.509407 12633 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:18:42.510607 12777 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:42.515211 15251 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:42.515563 15027 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:42.515875 15370 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:42.516054 15251 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:42.516788 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 43935.6us
I20260709 13:18:42.515271 12782 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:18:42.518011 12782 pending_rounds.cc:85] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 3
I20260709 13:18:42.518322 12782 pending_rounds.cc:107] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:18:42.518656 12782 raft_consensus.cc:2889] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:42.521685 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:42.524209 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:42.524221 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 99287.4us
I20260709 13:18:42.524907 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:42.525986 12709 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:42.531813 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:42.510573 15356 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:42.512027 12527 catalog_manager.cc:5697] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 2 to 6, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 6 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:42.459117 15252 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:42.538527 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:42.536194 12707 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:42.550153 12627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1246) took 1663 ms. Trace:
I20260709 13:18:42.550748 12627 rpcz_store.cc:276] 0709 13:18:40.886357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.352393 (+1466036us) service_pool.cc:224] Handling call
0709 13:18:42.461623 (+109230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:42.466410 (+  4787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.549862 (+ 83452us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54001595d0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:42.550037 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":250126080}
I20260709 13:18:42.553300 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:42.556438 15386 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
W20260709 13:18:42.557428 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 78397.7us
I20260709 13:18:42.559262 15252 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:42.559957 15252 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:42.562901 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:42.563901 12627 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:18:42.564828 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:42.566704 12633 raft_consensus.cc:1240] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:18:42.566475 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:42.568013 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:42.568504 12627 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 6: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:42.569316 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:42.571033 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.572589 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:42.573161 13384 raft_consensus.cc:1240] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:42.573767 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:42.575587 12776 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:18:42.575982 12776 pending_rounds.cc:85] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 3
I20260709 13:18:42.576237 12776 pending_rounds.cc:107] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:18:42.576457 12776 raft_consensus.cc:2889] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:42.577106 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:42.577942 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:42.586406 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:42.597887 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:42.599072 12778 raft_consensus.cc:2410] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:18:42.600157 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:42.600801 13380 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:42.601892 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:42.603433 13376 raft_consensus.cc:2410] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:18:42.606477 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:42.607074 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:42.600651 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:42.605140 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:42.746953 12628 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:18:42.732450 13380 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:18:42.822214 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:42.616366 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:42.906363 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:42.876747 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:42.743245 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 232900us
I20260709 13:18:42.726974 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:42.842182 12630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 309) took 1928 ms. Trace:
I20260709 13:18:42.619270 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:42.785219 12631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 314) took 1747 ms. Trace:
I20260709 13:18:42.855078 12701 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
W20260709 13:18:42.981848 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.615556 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:42.730537 12781 raft_consensus.cc:2410] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:18:42.996030 12633 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
W20260709 13:18:42.996475 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.748586 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:42.977303 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:42.876686 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:42.987015 12629 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:42.626052 12598 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:42.600168 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:42.998397 12709 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 6: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:43.000404 13349 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:42.701713 13385 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:42.784440 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:42.961932 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:42.900461 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:42.963491 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:42.944490 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:43.009914 15386 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:42.622020 12527 catalog_manager.cc:5697] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 3 to 4, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 4 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:18:43.008700 13379 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:18:42.996035 12701 pending_rounds.cc:85] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 3
W20260709 13:18:42.881752 15265 consensus_meta.cc:287] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.717s	user 0.008s	sys 0.008s
I20260709 13:18:42.996687 12774 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:42.835798 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:43.017516 12634 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.004s	sys 0.017s
I20260709 13:18:42.872371 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:42.702152 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:43.043861 12634 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.004s	sys 0.018s
I20260709 13:18:43.011142 12628 pending_rounds.cc:85] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 3
I20260709 13:18:43.043968 12701 pending_rounds.cc:107] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:18:43.044692 12628 pending_rounds.cc:107] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:18:43.044828 12701 raft_consensus.cc:2889] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:43.044979 12628 raft_consensus.cc:2889] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:42.608050 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:43.052887 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 50116.9us
I20260709 13:18:43.067294 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:43.018312 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:43.001827 15254 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:42.995954 12631 rpcz_store.cc:276] 0709 13:18:40.982944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.392996 (+1410052us) service_pool.cc:224] Handling call
0709 13:18:42.393399 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:42.394588 (+  1189us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:42.502332 (+107744us) log.cc:844] Serialized 14 byte log entry
0709 13:18:42.729782 (+227450us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:42.729850 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:42.729919 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.730017 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:42.730168 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67680,"prepare.queue_time_us":126495,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":6151,"raft.queue_time_us":136489,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":50203,"thread_start_us":173072,"threads_started":2}
I20260709 13:18:43.075910 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:43.079114 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:43.058311 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:43.083544 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:43.084574 12777 raft_consensus.cc:2410] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:18:43.085275 12632 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:42.743405 15252 consensus_queue.cc:1059] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.300s
I20260709 13:18:42.995568 12630 rpcz_store.cc:276] 0709 13:18:40.913660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.356594 (+1442934us) service_pool.cc:224] Handling call
0709 13:18:42.357053 (+   459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:42.357444 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:42.505232 (+147788us) log.cc:844] Serialized 14 byte log entry
0709 13:18:42.841731 (+336499us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:42.841811 (+    80us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:42.841859 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.841956 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:42.842103 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2099,"prepare.queue_time_us":163344,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":51,"raft.queue_time_us":23083,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":167,"thread_start_us":455456,"threads_started":2}
I20260709 13:18:43.084187 12778 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:42.955685 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:43.102469 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:42.988444 15027 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:43.117244 13380 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:43.054042 15293 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:43.135960 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:43.136945 12777 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:43.137501 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:43.138238 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:43.119539 12779 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:43.149773 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:43.150925 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:43.154001 12632 raft_consensus.cc:1240] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:18:43.154631 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:43.160481 12629 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:43.169057 12703 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:43.100754 15251 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:43.092249 15265 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.727394 15370 consensus_queue.cc:1059] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.411s
I20260709 13:18:43.185245 13383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 468) took 1339 ms. Trace:
W20260709 13:18:43.185923 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 58172.1us
W20260709 13:18:43.184178 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 106619us
I20260709 13:18:43.205494 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:43.214392 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 285624us
I20260709 13:18:43.150276 12527 catalog_manager.cc:5697] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:43.225860 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:43.224675 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:43.214135 12746 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:43.243364 13377 raft_consensus.cc:1240] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:18:43.232190 12779 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:43.251752 12625 raft_consensus.cc:1275] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:43.251875 14613 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:43.252899 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 710988us
I20260709 13:18:43.253306 14613 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:43.257711 12778 raft_consensus.cc:1240] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:18:43.265986 12775 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:43.287628 14613 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:43.187203 13383 rpcz_store.cc:276] 0709 13:18:41.618076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.618230 (+   154us) service_pool.cc:224] Handling call
0709 13:18:42.350363 (+732133us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b5800095198. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c7238bfc0 00007f4c72376929 00007f4c723763d2 00007f4c6b8edbd4
0709 13:18:42.350646 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:42.351016 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:18:42.586791 (+235775us) log.cc:844] Serialized 14 byte log entry
0709 13:18:42.944263 (+357472us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:42.944362 (+    99us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:42.956532 (+ 12170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.956862 (+   330us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:42.957026 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":30793,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":319,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":113,"spinlock_wait_cycles":56671616,"thread_start_us":296467,"threads_started":2}
I20260709 13:18:43.252425 12597 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:43.323102 13349 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:43.339412 12781 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:18:43.340018 12781 pending_rounds.cc:85] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 7
I20260709 13:18:43.340325 12781 pending_rounds.cc:107] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:18:43.340641 12781 raft_consensus.cc:2889] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:43.221179 15027 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:43.336608 12774 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:18:43.215371 15265 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:43.293485 15370 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.335135 15222 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.355082 13348 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:43.365372 12527 catalog_manager.cc:5697] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 4 to 5, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 5 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:43.395017 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.213109 15312 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.499s
I20260709 13:18:43.404943 12626 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:43.343521 15298 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:43.381296 15141 raft_consensus.cc:3055] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:18:43.420497 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 85749.5us
I20260709 13:18:43.428274 12673 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:43.439334 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 195176us
I20260709 13:18:43.441102 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:43.441869 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 189588us
I20260709 13:18:43.443427 15265 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:43.443846 15422 log.cc:927] Time spent T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.602s	user 0.000s	sys 0.002s
I20260709 13:18:43.433667 15222 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:43.446987 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 258924us
W20260709 13:18:43.450194 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 106126us
I20260709 13:18:43.450330 15436 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:43.452126 15437 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:43.351481 15251 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.107s
I20260709 13:18:43.423529 15141 raft_consensus.cc:740] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:43.437989 15254 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:43.459470 15027 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:43.418887 14934 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.462212 13348 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:43.412848 15370 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:43.484339 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 295419us
I20260709 13:18:43.488606 12598 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:43.475092 15436 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:43.476356 15265 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:43.504736 12597 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:43.485044 15254 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:43.519819 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 78876.1us
W20260709 13:18:43.522810 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.527740 15251 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:43.497381 12527 catalog_manager.cc:5697] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 1 to 3, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 3 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:18:43.528499 15251 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:43.495395 14934 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:43.529191 12527 catalog_manager.cc:5697] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 2 to 6, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 6 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:43.433058 15298 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:43.539201 12634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1255) took 2366 ms. Trace:
I20260709 13:18:43.536880 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:43.495193 15312 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.550484 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:43.532323 15430 log.cc:927] Time spent T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
I20260709 13:18:43.525128 15254 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:43.516191 15436 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:43.557679 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 137787us
I20260709 13:18:43.532887 15442 raft_consensus.cc:3055] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:43.545159 12634 rpcz_store.cc:276] 0709 13:18:41.173337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.394553 (+1221216us) service_pool.cc:224] Handling call
0709 13:18:42.395359 (+   806us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:42.395853 (+   494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:42.395891 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:42.396478 (+   587us) log.cc:844] Serialized 26 byte log entry
0709 13:18:43.044403 (+647925us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:43.044465 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:43.044555 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:43.538913 (+494358us) raft_consensus.cc:1685] finished
0709 13:18:43.538941 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:43.539124 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3527,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":23404,"thread_start_us":647481,"threads_started":1,"wal-append.queue_time_us":1140004}
I20260709 13:18:43.512565 15027 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:43.516499 15265 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:43.565110 12747 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:43.354449 15359 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.364s
I20260709 13:18:43.566048 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:43.531518 15251 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:43.561182 13349 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:43.567641 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 105575us
I20260709 13:18:43.534574 15370 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:43.531589 14934 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:43.564278 12673 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:43.537146 15294 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:43.575981 15027 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:43.559165 15312 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:43.576133 12527 catalog_manager.cc:5697] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 2 to 3, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 3 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:43.578692 15359 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:43.579324 12703 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:43.573590 15265 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:43.615343 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 55254.9us
I20260709 13:18:43.532491 15437 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:43.567153 15442 raft_consensus.cc:740] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:43.627533 12746 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:43.566938 15443 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:43.621256 15370 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:43.635910 12527 catalog_manager.cc:5697] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 1 to 4, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 4 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:43.640555 15437 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:43.641069 15441 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:43.634007 15251 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:43.634063 15443 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.646788 12746 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:43.636190 15359 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:43.648080 15027 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:43.649405 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 85217us
I20260709 13:18:43.595840 15141 consensus_queue.cc:260] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:43.659034 15027 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:43.657958 15359 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:43.648643 15251 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.686107 15251 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:43.647606 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 199584us
I20260709 13:18:43.649005 15443 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:43.685629 15359 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:43.678622 15222 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:43.680271 14934 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:43.645404 15441 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:43.703225 15312 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:43.706274 12527 catalog_manager.cc:5697] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 1 to 4, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 4 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:18:43.690133 15099 consensus_queue.cc:1059] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:18:43.732431 15099 raft_consensus.cc:3055] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:43.732842 15099 raft_consensus.cc:740] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:43.733954 15099 consensus_queue.cc:260] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:43.735702 15099 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:43.736063 15431 log.cc:927] Time spent T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.294s	user 0.012s	sys 0.000s
W20260709 13:18:43.738195 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 58142.4us
I20260709 13:18:43.737285 12672 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:43.724702 14934 raft_consensus.cc:697] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:43.751693 15222 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:43.776340 12672 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:43.780794 12673 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:43.741686 15442 consensus_queue.cc:260] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:43.773881 15447 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:43.749476 15265 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:43.761855 15027 consensus_queue.cc:1059] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:18:43.810045 14934 consensus_queue.cc:237] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:43.811631 15441 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:43.813529 15293 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:43.821363 15293 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.803421 15408 raft_consensus.cc:3055] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:43.803970 15359 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:43.765785 15370 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
W20260709 13:18:43.801357 15445 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
W20260709 13:18:43.796814 15423 log.cc:927] Time spent T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.826s	user 0.000s	sys 0.009s
I20260709 13:18:43.824779 15222 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:43.828478 15252 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:43.824405 15141 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:43.828281 15293 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:43.828119 15408 raft_consensus.cc:740] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:18:43.883203 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 388236us
I20260709 13:18:43.837935 15441 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:18:43.858253 15252 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:43.891803 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 330687us
W20260709 13:18:43.919853 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 442329us
I20260709 13:18:43.920462 12597 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:43.908499 15252 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:43.955312 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.904640 15356 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:43.966055 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:43.970005 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 76072.5us
I20260709 13:18:43.973991 15447 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:43.974824 15447 raft_consensus.cc:697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:43.986562 13349 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:43.990335 14613 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:43.992316 12628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2118) took 1378 ms. Trace:
I20260709 13:18:43.992707 12628 rpcz_store.cc:276] 0709 13:18:42.614115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.937271 (+323156us) service_pool.cc:224] Handling call
0709 13:18:42.973106 (+ 35835us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:43.177703 (+204597us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:43.177738 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:43.225033 (+ 47295us) log.cc:844] Serialized 26 byte log entry
0709 13:18:43.475465 (+250432us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:43.475528 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:43.475629 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:43.992105 (+516476us) raft_consensus.cc:1685] finished
0709 13:18:43.992128 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:43.992259 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":152451,"threads_started":1,"wal-append.queue_time_us":351080}
W20260709 13:18:43.996891 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 221676us
W20260709 13:18:43.997956 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 231023us
W20260709 13:18:44.004199 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:44.004849 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.010839 12672 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:44.017052 12632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2123) took 1397 ms. Trace:
W20260709 13:18:44.022328 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 334912us
W20260709 13:18:44.023605 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 260009us
I20260709 13:18:43.976078 15447 consensus_queue.cc:237] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:44.030588 13349 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:44.012868 15265 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:44.032142 12597 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:44.031790 15359 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:44.064132 12777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1748) took 1486 ms. Trace:
I20260709 13:18:43.978835 15408 consensus_queue.cc:260] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:44.042485 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 278471us
I20260709 13:18:44.033622 15370 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:44.018389 12632 rpcz_store.cc:276] 0709 13:18:42.615678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.172852 (+557174us) service_pool.cc:224] Handling call
0709 13:18:43.215878 (+ 43026us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:43.245946 (+ 30068us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:43.347506 (+101560us) log.cc:844] Serialized 14 byte log entry
0709 13:18:44.012455 (+664949us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:44.012510 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:44.012552 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:44.012614 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:44.012722 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":68402,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":80077,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"thread_start_us":613887,"threads_started":2,"wal-append.queue_time_us":608888}
I20260709 13:18:44.044564 12747 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:44.066666 15294 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:44.031975 15027 consensus_queue.cc:1243] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 639ab2db265f4747bb7afb93e833f530 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:18:44.067878 12673 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:44.083153 15453 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:44.083709 15453 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:44.084100 15453 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:44.078406 15294 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.084766 15294 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:44.078747 15462 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:44.086642 15480 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:44.087201 15480 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:44.085664 15466 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:44.099346 15450 log.cc:927] Time spent T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:18:44.101125 12706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1341) took 1398 ms. Trace:
I20260709 13:18:44.064759 12777 rpcz_store.cc:276] 0709 13:18:42.577543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.180617 (+603074us) service_pool.cc:224] Handling call
0709 13:18:43.221382 (+ 40765us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:43.287474 (+ 66092us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:43.726483 (+439009us) log.cc:844] Serialized 14 byte log entry
0709 13:18:44.063708 (+337225us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:44.063777 (+    69us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:44.063849 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:44.063931 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:44.064063 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":581548,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":12079,"thread_start_us":687848,"threads_started":3,"wal-append.queue_time_us":315523}
I20260709 13:18:44.101943 12630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2122) took 1486 ms. Trace:
W20260709 13:18:44.105608 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 71978.2us
I20260709 13:18:44.105067 12630 rpcz_store.cc:276] 0709 13:18:42.615356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.148859 (+533503us) service_pool.cc:224] Handling call
0709 13:18:43.185797 (+ 36938us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:43.337181 (+151384us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:43.337225 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:43.398717 (+ 61492us) log.cc:844] Serialized 26 byte log entry
0709 13:18:43.813006 (+414289us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b4800002958. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c722dbaa1 00007f4c722d2b37 00007f4c722d29ca 00007f4c723ca26f 00007f4c723c9f37 00007f4c723de55c 00007f4c723de508 00007f4c723de488 00007f4c723de443 00007f4c723dd6ec 00007f4c71307a96
0709 13:18:43.873749 (+ 60743us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:43.873789 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:43.873879 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:44.101653 (+227774us) raft_consensus.cc:1685] finished
0709 13:18:44.101678 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:44.101880 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":39406,"raft.run_cpu_time_us":9145,"raft.run_wall_time_us":498152,"spinlock_wait_cycles":319116032,"thread_start_us":436156,"threads_started":1,"wal-append.queue_time_us":597229}
W20260709 13:18:44.111430 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 67050.9us
I20260709 13:18:44.114867 15298 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:44.117792 15449 log.cc:927] Time spent T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 13:18:44.102201 15359 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:18:44.122825 15470 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:44.123481 15470 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:44.102739 15027 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.125929 15027 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:44.103721 15436 consensus_meta.cc:287] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.521s	user 0.010s	sys 0.002s
I20260709 13:18:44.089433 15480 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:44.102370 12706 rpcz_store.cc:276] 0709 13:18:42.702316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.702500 (+   184us) service_pool.cc:224] Handling call
0709 13:18:43.105818 (+403318us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b5800002798. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c7238bfc0 00007f4c72376929 00007f4c723763d2 00007f4c6b8edbd4
0709 13:18:43.121665 (+ 15847us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:43.428727 (+307062us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:43.428785 (+    58us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:43.439954 (+ 11169us) log.cc:844] Serialized 52 byte log entry
0709 13:18:43.743577 (+303623us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:43.743617 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:43.743689 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:44.100793 (+357104us) raft_consensus.cc:1685] finished
0709 13:18:44.100814 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:44.101058 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38921,"raft.queue_time_us":235378,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"spinlock_wait_cycles":44741888,"thread_start_us":474058,"threads_started":2,"wal-append.queue_time_us":434946}
I20260709 13:18:44.127570 15293 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:44.141023 15443 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:44.142144 15027 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:44.143520 15293 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:44.105291 15466 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:44.147444 15294 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:44.148598 15458 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:18:44.149968 15294 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:44.151026 13348 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:44.153050 15462 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:18:44.153864 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 147815us
I20260709 13:18:44.154661 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:44.153321 13348 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:44.161442 15458 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:44.161640 15293 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:44.162307 15293 raft_consensus.cc:697] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:44.162518 15478 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:44.145612 15436 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:44.163048 15478 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:44.163246 15293 consensus_queue.cc:237] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:44.166532 15436 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:44.167956 12597 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:44.171031 15456 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:44.155604 15467 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:44.156574 15480 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:44.160981 15254 consensus_meta.cc:287] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.548s	user 0.004s	sys 0.008s
W20260709 13:18:44.164096 13349 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 2.199s (SENT) [suppressed 3 similar messages]
W20260709 13:18:44.168617 15386 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.994s	user 0.015s	sys 0.012s
I20260709 13:18:44.149281 15252 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:44.167196 15478 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:44.188580 12597 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:44.179404 15456 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:44.178133 15467 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.194667 15456 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.194756 15467 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:44.195130 15456 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:44.198992 12747 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:44.185416 15386 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.048s	user 0.016s	sys 0.012s
I20260709 13:18:44.194157 15465 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.204653 15386 raft_consensus.cc:697] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:44.204855 15465 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:44.206265 15386 consensus_queue.cc:237] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:44.169576 15470 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:44.207922 15465 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:44.210558 15465 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:44.211191 15465 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.211730 15465 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:44.218073 14934 raft_consensus.cc:3055] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:44.217906 15462 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:18:44.241997 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 53564.7us
I20260709 13:18:44.246435 15489 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.247185 15489 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:44.250123 15489 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:44.202411 15443 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:44.253522 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:44.251247 12597 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:44.183619 15480 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:44.257879 13382 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 6: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:44.257645 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:44.258507 13380 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:44.241104 15470 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:44.184777 15254 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:44.218739 15461 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:18:44.251343 12747 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:44.226792 14934 raft_consensus.cc:740] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:44.258746 12597 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:44.241104 15458 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.267666 15458 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:44.263609 15485 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:44.270294 15485 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:44.271396 15466 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:44.272486 15485 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:44.259450 15470 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.273403 15470 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:44.261147 15489 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:44.273916 15252 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:44.274343 15436 consensus_queue.cc:1059] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:44.274520 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:44.273938 15489 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:44.275661 15489 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:44.264638 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:44.267213 15251 consensus_meta.cc:287] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.579s	user 0.006s	sys 0.005s
I20260709 13:18:44.278508 15251 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:44.280545 15251 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:44.282132 14934 consensus_queue.cc:260] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:44.283416 14934 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:44.262200 15027 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:44.292624 15480 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:44.292930 15027 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:44.295730 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 222280us
I20260709 13:18:44.296052 15027 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:44.297602 15442 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:44.298511 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:44.298882 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 231085us
I20260709 13:18:44.299420 12700 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 6: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:44.300485 12673 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:44.300494 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:44.302968 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 51689.1us
I20260709 13:18:44.282589 12597 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:44.311067 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:44.311944 12703 raft_consensus.cc:2410] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:18:44.307169 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:44.317166 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.305601 15293 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:18:44.326591 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 162532us
I20260709 13:18:44.316257 15099 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:44.328532 15099 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.328611 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.328948 15099 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:44.329281 12700 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:18:44.306778 15466 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:44.334401 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.322341 15437 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:44.338964 15437 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.339285 15474 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:44.339499 15437 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:44.339798 15474 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.340230 15474 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:44.341709 15476 consensus_queue.cc:1059] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:18:44.343375 15356 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:44.343806 15356 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:44.344175 15356 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:44.314082 15451 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:18:44.345981 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.346719 12700 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:44.335996 15443 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:44.335055 12707 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:44.347470 15443 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:44.349035 15470 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:44.350817 15254 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:44.358058 15476 raft_consensus.cc:3055] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:44.358587 15476 raft_consensus.cc:740] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:44.356815 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:44.359573 15476 consensus_queue.cc:260] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:44.360384 15470 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:44.360888 15476 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:44.365223 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.356683 15099 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:44.376928 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.377943 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.378579 12634 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:44.379841 15099 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:44.328096 15293 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.395776 15496 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.395988 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.396768 12703 raft_consensus.cc:2393] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:44.396392 15496 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:44.397372 15478 raft_consensus.cc:3055] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:44.397825 15478 raft_consensus.cc:740] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:44.399062 12627 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:18:44.333467 15466 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:44.394419 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 81693.2us
I20260709 13:18:44.405617 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:44.407025 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.408305 15436 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:44.369982 15443 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:44.410384 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 82828.2us
I20260709 13:18:44.411396 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:44.413137 12703 raft_consensus.cc:2410] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:18:44.414016 15099 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:44.395340 15293 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:44.416950 15496 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:44.411387 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.429869 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.431432 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.432282 12626 raft_consensus.cc:2393] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:44.433192 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.434305 12626 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 3.
I20260709 13:18:44.435616 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.437062 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.439522 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:44.404883 15466 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:44.440941 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.437841 12626 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:18:44.446445 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.451205 12779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1753) took 1466 ms. Trace:
I20260709 13:18:44.421032 15436 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:44.456153 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.451771 12779 rpcz_store.cc:276] 0709 13:18:42.985166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.261521 (+276355us) service_pool.cc:224] Handling call
0709 13:18:43.315208 (+ 53687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:43.422232 (+107024us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:43.783712 (+361480us) log.cc:844] Serialized 14 byte log entry
0709 13:18:44.080717 (+297005us) log.cc:844] Serialized 14 byte log entry
0709 13:18:44.095601 (+ 14884us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:44.095648 (+    47us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:44.095697 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:44.450895 (+355198us) spinlock_profiling.cc:243] Waited 73.4 ms on lock 0x7b540040e554. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:44.450965 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:44.451129 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14637,"prepare.queue_time_us":448624,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":99,"raft.queue_time_us":191755,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":194,"spinlock_wait_cycles":168927616,"thread_start_us":585723,"threads_started":3,"wal-append.queue_time_us":352029}
I20260709 13:18:44.468214 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:44.437458 15312 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:18:44.486773 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.487012 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.487787 15489 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:44.488693 12708 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:44.488822 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.489353 12629 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:18:44.489640 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:44.489921 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.490160 12708 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:44.490190 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.490821 12629 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:44.491991 12629 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:44.487990 12707 raft_consensus.cc:2393] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 7: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:44.424082 15467 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:44.493865 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.494438 12707 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:44.495215 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:44.496165 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.497093 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:44.497579 12707 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:44.485376 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 51844.9us
I20260709 13:18:44.498834 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.499034 12626 raft_consensus.cc:2393] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:44.401765 15453 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:44.487883 15456 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:44.503883 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:44.508790 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.510809 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.462313 15436 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:44.540490 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:44.541287 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.459797 15443 consensus_queue.cc:1048] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:18:44.551718 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.552028 15456 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:44.553313 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.553403 12631 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:44.554692 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:44.555203 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.555292 12705 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:18:44.556252 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:44.556072 15293 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:44.557201 12632 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:18:44.557520 15479 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:44.557962 15479 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:44.558310 15479 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:44.559366 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.559947 12629 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:44.554108 12708 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:44.560770 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.561170 12708 raft_consensus.cc:1240] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:18:44.561612 15490 log.cc:927] Time spent T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:18:44.561933 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.562986 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:44.563949 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.564248 15453 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:44.564916 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:44.566572 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.569430 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:44.575928 12629 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 7: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:44.579734 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:44.581832 12782 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:18:44.582705 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.546974 15436 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:44.608450 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.611162 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:44.606611 12701 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:44.609963 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.617745 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.620070 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:44.625134 12779 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:18:44.625481 12779 pending_rounds.cc:85] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 6
I20260709 13:18:44.625725 12779 pending_rounds.cc:107] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:18:44.625962 12779 raft_consensus.cc:2889] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:44.630594 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.636322 12709 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:18:44.635882 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.638206 12630 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:44.633781 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:44.638780 12629 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:44.540196 15478 consensus_queue.cc:260] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:44.646162 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 235016us
I20260709 13:18:44.708693 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:44.710440 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.711848 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.630654 15356 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:44.670512 12597 connection.cc:256] RPC call timeout handler was delayed by -1.220727474999876s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17310000000000003s for a response
W20260709 13:18:44.723357 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 80317.2us
W20260709 13:18:44.728379 12597 connection.cc:256] RPC call timeout handler was delayed by -1.1516410489998634s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1817s for a response
I20260709 13:18:44.749001 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:44.750934 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.762526 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:44.672194 15293 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:44.777288 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:44.780217 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.785506 12747 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:44.782742 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.786811 12630 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 9: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:44.789155 15441 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:44.791409 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.794195 15480 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:44.794776 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:44.821517 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:44.828797 12632 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:44.823890 15027 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:44.831605 15027 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.832015 15027 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:44.824677 12747 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:44.827852 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 75428.1us
I20260709 13:18:44.835424 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.835847 12597 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:44.792145 12630 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:18:44.764262 15478 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:44.848594 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:44.850191 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:44.824911 15501 log.cc:927] Time spent T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
W20260709 13:18:44.825677 15505 log.cc:927] Time spent T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:18:44.849601 12634 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:44.822721 15458 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:44.865077 12630 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:44.823487 15480 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:44.846380 15500 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:44.861814 15507 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:44.857546 15441 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:44.875228 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:44.873718 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.876300 12701 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:44.880914 12597 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:44.883891 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.873610 15480 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:44.878089 15441 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.879540 15467 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:44.893864 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 633937us
I20260709 13:18:44.894716 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.894886 12746 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:44.883890 15293 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:44.898375 12597 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:44.891503 15507 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:44.901769 15507 raft_consensus.cc:697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:44.903009 15507 consensus_queue.cc:237] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:44.904228 15506 log.cc:927] Time spent T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 13:18:44.892061 15441 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:44.914086 15356 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:44.914795 12626 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:18:44.920749 15437 consensus_meta.cc:287] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.581s	user 0.009s	sys 0.002s
I20260709 13:18:44.921262 15437 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:44.922065 15467 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:44.922703 15467 consensus_queue.cc:1059] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:44.923465 15437 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:44.927757 14934 raft_consensus.cc:3055] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:44.928565 14934 raft_consensus.cc:740] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:44.930238 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.929756 14934 consensus_queue.cc:260] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:44.931717 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:44.932160 15509 log.cc:927] Time spent T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:18:44.932474 12709 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 6: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:44.933264 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:44.933911 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.940470 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.941669 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.942327 12708 raft_consensus.cc:2393] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:44.943059 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.944080 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.945004 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:44.934522 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.946372 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.946653 12703 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 7: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:44.946928 12701 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:44.949723 12629 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:44.936909 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:44.937518 15356 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:44.950517 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:44.955515 12709 raft_consensus.cc:2410] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:18:44.956404 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.957535 12709 raft_consensus.cc:2393] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 7: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:44.936841 12707 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:18:44.963228 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:44.963999 12707 raft_consensus.cc:2393] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 6: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:44.958214 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:44.952988 15441 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:44.967444 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:44.972200 15408 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:44.975152 15465 consensus_meta.cc:287] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.721s	user 0.009s	sys 0.000s
W20260709 13:18:44.975723 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 80926us
I20260709 13:18:45.062252 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:45.063695 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:45.058187 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 139214us
I20260709 13:18:45.064447 13380 raft_consensus.cc:2393] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 7: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:44.990484 12633 raft_consensus.cc:1275] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:44.991755 15479 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:45.070261 15356 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.070777 15356 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:45.071873 15479 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:45.072592 13381 raft_consensus.cc:1240] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:45.074594 13376 raft_consensus.cc:1240] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:45.074865 15441 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:45.075757 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:45.050937 15462 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:18:45.080215 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:45.082067 15436 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:44.995364 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:45.083112 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:44.965937 15293 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
W20260709 13:18:44.957429 15442 consensus_meta.cc:287] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.659s	user 0.008s	sys 0.000s
I20260709 13:18:45.087244 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:45.081063 15480 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:45.099956 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:45.091452 15436 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:45.069118 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:45.082162 15027 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:45.011785 15489 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:45.083630 15465 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:45.090690 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:45.082216 15386 consensus_queue.cc:1059] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:18:45.120692 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:45.121464 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:45.097734 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:18:45.127717 15476 consensus_meta.cc:287] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.766s	user 0.009s	sys 0.005s
I20260709 13:18:45.127550 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:45.143043 12779 raft_consensus.cc:2410] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:18:45.128188 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:45.143628 12774 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:18:45.145290 12705 raft_consensus.cc:2410] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:18:45.128829 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:45.129161 12778 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:45.030463 12632 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
W20260709 13:18:45.103978 15466 consensus_meta.cc:287] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.637s	user 0.011s	sys 0.000s
I20260709 13:18:45.147704 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:45.147905 15466 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:45.149152 12781 raft_consensus.cc:2393] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 9: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:45.153301 12781 raft_consensus.cc:1240] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:45.153846 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:45.154484 12775 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:45.133383 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:45.134290 13379 raft_consensus.cc:2393] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 7: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:45.134721 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:45.159454 15099 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:45.159693 12783 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:45.160094 15099 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:45.136126 12782 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:45.123581 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:18:45.133730 15474 consensus_meta.cc:287] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.793s	user 0.012s	sys 0.000s
I20260709 13:18:45.146296 12777 raft_consensus.cc:2410] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:18:45.146847 12598 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:45.171103 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:45.173105 12629 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 7: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:45.177158 12634 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:18:45.178506 13384 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:45.175941 12597 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:45.188578 12630 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:45.191093 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:45.191934 12774 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:45.145866 13378 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:45.196779 15489 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:45.186076 15468 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:45.197018 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 76453.1us
W20260709 13:18:45.201915 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 55149.8us
I20260709 13:18:45.202463 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:45.204456 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:45.186614 15436 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:18:45.211294 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 144305us
I20260709 13:18:45.186244 15498 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.206179 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:45.206166 13379 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:45.228026 13384 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:18:45.196991 15517 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:45.234612 15517 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:45.235092 15517 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:45.236208 14934 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:45.201366 15500 consensus_queue.cc:1059] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:18:45.204658 15251 consensus_queue.cc:1059] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:18:45.203701 15522 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.238695 15522 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:45.240427 15465 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:45.240720 15522 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:45.242394 15027 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:45.243659 15498 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:45.245111 15356 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:45.233539 15453 consensus_queue.cc:1059] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 13:18:45.182511 15474 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:45.227603 15441 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:45.246567 15441 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:45.246953 15441 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:45.247769 15474 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:45.248334 15479 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:45.222165 15470 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:45.233186 12775 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:18:45.228434 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:45.245796 15498 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:45.189448 15519 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:45.228397 13384 pending_rounds.cc:85] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd: Aborting all ops after (but not including) 7
I20260709 13:18:45.254681 12774 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:45.276770 13384 pending_rounds.cc:107] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:18:45.277158 13384 raft_consensus.cc:2889] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:45.258237 12747 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:45.261090 12783 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:45.263576 12597 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:45.264827 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:45.266001 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:45.285699 12782 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:45.286209 12775 raft_consensus.cc:2393] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 8: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:45.276666 12673 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:45.295189 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:45.301113 12700 raft_consensus.cc:2410] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:18:45.303110 12747 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:45.304623 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:18:45.306123 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 48737.5us
I20260709 13:18:45.306599 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:45.307798 12673 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:45.256592 15440 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.317123 12597 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:45.321221 12597 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:45.319600 12778 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:45.276319 15370 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:45.294401 15479 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:45.305327 12747 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:45.206768 15468 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:45.310472 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:45.326167 13384 raft_consensus.cc:2393] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 9: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:45.324090 12597 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:45.332841 12747 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:45.317293 15523 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:45.318629 15466 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:45.322077 15027 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:45.323251 15451 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:45.325531 15440 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:45.320053 15474 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:45.337703 15462 raft_consensus.cc:3055] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:45.321532 15099 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:45.331398 15442 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:45.324795 15500 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:45.322598 15507 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:45.300545 15251 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:18:45.328003 15479 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:45.330045 15496 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:45.319645 15436 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:18:45.348249 12747 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:45.335682 15480 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:45.363211 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 137299us
I20260709 13:18:45.343456 15523 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.358484 15533 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:45.346764 15489 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:45.352587 15474 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:45.371572 12747 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:45.324502 15453 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:18:45.353901 15462 raft_consensus.cc:740] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:45.362509 15466 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:45.352761 15451 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.362391 15442 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:45.369843 15479 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:45.375520 15027 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:45.334709 15370 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:45.380649 15533 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:45.377444 15523 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:45.383348 15474 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:45.405908 15507 raft_consensus.cc:697] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:45.394621 15451 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:45.396445 15442 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:45.426043 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 50876.4us
I20260709 13:18:45.427383 12598 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:45.428292 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 105151us
I20260709 13:18:45.396747 15439 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:45.433354 12747 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:45.434300 12778 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:45.416524 15533 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:45.399220 15492 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.159s
W20260709 13:18:45.435884 15515 log.cc:927] Time spent T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.238s	user 0.001s	sys 0.001s
I20260709 13:18:45.440358 15436 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:45.465128 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:45.464987 13348 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e
W20260709 13:18:45.467732 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 142402us
I20260709 13:18:45.473408 12673 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:45.482326 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:45.483721 12708 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:45.489883 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:45.490119 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:45.488652 12673 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:45.493477 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:45.498261 15458 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:45.498788 15458 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.499145 15458 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:45.500103 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 72889.9us
I20260709 13:18:45.501459 12598 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:45.507357 12709 raft_consensus.cc:2393] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:45.508451 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:45.474820 15436 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.509035 12704 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:45.469583 15500 raft_consensus.cc:697] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:45.514047 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:45.514628 12709 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:18:45.516592 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:45.517673 12598 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:45.479362 15496 raft_consensus.cc:697] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:45.539249 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:45.541163 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:45.520563 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:45.546358 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 57061.5us
I20260709 13:18:45.546573 12701 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:45.551376 15438 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.551904 15438 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:45.553757 15529 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:45.555521 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:45.557018 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:45.562358 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:45.563977 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:45.564615 12701 raft_consensus.cc:2376] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7. Re-sending same reply.
I20260709 13:18:45.568382 12708 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:18:45.570106 15534 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:45.572038 15534 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:18:45.574343 15528 log.cc:927] Time spent T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:18:45.574465 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:45.575203 12703 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:45.576515 12703 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: [6.7-8.8]
I20260709 13:18:45.577947 12703 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:45.579591 12703 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:45.440570 15439 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:45.565551 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:45.581496 12709 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:18:45.583839 12672 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:45.590713 15370 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:45.584295 12704 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:18:45.584882 15470 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:45.586292 15439 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:45.584753 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:45.587414 12701 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:45.600404 15439 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:45.605777 15468 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:45.611596 12747 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:45.613744 12747 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:45.621021 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:45.622741 12747 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:45.573405 15534 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:45.514211 15436 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:45.706065 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:45.706848 12780 consensus_meta.cc:287] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.535s	user 0.008s	sys 0.000s
I20260709 13:18:45.707166 12780 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:45.629042 15251 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:45.711973 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:45.679073 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:45.679832 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 191147us
W20260709 13:18:45.539515 15515 log.cc:927] Time spent T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:18:45.667821 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 45192.6us
I20260709 13:18:45.601051 15441 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:45.714489 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:45.444506 15462 consensus_queue.cc:260] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:45.550369 15437 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:45.689376 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:45.716987 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:45.690157 15543 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:45.609756 15489 raft_consensus.cc:697] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:45.690546 15536 consensus_queue.cc:1048] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.143s
I20260709 13:18:45.692938 15453 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:45.694622 15532 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:45.726989 15462 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:45.582291 15458 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:45.735891 15458 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:45.696372 15517 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:45.698097 15496 consensus_queue.cc:237] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:45.739593 15555 consensus_queue.cc:1059] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:45.708151 12779 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:18:45.740590 15555 consensus_queue.cc:1059] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:45.628916 15470 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:18:45.702353 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 223199us
I20260709 13:18:45.494484 15099 consensus_queue.cc:1059] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.185s
I20260709 13:18:45.708611 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:45.713449 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:45.685091 12777 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:45.706465 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:45.685885 15546 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:45.685319 15027 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:45.719906 12781 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7. Ops: []
W20260709 13:18:45.678485 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 213556us
I20260709 13:18:45.758927 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:45.689677 15534 raft_consensus.cc:697] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:45.719064 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:45.626861 15529 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:45.724004 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:45.702803 15440 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:45.739995 13378 consensus_meta.cc:287] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.518s	user 0.005s	sys 0.005s
I20260709 13:18:45.699781 12706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1627) took 1402 ms. Trace:
I20260709 13:18:45.712491 15519 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:45.777366 12706 rpcz_store.cc:276] 0709 13:18:44.297673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:44.301648 (+  3975us) service_pool.cc:224] Handling call
0709 13:18:44.301986 (+   338us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:44.402156 (+100170us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:18:44.438945 (+ 36789us) log.cc:844] Serialized 14 byte log entry
0709 13:18:44.607558 (+168613us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:44.624135 (+ 16577us) log.cc:844] Serialized 52 byte log entry
0709 13:18:44.624392 (+   257us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:45.686008 (+1061616us) spinlock_profiling.cc:243] Waited 8 ms on lock 0x7b1800311440. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6cf5a4bc 00007f4c6a2a848b 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c2455 00007f4c70daa3c7 00007f4c6d3bc9a7 00007f4c6d3cc0bf
0709 13:18:45.686142 (+   134us) log.cc:844] Serialized 14 byte log entry
0709 13:18:45.686357 (+   215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:45.686431 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:45.699539 (+ 13108us) raft_consensus.cc:1685] finished
0709 13:18:45.699562 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:45.699723 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52138,"prepare.queue_time_us":71739,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":1869,"raft.queue_time_us":613026,"raft.run_cpu_time_us":341,"raft.run_wall_time_us":4538,"spinlock_wait_cycles":18403584,"thread_start_us":293116,"threads_started":3,"wal-append.queue_time_us":176910}
I20260709 13:18:45.747412 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:45.759754 12703 raft_consensus.cc:2410] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:18:45.782039 12774 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:45.508222 15507 consensus_queue.cc:237] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:45.766484 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:45.757419 15458 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.767040 12779 raft_consensus.cc:606] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Received request to transfer leadership to TS 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:45.789933 12779 raft_consensus.cc:614] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:18:45.767530 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 48524.3us
I20260709 13:18:45.775630 12776 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 7: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:45.791090 12747 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:45.776552 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:45.792550 12781 raft_consensus.cc:2393] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
W20260709 13:18:45.793771 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 102333us
I20260709 13:18:45.741791 15470 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:45.771271 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:45.721315 15543 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:45.741521 15555 raft_consensus.cc:3055] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:45.786736 12633 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:45.786618 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:45.777323 13378 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:45.805717 15494 raft_consensus.cc:3055] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:45.801415 15458 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:45.806150 15494 raft_consensus.cc:740] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:45.808573 15494 consensus_queue.cc:260] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:45.800637 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:45.801606 12704 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:45.813675 12747 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:45.795496 12597 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:45.819591 12747 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:45.763480 15437 raft_consensus.cc:697] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:45.795810 13384 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:45.777377 14934 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:18:45.827016 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 309481us
I20260709 13:18:45.833909 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:45.760205 15453 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:45.836841 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:45.845842 12598 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:45.810407 15470 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:45.811357 15555 raft_consensus.cc:740] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:45.816938 15494 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:45.856361 12598 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:45.788920 15438 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:45.812536 15543 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:45.764038 15523 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:45.820627 15027 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:45.825441 15251 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:18:45.867800 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 142825us
I20260709 13:18:45.587400 15500 consensus_queue.cc:237] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:45.831218 15529 consensus_queue.cc:1059] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:18:45.851421 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:45.841432 14934 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:45.855223 15447 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:18:45.915488 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:45.915879 15370 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:45.904924 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:45.917531 13384 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:45.797333 15549 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 13:18:45.858982 15497 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:18:45.916859 15538 log.cc:927] Time spent T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.382s	user 0.000s	sys 0.001s
I20260709 13:18:45.860647 15466 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:18:45.921433 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 155433us
I20260709 13:18:45.927451 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:45.870993 15027 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:45.930012 12746 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:45.925880 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 69602.5us
W20260709 13:18:45.929877 15479 consensus_meta.cc:287] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.510s	user 0.007s	sys 0.004s
I20260709 13:18:45.946808 12703 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:18:45.950208 13377 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:45.926573 15466 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.925689 15447 raft_consensus.cc:697] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:45.954155 13383 raft_consensus.cc:1240] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:18:45.935622 15027 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:45.936167 15496 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:45.862309 15489 consensus_queue.cc:237] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:45.916893 15370 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:45.961508 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:45.963788 15525 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:45.967540 12672 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:45.972306 12598 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:45.982209 12598 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:45.940282 15480 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:18:45.956959 15466 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:45.966348 15441 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:45.905393 15534 consensus_queue.cc:237] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:45.955796 15479 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:45.980129 15467 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:45.992027 12598 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:45.946334 15437 consensus_queue.cc:237] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:45.996342 15566 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:45.980204 15532 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:46.039744 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:46.040397 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:46.040836 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 48854.7us
I20260709 13:18:46.066435 12598 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:46.072377 12598 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:46.073493 15386 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:18:46.073217 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 143197us
W20260709 13:18:46.041455 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 196617us
I20260709 13:18:45.987936 15561 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:46.049682 15453 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:46.050422 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:46.038679 15251 raft_consensus.cc:697] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:45.995378 15099 consensus_queue.cc:1059] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:18:46.063761 12775 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:46.054740 15497 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:46.053900 15523 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:46.043363 15467 raft_consensus.cc:697] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:46.080011 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:45.957507 15555 consensus_queue.cc:260] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:46.093199 15555 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:46.093931 15479 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:45.983672 15525 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.095750 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:46.096624 12779 raft_consensus.cc:2393] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 9: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:46.097811 15525 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:46.084502 12598 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:46.076799 12747 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:46.091068 15467 consensus_queue.cc:237] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.097489 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:46.070187 15566 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.110730 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:46.111997 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:46.112143 15440 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.112653 15566 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:46.049734 15447 consensus_queue.cc:237] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:46.112759 15440 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:46.115067 15440 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:46.115211 15543 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.115597 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:46.112675 12779 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:18:46.117578 12782 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7. Ops: []
W20260709 13:18:46.121076 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 44409.1us
I20260709 13:18:46.122004 12747 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:46.154834 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 70408.7us
I20260709 13:18:46.155732 12598 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:46.157976 15559 log.cc:927] Time spent T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:18:46.115797 15568 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:46.168341 15451 consensus_meta.cc:287] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.732s	user 0.008s	sys 0.000s
W20260709 13:18:46.190203 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 117444us
W20260709 13:18:46.168637 15474 consensus_meta.cc:287] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.744s	user 0.008s	sys 0.000s
I20260709 13:18:46.191108 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:46.192780 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:46.198987 13349 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:46.122360 15545 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:46.098169 15544 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:46.157492 12598 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:46.208817 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 51411.6us
I20260709 13:18:46.210080 12598 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:46.122512 15557 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:46.089087 15575 consensus_queue.cc:1059] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.093s
I20260709 13:18:46.182116 15497 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:46.182925 15543 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:46.153663 15479 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:46.183812 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:46.133088 15441 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.122625 15553 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.203536 13349 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:46.206053 12705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1641) took 1656 ms (client timeout 2150 ms). Trace:
I20260709 13:18:46.215080 15545 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:46.261821 15545 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:46.208397 15442 consensus_meta.cc:287] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.765s	user 0.013s	sys 0.000s
I20260709 13:18:46.265740 15552 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:46.265910 15442 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.266180 15027 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:46.266898 15553 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.268199 15027 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:46.268312 15442 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:46.259603 12705 rpcz_store.cc:269] 0709 13:18:44.549950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:44.556051 (+  6101us) service_pool.cc:224] Handling call
0709 13:18:46.201423 (+1645372us) spinlock_profiling.cc:243] Waited 588 ms on lock 0x7b54001ccad4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:46.205968 (+  4545us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1352685824}
I20260709 13:18:46.235375 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:46.197183 15492 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:46.161293 15470 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:46.199760 15451 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:46.151238 14934 raft_consensus.cc:697] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:46.163038 15568 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:46.253417 12778 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
W20260709 13:18:46.219372 15533 consensus_meta.cc:287] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.746s	user 0.008s	sys 0.003s
I20260709 13:18:46.130283 15574 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:18:46.260895 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.251953 15544 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:46.200701 15474 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:46.191048 15436 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:46.277963 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:46.270506 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:46.277482 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:46.293849 12702 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:46.271052 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 67595.7us
W20260709 13:18:46.303789 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.255777 15581 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.306253 12627 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:18:46.282829 15370 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:46.307977 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:46.269513 15553 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:46.280894 15479 raft_consensus.cc:697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:18:46.311908 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.275167 15470 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:46.269796 15524 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:46.332160 12783 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:46.282269 15525 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:46.332553 13349 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:46.278688 15523 consensus_queue.cc:1059] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:18:46.280018 15441 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:46.348778 13349 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:46.307003 15574 raft_consensus.cc:697] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:46.253798 15251 consensus_queue.cc:237] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.353777 13349 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:46.360388 15574 consensus_queue.cc:237] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:46.362376 15521 log.cc:927] Time spent T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 1.144s	user 0.006s	sys 0.006s
W20260709 13:18:46.364445 12700 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1639) took 1915 ms (client timeout 1751 ms). Trace:
W20260709 13:18:46.364938 12700 rpcz_store.cc:269] 0709 13:18:44.448877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:44.449005 (+   128us) service_pool.cc:224] Handling call
0709 13:18:46.364101 (+1915096us) spinlock_profiling.cc:243] Waited 826 ms on lock 0x7b54001d8b54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:46.364359 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1899458944}
I20260709 13:18:46.366873 15465 raft_consensus.cc:3055] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:46.367367 15465 raft_consensus.cc:740] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:46.368460 15465 consensus_queue.cc:260] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:46.298736 15533 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.374811 15536 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:46.375824 15533 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:46.376789 12709 raft_consensus.cc:2393] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:46.377068 15536 consensus_queue.cc:1059] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:46.378190 15579 consensus_queue.cc:1059] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:18:46.378839 12672 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:46.379173 15579 raft_consensus.cc:3055] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:46.379695 15579 raft_consensus.cc:740] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:46.380028 15533 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:46.332552 15581 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.385417 15581 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:46.386368 15583 log.cc:927] Time spent T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:18:46.387692 14934 consensus_queue.cc:237] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.388664 15466 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:46.390918 15466 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:46.350018 15370 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.356315 15524 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:18:46.408346 15578 log.cc:927] Time spent T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:18:46.408685 15524 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:46.409255 15524 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:46.411984 15436 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:46.351346 15462 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:46.414070 15462 raft_consensus.cc:697] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 9 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:46.415238 15557 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:46.415356 15462 consensus_queue.cc:237] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.352886 15565 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:46.417204 15568 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:46.358403 15461 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:18:46.417191 15479 consensus_queue.cc:237] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:46.393760 12778 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:46.386188 12672 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:46.420579 12702 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:18:46.403548 15545 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:46.403604 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:46.424988 12703 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:46.407085 15441 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:46.407701 15370 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.422016 12747 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:46.422385 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:46.430948 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:46.434691 13380 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:46.436384 12746 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:46.438262 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:46.439807 12672 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:46.439992 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:46.441357 12672 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:46.443006 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:46.440914 12700 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:46.446470 13385 raft_consensus.cc:2393] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 7: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:46.400620 15465 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:46.380674 15579 consensus_queue.cc:260] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.458473 15561 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:46.459317 15579 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:46.460103 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 247164us
I20260709 13:18:46.467865 15370 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:46.442940 15461 consensus_queue.cc:1059] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:18:46.451563 15466 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:46.475399 15461 raft_consensus.cc:3055] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:46.475876 15461 raft_consensus.cc:740] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:46.476303 15474 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:46.432554 15568 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:46.477679 15568 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:46.478179 15568 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:46.487398 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:46.446310 15441 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:46.494511 12702 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:46.486223 15466 raft_consensus.cc:697] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:46.477016 15461 consensus_queue.cc:260] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.516218 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:46.490753 12598 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:46.444451 15451 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:46.451023 15533 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:46.479302 15458 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:46.492779 15441 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:46.521445 12706 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:18:46.494252 15561 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.517536 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:46.527045 12598 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:46.530220 12703 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:18:46.531453 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:46.532403 12706 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:46.536242 15592 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:18:46.537276 15494 consensus_meta.cc:287] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.669s	user 0.011s	sys 0.000s
I20260709 13:18:46.522982 15534 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:46.525966 15441 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:46.559863 12598 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:46.561189 12598 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:46.562669 12598 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:46.564023 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:46.564378 15447 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:46.564716 12781 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:46.565138 15447 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:46.565748 15456 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:46.547000 15592 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:46.525132 15466 consensus_queue.cc:237] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:46.574394 13376 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:18:46.574560 15545 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:46.576313 12781 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
I20260709 13:18:46.577549 13348 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:46.593708 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:46.595459 13348 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:46.596591 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:46.598573 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:46.609213 12706 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:18:46.610914 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:46.620858 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:46.624001 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:46.625209 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:46.626469 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:46.593231 15386 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:46.628690 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:46.629259 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:46.663458 12632 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:46.629522 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:46.629840 15565 raft_consensus.cc:3055] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:46.631109 12777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36488 (request call id 1769) took 1365 ms. Trace:
I20260709 13:18:46.534444 15561 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.664397 12777 rpcz_store.cc:276] 0709 13:18:45.266159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.288093 (+ 21934us) service_pool.cc:224] Handling call
0709 13:18:46.630925 (+1342832us) spinlock_profiling.cc:243] Waited 378 ms on lock 0x7b54000cbf54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:46.631045 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":868867968}
I20260709 13:18:46.634550 15438 raft_consensus.cc:3055] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:46.635948 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:46.638339 15525 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:46.666335 15438 raft_consensus.cc:740] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:46.665221 15565 raft_consensus.cc:740] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:46.622251 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:46.559217 15534 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.682901 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:46.683497 12634 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:46.683501 12747 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:46.683957 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:46.684659 15534 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:46.665629 15386 raft_consensus.cc:697] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:46.685508 15466 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:46.643746 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:46.687106 12633 raft_consensus.cc:2393] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 7: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:46.685884 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:46.687898 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:46.688583 12633 raft_consensus.cc:2393] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:46.689108 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:46.648222 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:46.689992 12629 raft_consensus.cc:2393] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:46.690191 15466 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:46.690752 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:46.691304 12629 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:46.691636 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:46.691910 15561 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:46.692660 15466 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:46.653213 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:46.693775 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:46.694453 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:46.654870 15519 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:46.667538 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:46.695796 12631 raft_consensus.cc:2393] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 9: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:46.667644 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:46.696259 12783 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:46.672168 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:46.672812 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:46.674093 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:46.697096 12628 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:46.697376 12630 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:46.674643 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:18:46.675127 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.675750 12625 raft_consensus.cc:2410] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:18:46.674096 15438 consensus_queue.cc:260] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:46.624409 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:46.679850 15525 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.701915 12775 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:18:46.689500 15524 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.714015 15386 consensus_queue.cc:237] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.715260 15524 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:46.716622 15480 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:46.717216 15480 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:46.718153 15558 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:46.689296 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:46.719408 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:46.719959 12633 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:46.642405 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:46.647341 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:46.720754 12781 raft_consensus.cc:2393] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 8: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:46.720971 12706 raft_consensus.cc:2410] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:18:46.703269 15525 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.704191 15438 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:46.702267 15519 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:46.705080 12626 raft_consensus.cc:2393] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:46.706522 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:46.729095 12631 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 9: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:46.729465 12629 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:18:46.708331 12783 raft_consensus.cc:1240] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:46.708202 15568 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.732965 12781 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:46.712783 15441 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.722837 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:46.739436 12775 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:46.740944 12775 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 7. Current term is 8. Ops: [6.6-7.7]
I20260709 13:18:46.722005 12747 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:46.724278 12628 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:46.737150 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:46.744062 12631 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:46.746575 12775 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:18:46.747903 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:46.748811 12629 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:46.751796 12781 raft_consensus.cc:1275] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:46.717233 15560 raft_consensus.cc:3055] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:46.734812 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:46.732815 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:46.763113 12626 raft_consensus.cc:2410] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:18:46.700796 15466 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:18:46.755054 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.725272 12632 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:18:46.753436 12781 raft_consensus.cc:1275] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:46.758534 12706 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 13:18:46.731348 15529 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.780s	user 0.013s	sys 0.012s
I20260709 13:18:46.722630 15099 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:18:46.770572 15529 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.013s	sys 0.013s
W20260709 13:18:46.770941 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 49028.9us
I20260709 13:18:46.772266 12747 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:46.773352 15529 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:46.773509 12747 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:46.773833 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:46.774384 15251 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:46.775460 12628 raft_consensus.cc:1240] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:18:46.777988 12783 raft_consensus.cc:1240] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:46.780625 12631 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
W20260709 13:18:46.781587 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.781704 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:46.782836 15600 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:46.783389 15600 raft_consensus.cc:697] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:46.784271 15600 consensus_queue.cc:237] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:46.766358 12781 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:46.767815 15575 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:46.768471 12700 raft_consensus.cc:2393] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 8: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:46.790679 12701 raft_consensus.cc:1240] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:18:46.769696 12706 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:18:46.791638 12706 pending_rounds.cc:85] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 6
I20260709 13:18:46.792121 12706 pending_rounds.cc:107] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:18:46.792894 12706 raft_consensus.cc:2889] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:46.793586 12777 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:18:46.798831 15356 raft_consensus.cc:3055] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:46.799392 15356 raft_consensus.cc:740] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:46.800302 15356 consensus_queue.cc:260] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:46.801086 15525 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:46.802006 12780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2022) took 1078 ms. Trace:
I20260709 13:18:46.802423 12780 rpcz_store.cc:276] 0709 13:18:45.723495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.724640 (+  1145us) service_pool.cc:224] Handling call
0709 13:18:45.725171 (+   531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:45.781018 (+ 55847us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:45.781051 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:45.781088 (+    37us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:45.781121 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:46.801761 (+1020640us) spinlock_profiling.cc:243] Waited 55.6 ms on lock 0x7b540034a654. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:46.801812 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:46.801938 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":127900544}
I20260709 13:18:46.803767 12634 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:46.806356 12628 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:46.792104 12747 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:46.809769 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:46.811393 12633 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:46.813158 13377 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:46.815752 12626 raft_consensus.cc:1240] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:46.816118 12747 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:46.761627 15560 raft_consensus.cc:740] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:46.829923 12747 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:46.822093 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:46.833381 13349 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:46.835502 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:46.819909 15558 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.818849 15479 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:46.812216 15551 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:46.838655 15558 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.839380 15524 consensus_queue.cc:1059] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:46.839649 15551 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.840487 15524 raft_consensus.cc:3055] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:46.840869 15524 raft_consensus.cc:740] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:46.838755 15251 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:46.841745 15524 consensus_queue.cc:260] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.842425 15251 raft_consensus.cc:697] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:46.842519 15551 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:46.843039 15524 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:46.843446 15251 consensus_queue.cc:237] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.844220 15568 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:46.844359 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:46.838613 15579 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:46.845000 13377 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:46.819269 15466 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:46.820101 15575 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:46.850154 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:46.850783 12633 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:46.801497 15356 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:46.838658 15467 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:46.846488 15579 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:46.818928 15574 consensus_queue.cc:1059] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:18:46.860595 12707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1367) took 1389 ms. Trace:
I20260709 13:18:46.841555 15558 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:46.842667 15441 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:46.832810 15529 consensus_queue.cc:1059] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:18:46.848302 15480 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:46.839383 15479 raft_consensus.cc:697] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:46.873003 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:46.856105 15466 raft_consensus.cc:697] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:46.856827 15579 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:46.849649 15462 consensus_queue.cc:1048] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:18:46.857086 15575 raft_consensus.cc:697] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:46.866331 15467 raft_consensus.cc:697] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:46.903416 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:46.917162 12781 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:18:46.906267 12704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1681) took 1186 ms. Trace:
I20260709 13:18:46.855520 15458 consensus_queue.cc:1059] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
W20260709 13:18:46.919739 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 82864.6us
I20260709 13:18:46.921224 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:46.918087 12704 rpcz_store.cc:276] 0709 13:18:45.719981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.811921 (+ 91940us) service_pool.cc:224] Handling call
0709 13:18:45.828084 (+ 16163us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:45.828343 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:45.867931 (+ 39588us) log.cc:844] Serialized 14 byte log entry
0709 13:18:46.813999 (+946068us) spinlock_profiling.cc:243] Waited 2.15 ms on lock 0x7b18004105c0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf5223a 00007f4c6d3bca2a
0709 13:18:46.904929 (+ 90930us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:46.905283 (+   354us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:46.905331 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:46.905400 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:46.906196 (+   796us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24766,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":52,"raft.queue_time_us":420,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":2739,"spinlock_wait_cycles":4936960,"thread_start_us":89656,"threads_started":1}
I20260709 13:18:46.925932 13348 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:46.915849 15441 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:46.925060 15522 raft_consensus.cc:3055] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:46.951082 15467 consensus_queue.cc:237] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:46.954985 12704 raft_consensus.cc:3055] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:46.955418 12704 raft_consensus.cc:740] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:46.955456 12703 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:18:46.956319 12704 consensus_queue.cc:260] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:46.957600 12704 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:46.957818 15466 consensus_queue.cc:237] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:46.868449 15560 consensus_queue.cc:260] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:46.971232 15575 consensus_queue.cc:237] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:46.972957 12633 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:46.976941 12704 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:18:46.930184 15550 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:46.979525 15550 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:46.979975 15550 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:46.928442 15441 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:46.981978 15441 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:46.999030 15479 consensus_queue.cc:237] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:46.997536 12630 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:47.007045 13377 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:18:46.987452 15522 raft_consensus.cc:740] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:18:47.015744 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:47.031541 15599 log.cc:927] Time spent T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.425s	user 0.002s	sys 0.000s
I20260709 13:18:47.034721 12746 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:47.037640 12672 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:47.038640 12672 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:47.041018 12672 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:47.041815 15597 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:47.042265 15597 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:47.042606 15597 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:47.045428 15141 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:47.046090 15141 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:47.046444 15141 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:47.008837 15558 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:47.046758 15534 consensus_queue.cc:1059] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:47.047230 15558 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:47.047788 15534 consensus_queue.cc:1059] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:47.048583 15534 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:47.049222 15534 raft_consensus.cc:3055] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:47.049383 15554 raft_consensus.cc:3055] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:47.049858 15554 raft_consensus.cc:740] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:47.050598 15554 consensus_queue.cc:260] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:47.051728 12702 raft_consensus.cc:1240] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:47.051760 15554 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:47.053437 12781 raft_consensus.cc:1240] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:47.054052 12774 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36478 (request call id 2019) took 1367 ms. Trace:
I20260709 13:18:47.054399 12774 rpcz_store.cc:276] 0709 13:18:45.686727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.686948 (+   221us) service_pool.cc:224] Handling call
0709 13:18:47.053910 (+1366962us) spinlock_profiling.cc:243] Waited 24.1 ms on lock 0x7b540044bbd4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:47.054001 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55340288}
I20260709 13:18:47.056111 12708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1378) took 1539 ms. Trace:
I20260709 13:18:47.056466 12708 rpcz_store.cc:276] 0709 13:18:45.516622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.569403 (+ 52781us) service_pool.cc:224] Handling call
0709 13:18:47.055933 (+1486530us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54003ebcd4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:47.056053 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2417193856}
I20260709 13:18:47.057838 15534 raft_consensus.cc:740] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:47.051643 12709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1675) took 1344 ms. Trace:
I20260709 13:18:47.049412 15558 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:47.059265 12709 rpcz_store.cc:276] 0709 13:18:45.707253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.713168 (+  5915us) service_pool.cc:224] Handling call
0709 13:18:47.051434 (+1338266us) spinlock_profiling.cc:243] Waited 618 ms on lock 0x7b540014b754. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:47.051584 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1422322560}
I20260709 13:18:47.061239 12774 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:47.063453 12774 raft_consensus.cc:1240] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:47.067703 12780 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:47.067976 15522 consensus_queue.cc:260] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:47.069767 15522 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:47.071210 15557 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:47.071985 15557 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:47.074318 15557 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:47.058897 15534 consensus_queue.cc:260] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:47.075976 12597 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:47.076990 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:47.078186 15603 log.cc:927] Time spent T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 13:18:47.080638 12774 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:18:47.053324 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:47.084034 12597 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:47.092872 15557 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:47.093863 15551 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:47.094291 15551 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:47.094686 15551 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:47.095223 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:47.095978 12781 raft_consensus.cc:2393] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 8: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:47.098151 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:47.100780 15441 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:47.101296 12782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2017) took 1420 ms. Trace:
I20260709 13:18:47.101735 12782 rpcz_store.cc:276] 0709 13:18:45.680921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.681186 (+   265us) service_pool.cc:224] Handling call
0709 13:18:46.117358 (+436172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:46.117964 (+   606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:47.101077 (+983113us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b540044c0d0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:47.101220 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":986303744}
I20260709 13:18:47.102254 15557 raft_consensus.cc:697] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:47.102434 15141 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:47.103188 15557 consensus_queue.cc:237] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:47.104558 15478 raft_consensus.cc:3055] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:47.104957 15478 raft_consensus.cc:740] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:47.106035 12703 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:18:47.106491 12703 pending_rounds.cc:85] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 3
I20260709 13:18:47.106731 12703 pending_rounds.cc:107] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:18:47.101316 12627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 346) took 1342 ms. Trace:
I20260709 13:18:47.107192 12703 raft_consensus.cc:2889] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:47.107242 12627 rpcz_store.cc:276] 0709 13:18:45.759174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.759437 (+   263us) service_pool.cc:224] Handling call
0709 13:18:45.781444 (+ 22007us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:46.325774 (+544330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:47.101080 (+775306us) spinlock_profiling.cc:243] Waited 523 ms on lock 0x7b54002db9d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:47.101259 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1201883520}
W20260709 13:18:47.108007 15606 log.cc:927] Time spent T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:18:47.109364 15542 consensus_queue.cc:1059] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:47.109510 15548 consensus_queue.cc:1059] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:47.110167 15542 consensus_queue.cc:1243] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Peer 3d539f8d95d343d8b1696803f18b30d2 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:18:47.110625 15592 raft_consensus.cc:3055] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:47.110700 15542 raft_consensus.cc:3055] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:47.111099 15592 raft_consensus.cc:740] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:47.111320 15542 raft_consensus.cc:740] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:47.112092 15592 consensus_queue.cc:260] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:47.112466 15542 consensus_queue.cc:260] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:47.113848 15542 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:47.113955 15523 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:18:47.114181 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:47.114645 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.114588 15523 raft_consensus.cc:697] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:47.116226 15456 consensus_queue.cc:1059] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:47.117729 15453 raft_consensus.cc:3055] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:47.118208 15453 raft_consensus.cc:740] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:47.115549 15523 consensus_queue.cc:237] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:47.118944 15453 consensus_queue.cc:260] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:47.121907 15441 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:47.122431 12778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36478 (request call id 2040) took 1003 ms. Trace:
I20260709 13:18:47.122938 12778 rpcz_store.cc:276] 0709 13:18:46.118654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:46.291245 (+172591us) service_pool.cc:224] Handling call
0709 13:18:47.122267 (+831022us) spinlock_profiling.cc:243] Waited 67.1 ms on lock 0x7b54001fe354. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:47.122376 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":154390272}
I20260709 13:18:47.122543 12779 raft_consensus.cc:2393] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 8: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:47.105965 15478 consensus_queue.cc:260] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:47.124573 15517 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:47.125340 15478 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:47.127244 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:47.127838 12776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36478 (request call id 2015) took 1449 ms. Trace:
I20260709 13:18:47.128182 12776 rpcz_store.cc:276] 0709 13:18:45.678112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.678729 (+   617us) service_pool.cc:224] Handling call
0709 13:18:47.127657 (+1448928us) spinlock_profiling.cc:243] Waited 61.2 ms on lock 0x7b540042bdd4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:47.127787 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":140661120}
I20260709 13:18:47.130270 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:47.132633 13381 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:47.134469 15461 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:47.135989 12780 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:18:47.104586 15141 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:47.138815 15517 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:47.113286 15592 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:47.148897 15551 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:47.151042 15519 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:47.107342 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.149585 15560 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:47.153223 12597 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:47.184422 15517 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:18:47.187172 15500 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:47.190294 15567 consensus_queue.cc:1059] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:18:47.190920 15442 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:47.191455 15495 consensus_queue.cc:1059] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:47.192493 15495 raft_consensus.cc:3055] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:47.193013 15495 raft_consensus.cc:740] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:47.192363 15141 consensus_queue.cc:1048] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:47.216295 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:47.218807 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:47.219118 12634 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7. Ops: [5.3-6.4]
I20260709 13:18:47.219636 12630 raft_consensus.cc:2410] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:18:47.247623 15141 consensus_queue.cc:1243] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer 3d539f8d95d343d8b1696803f18b30d2 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:18:47.201768 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:47.249027 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 51513.3us
I20260709 13:18:47.254467 12777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1770) took 1459 ms. Trace:
I20260709 13:18:47.255167 12777 rpcz_store.cc:276] 0709 13:18:45.795288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:46.705430 (+910142us) service_pool.cc:224] Handling call
0709 13:18:46.705792 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:46.794037 (+ 88245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:47.253948 (+459911us) spinlock_profiling.cc:243] Waited 87.5 ms on lock 0x7b540011c954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:47.254375 (+   427us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":201239936}
I20260709 13:18:47.075997 15534 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:47.215133 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:47.258673 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:47.260862 12626 raft_consensus.cc:2393] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 5: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:18:47.261734 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.265587 12628 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 4. Current term is 5. Ops: []
W20260709 13:18:47.269708 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.269265 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:47.270596 12776 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:47.278663 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:47.284942 12783 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:47.290527 13382 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:47.190965 15551 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:47.294906 12625 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:47.298882 12628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:18:47.300020 12628 raft_consensus.cc:606] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Received request to transfer leadership to TS 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:47.300418 12628 raft_consensus.cc:614] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:47.295051 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:18:47.308368 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 315519us
I20260709 13:18:47.318476 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:47.325124 12626 raft_consensus.cc:1240] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:47.297323 15597 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:47.325528 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:47.335574 12627 raft_consensus.cc:1240] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:47.336526 13349 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:47.343122 12776 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:47.343902 12777 raft_consensus.cc:1240] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:18:47.346510 13349 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:47.343489 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:47.352680 12598 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:47.283123 15312 consensus_queue.cc:1059] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:18:47.358124 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:47.318449 15141 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:47.354095 12781 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:18:47.354135 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:47.355536 12598 leader_election.cc:336] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 2.223s (SENT)
I20260709 13:18:47.361714 12598 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:47.355664 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:47.343039 15436 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:47.284238 15566 consensus_queue.cc:1059] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:18:47.359004 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:47.318737 15517 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:47.346074 15474 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:47.370729 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:47.262394 15550 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:47.360139 15141 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:47.368089 13349 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:47.345813 15536 raft_consensus.cc:2764] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:18:47.362725 15453 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:47.377022 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.358522 15548 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:47.367635 15551 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:47.397075 12701 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:47.397146 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:47.370329 12776 raft_consensus.cc:2393] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:47.388398 12774 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:47.393184 12598 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:47.394691 12709 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:18:47.379341 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:47.346477 15386 consensus_queue.cc:1059] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:18:47.400111 12627 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:18:47.369581 15567 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:47.404098 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:47.406219 12776 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:47.375566 15524 consensus_queue.cc:1059] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:18:47.377063 15517 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:47.409586 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:47.410444 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:47.410837 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:47.411038 15517 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:47.411934 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:47.412751 13382 raft_consensus.cc:2410] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:18:47.412858 12782 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:47.413385 12630 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 13:18:47.413775 12630 pending_rounds.cc:85] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 7
I20260709 13:18:47.413826 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:47.414215 12630 pending_rounds.cc:107] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:18:47.414484 12630 raft_consensus.cc:2889] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:47.416806 12708 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:18:47.368683 15495 consensus_queue.cc:260] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:47.418010 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:47.418500 15495 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:47.422063 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:47.423164 15519 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:47.424407 15254 consensus_queue.cc:1059] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:18:47.425514 15254 raft_consensus.cc:3055] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:47.425907 15254 raft_consensus.cc:740] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:47.426112 13378 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:47.390738 15474 raft_consensus.cc:697] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:47.426810 15254 consensus_queue.cc:260] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:47.427717 15474 consensus_queue.cc:237] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:47.385473 15141 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:47.426836 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:47.432528 12783 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:47.433293 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:47.434175 12628 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:47.434803 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:47.435195 12631 raft_consensus.cc:1240] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:47.435510 12634 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:18:47.408924 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:47.402254 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:18:47.403741 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:47.438117 12781 raft_consensus.cc:2393] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:47.442039 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:47.444417 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:47.445218 12632 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:18:47.393488 15312 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:47.369270 15436 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:47.405086 15500 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:47.393539 15535 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:47.456997 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.472965 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:18:47.478974 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.484972 12632 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:18:47.445815 15254 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:47.451493 15579 consensus_meta.cc:287] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.537s	user 0.006s	sys 0.003s
I20260709 13:18:47.490840 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:47.495312 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:47.494050 12783 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:18:47.496863 12631 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:47.497344 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:47.498744 12782 raft_consensus.cc:1240] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:47.499428 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:47.499993 12631 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:47.500598 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.502200 12633 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:18:47.477703 15566 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.503947 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:47.443817 15465 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:18:47.500895 12672 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:47.505087 12703 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:18:47.498911 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:47.498098 12783 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:47.507838 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:47.512629 12598 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:47.512989 12628 raft_consensus.cc:1240] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:47.509820 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:47.513751 12629 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:47.510208 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:47.518004 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:47.507246 15496 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 13:18:47.525804 15555 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:47.526630 15550 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:47.527643 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.492067 15438 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.536877 13383 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:47.537921 15551 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:18:47.538554 15622 log.cc:927] Time spent T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:18:47.530186 12782 pending_rounds.cc:85] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 3
I20260709 13:18:47.539748 12782 pending_rounds.cc:107] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:18:47.529618 15566 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:47.540060 12782 raft_consensus.cc:2889] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:47.540586 15099 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:47.541060 15099 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:47.526654 15555 raft_consensus.cc:697] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:18:47.543308 15621 log.cc:927] Time spent T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:18:47.504163 15437 consensus_queue.cc:1059] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:18:47.542743 15555 consensus_queue.cc:237] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:47.549453 15437 consensus_queue.cc:1243] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer 3d539f8d95d343d8b1696803f18b30d2 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:18:47.549957 15451 raft_consensus.cc:3055] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:47.489833 15517 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:18:47.550336 15451 raft_consensus.cc:740] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:47.551316 15438 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:47.553802 12775 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:18:47.551275 15451 consensus_queue.cc:260] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:47.557597 12629 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
W20260709 13:18:47.564472 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.560915 12631 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:18:47.556214 12673 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:47.506126 15535 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:47.571600 12673 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:47.492084 15557 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 13:18:47.578614 12673 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:47.507514 15545 consensus_queue.cc:1059] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:18:47.570422 15437 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.507513 15579 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:47.616945 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:47.622382 12783 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:18:47.579882 15440 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:18:47.631070 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.616695 15370 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:47.619084 15451 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:47.579782 15554 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:47.627219 15536 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:47.630774 15312 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:47.621599 15437 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:47.678990 15579 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:47.683355 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 58982.1us
I20260709 13:18:47.695813 12673 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:47.698385 15626 log.cc:927] Time spent T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:18:47.700387 15141 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:47.701069 15468 raft_consensus.cc:3055] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:47.701653 15468 raft_consensus.cc:740] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:47.703557 12702 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:47.668618 15312 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:47.706429 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:47.710685 12625 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:47.668725 15536 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:47.716233 12673 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:47.711882 15555 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.692714 15436 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:47.702530 15141 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:47.712587 15560 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:47.702520 15468 consensus_queue.cc:260] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:47.730832 12673 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:47.766476 12702 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
W20260709 13:18:47.744904 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:47.786942 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 56178.6us
I20260709 13:18:47.729630 15634 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:47.788509 15440 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:47.727923 15555 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:47.790055 15498 consensus_queue.cc:1059] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:18:47.790764 15555 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:47.731122 15625 log.cc:927] Time spent T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:18:47.792682 15438 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:47.796013 15613 log.cc:927] Time spent T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
W20260709 13:18:47.723656 15461 consensus_meta.cc:287] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.003s
I20260709 13:18:47.749683 15437 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:47.767768 15629 log.cc:927] Time spent T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
W20260709 13:18:47.771544 15612 log.cc:927] Time spent T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.411s	user 0.001s	sys 0.000s
I20260709 13:18:47.804559 15437 consensus_queue.cc:1059] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:47.805161 12673 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:47.806066 15437 raft_consensus.cc:3055] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:47.806141 12673 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:47.806525 15437 raft_consensus.cc:740] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:47.807768 12673 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:47.781203 15525 raft_consensus.cc:3055] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:47.758422 15535 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:47.764601 15312 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:47.806941 15555 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:47.813722 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:47.813649 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:47.815068 12626 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:18:47.812458 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:47.763410 15536 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:47.811180 12633 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:18:47.817925 12673 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:47.819396 15535 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:47.819801 15535 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:47.820186 15535 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:46.863933 12707 rpcz_store.cc:276] 0709 13:18:45.471592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.471762 (+   170us) service_pool.cc:224] Handling call
0709 13:18:45.472054 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:45.472263 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:45.485909 (+ 13646us) log.cc:844] Serialized 14 byte log entry
0709 13:18:46.640067 (+1154158us) spinlock_profiling.cc:243] Waited 4.07 ms on lock 0x7b18004348c0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf5223a 00007f4c6d3bca2a
0709 13:18:46.859920 (+219853us) log.cc:844] Serialized 14 byte log entry
0709 13:18:46.860148 (+   228us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:46.860213 (+    65us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:46.860273 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:46.860365 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:46.860522 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13484,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":3476,"raft.queue_time_us":5912,"spinlock_wait_cycles":9352448,"thread_start_us":219245,"threads_started":1}
I20260709 13:18:47.822077 15568 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:18:47.802865 15632 log.cc:927] Time spent T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:18:47.810474 15560 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:47.788506 15634 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:47.837127 12634 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
W20260709 13:18:47.838636 15631 log.cc:927] Time spent T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.141s	user 0.003s	sys 0.000s
I20260709 13:18:47.823357 15437 consensus_queue.cc:260] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:47.842933 15546 raft_consensus.cc:3055] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:47.748708 15554 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:18:47.841271 15099 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:47.860147 15437 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:47.861796 15546 raft_consensus.cc:740] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:47.811103 15525 raft_consensus.cc:740] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:47.812796 15438 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:47.868471 13377 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:18:47.869216 12776 raft_consensus.cc:1240] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:47.850060 15634 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:47.875835 15386 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.797230 15565 consensus_queue.cc:260] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:47.883651 15438 raft_consensus.cc:697] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:47.907603 12781 raft_consensus.cc:1240] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:47.910276 15468 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:47.911614 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:47.912514 12707 raft_consensus.cc:2393] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 7: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:47.915386 12705 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:47.919186 12709 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:47.920373 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:47.923307 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.925005 12708 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:18:47.926987 12700 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:18:47.928881 12703 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:47.914476 15565 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:47.932442 12747 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:47.933980 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:47.937067 12625 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:47.892175 15386 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:47.939172 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:47.942087 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:47.952549 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:47.943547 15519 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:47.911902 15525 consensus_queue.cc:260] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:47.958093 15519 raft_consensus.cc:697] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:47.958648 15525 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:47.958979 15519 consensus_queue.cc:237] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:47.937963 15529 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
W20260709 13:18:47.952121 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.941728 15436 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:47.963798 15597 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:47.939939 15441 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:47.966180 15441 raft_consensus.cc:697] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:47.967344 15441 consensus_queue.cc:237] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:47.969429 15099 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:47.974469 12782 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:18:47.974871 12782 pending_rounds.cc:85] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 3
I20260709 13:18:47.975071 12782 pending_rounds.cc:107] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:18:47.975268 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:47.975561 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:47.976488 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:47.976660 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:47.978127 12774 raft_consensus.cc:1240] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:47.979279 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:47.980051 12778 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:47.980881 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:47.975286 12782 raft_consensus.cc:2889] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:47.977520 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:47.981454 12780 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:18:47.986835 12705 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:18:47.988615 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 46787.8us
I20260709 13:18:47.901743 15546 consensus_queue.cc:260] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:47.944664 15370 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:47.993880 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:47.995438 12672 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:47.995488 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:47.998571 12702 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:48.001077 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:47.979300 15566 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:48.004317 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.996948 15436 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 13:18:48.007115 12672 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:47.993530 15467 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:48.010957 12672 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:48.000195 15572 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:48.005836 15546 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:48.019765 12702 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:48.008095 15533 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:48.024923 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:48.007243 15478 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:18:47.955520 15438 consensus_queue.cc:237] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:48.006292 15099 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:48.014163 15370 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:48.037261 12626 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:18:48.039367 12626 pending_rounds.cc:85] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 4
I20260709 13:18:48.039623 12626 pending_rounds.cc:107] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:18:48.039880 12626 raft_consensus.cc:2889] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:48.039785 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:48.033155 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:48.033404 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:48.035086 12629 raft_consensus.cc:1275] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:48.035907 12628 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:48.038012 12706 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:48.006910 15529 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:18:48.023716 15451 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:48.038075 15463 raft_consensus.cc:3055] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:48.045408 12707 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:18:48.049885 12625 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:18:48.052250 12701 raft_consensus.cc:1240] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:18:48.054207 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:48.013962 15575 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:18:48.032394 15574 raft_consensus.cc:3055] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:48.062706 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:48.051291 15637 log.cc:927] Time spent T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:18:48.022897 15480 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:18:48.066558 15523 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:48.075177 15523 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:48.044720 15562 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:48.067778 15498 consensus_queue.cc:1059] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
W20260709 13:18:48.088238 15636 log.cc:927] Time spent T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:18:48.056010 15463 raft_consensus.cc:740] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:48.067481 15574 raft_consensus.cc:740] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:48.087965 15529 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:48.071408 15474 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:48.083309 15572 consensus_queue.cc:1048] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:18:48.090829 15574 consensus_queue.cc:260] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:48.092168 15574 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:48.094049 15533 consensus_queue.cc:1059] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:48.094880 12672 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:48.095119 15562 raft_consensus.cc:3055] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:48.095531 15562 raft_consensus.cc:740] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:48.095420 15463 consensus_queue.cc:260] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:48.096280 15099 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:48.096694 15463 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:48.096386 15562 consensus_queue.cc:260] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:48.097834 15562 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:48.102267 12672 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:48.084282 15498 raft_consensus.cc:3055] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:48.125658 12782 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:18:48.147686 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 78622.6us
I20260709 13:18:48.127588 15561 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:18:48.152088 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 46412.6us
I20260709 13:18:48.128136 15498 raft_consensus.cc:740] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:48.142899 15525 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.162266 12597 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:48.127875 15451 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:18:48.142313 15634 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:48.182010 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.154423 15561 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:48.173004 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.186892 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:48.175959 15542 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:48.198964 15535 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:48.203755 12597 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:48.188546 12782 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:18:48.205137 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:48.205766 12627 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:48.189965 12779 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:18:48.190549 12780 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:18:48.217307 12627 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:18:48.193651 15548 consensus_queue.cc:1059] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:18:48.219851 12673 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:48.185173 15557 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.192734 15561 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:48.229327 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:48.230175 12777 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:48.232564 12779 raft_consensus.cc:1275] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:48.183640 15554 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:18:48.173499 15525 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:48.241325 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:48.240409 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:48.248266 13349 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:48.248806 13377 raft_consensus.cc:2368] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8.
I20260709 13:18:48.243089 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:48.240993 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:48.250290 13379 raft_consensus.cc:2393] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 7: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:48.197095 15498 consensus_queue.cc:260] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:48.185947 15592 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:18:48.247881 13385 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:18:48.252085 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:48.257800 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:48.221454 15535 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:48.259549 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:48.227979 15494 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:48.261004 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:48.265527 13379 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:48.231081 15542 raft_consensus.cc:697] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:48.265606 12628 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: []
W20260709 13:18:48.267606 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.269052 12775 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:18:48.269915 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:48.270457 12782 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:48.282348 13385 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:18:48.287635 12782 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:18:48.300931 13385 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:48.290551 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:48.254300 15496 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:48.293452 12672 leader_election.cc:400] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8.
W20260709 13:18:48.303619 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 80715.8us
I20260709 13:18:48.304188 12672 leader_election.cc:403] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:18:48.304256 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:48.236831 15557 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:48.302227 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:48.311020 12781 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:48.261781 15458 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:48.252017 15254 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:18:48.270841 15572 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.277724 15461 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:18:48.318883 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 56320.2us
I20260709 13:18:48.326052 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:48.262028 15536 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:18:48.297709 15494 raft_consensus.cc:697] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:48.294924 15312 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.266672 15596 consensus_queue.cc:1048] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:18:48.330615 12672 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:48.308969 15467 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.338564 15312 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:48.308426 15474 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:48.313809 15441 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.305164 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:48.322206 15498 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:48.323267 15451 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8.
W20260709 13:18:48.450332 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 142767us
W20260709 13:18:48.392596 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.438410 12781 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:48.328292 15572 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:48.337260 15437 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:48.445016 15535 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.358655 13376 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:18:48.446172 15597 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.323101 15458 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:48.347411 15438 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:48.352161 15560 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:48.468077 15566 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.453331 15549 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
W20260709 13:18:48.584656 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 275759us
I20260709 13:18:48.504570 15438 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:48.356113 15441 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:48.447419 15542 consensus_queue.cc:237] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:48.581898 15645 log.cc:927] Time spent T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
I20260709 13:18:48.349252 15467 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:48.362597 15474 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:48.567711 15648 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:18:48.608608 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:48.494505 15254 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:48.475903 15647 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.144s
I20260709 13:18:48.532071 15546 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.632726 12597 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:48.601433 15438 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:48.638176 12709 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:48.579324 15566 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:48.608178 15525 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:48.650914 12777 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:18:48.506083 15535 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:48.534134 15597 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:48.663470 12706 raft_consensus.cc:1275] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:48.493667 15494 consensus_queue.cc:237] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:48.621995 15548 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:18:48.636969 15453 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:48.656260 15574 consensus_meta.cc:287] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.564s	user 0.009s	sys 0.005s
W20260709 13:18:48.669415 15641 log.cc:927] Time spent T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.349s	user 0.000s	sys 0.001s
I20260709 13:18:48.651646 15652 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:18:48.674189 15642 log.cc:927] Time spent T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.360s	user 0.001s	sys 0.000s
I20260709 13:18:48.624336 15650 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 13:18:48.601991 15519 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:18:48.657462 15651 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:48.664924 15557 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:48.635440 15461 raft_consensus.cc:697] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:48.674417 15560 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.645529 15546 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:48.629616 15648 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:48.657634 15654 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
W20260709 13:18:48.734407 12634 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56832 (request call id 2172) took 1300 ms (client timeout 1580 ms). Trace:
W20260709 13:18:48.734827 12634 rpcz_store.cc:269] 0709 13:18:47.433548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.441738 (+  8190us) service_pool.cc:224] Handling call
0709 13:18:48.734243 (+1292505us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b54002cdb54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:48.734346 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":906985216}
W20260709 13:18:48.725111 15646 log.cc:927] Time spent T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.364s	user 0.001s	sys 0.000s
I20260709 13:18:48.720196 15658 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:48.740681 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 98761.8us
I20260709 13:18:48.755231 12597 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:48.726763 15557 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.707307 15453 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:48.729244 15468 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.729889 15659 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.737032 15312 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:48.795564 15467 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:48.795275 15652 raft_consensus.cc:697] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:48.817615 15456 raft_consensus.cc:3055] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:48.818162 15456 raft_consensus.cc:740] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:48.819046 15456 consensus_queue.cc:260] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:48.820330 15456 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:48.821683 15649 log.cc:927] Time spent T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
W20260709 13:18:48.829726 15639 log.cc:927] Time spent T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.594s	user 0.001s	sys 0.003s
I20260709 13:18:48.830476 15566 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:48.832521 12703 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:18:48.832980 12703 pending_rounds.cc:85] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 3
I20260709 13:18:48.833228 12703 pending_rounds.cc:107] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:18:48.833503 12703 raft_consensus.cc:2889] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:48.725519 15651 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:48.732967 15560 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:48.773729 15572 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:48.843811 12779 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.009s	sys 0.018s
W20260709 13:18:48.840149 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 84977.7us
W20260709 13:18:48.858362 12779 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.009s	sys 0.018s
I20260709 13:18:48.858748 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:48.866154 12628 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:18:48.875473 12630 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:18:48.883036 12633 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:48.768133 15557 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:48.875224 15650 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:48.892850 12627 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:48.760907 15658 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:48.791513 15468 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:48.895598 15496 consensus_meta.cc:287] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.579s	user 0.010s	sys 0.002s
I20260709 13:18:48.858372 15441 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:48.877115 15572 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:48.799074 15659 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:48.920536 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:48.899212 15670 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:48.913535 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:48.922600 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 94923.6us
W20260709 13:18:48.923017 13349 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.003s	sys 0.010s
W20260709 13:18:48.923377 13349 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.003s	sys 0.010s
W20260709 13:18:48.923681 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 633712us
I20260709 13:18:48.902745 15496 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:48.904716 15458 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:48.967206 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 47922.9us
I20260709 13:18:48.991518 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:48.912739 15535 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:48.884974 15652 consensus_queue.cc:237] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:49.033425 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 174754us
I20260709 13:18:49.002763 15659 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:48.962172 15667 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.003666 15468 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:49.004345 15663 log.cc:927] Time spent T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:18:48.994627 15597 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:49.007485 15463 consensus_meta.cc:287] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.910s	user 0.011s	sys 0.000s
W20260709 13:18:49.008579 15669 log.cc:927] Time spent T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:18:48.908376 15461 consensus_queue.cc:237] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:49.016773 15453 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:48.967082 15672 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.017808 15474 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:49.019320 15561 consensus_meta.cc:287] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.767s	user 0.008s	sys 0.003s
I20260709 13:18:49.118175 15561 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:49.120254 15561 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:49.021047 15565 consensus_meta.cc:287] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 1.082s	user 0.010s	sys 0.002s
I20260709 13:18:49.022565 15560 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:49.121397 15462 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.122013 15672 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:49.122263 15462 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:49.124250 15672 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:49.035669 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:48.992561 15660 log.cc:927] Time spent T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 13:18:48.987936 15536 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:49.127265 15536 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:49.129262 15536 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:48.921134 15572 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:49.045586 15254 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:49.137622 15572 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:49.056474 15549 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:49.138244 15549 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:49.140220 15549 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:49.049908 15458 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.140898 15458 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:49.142603 15658 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:49.142954 15458 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:49.054117 15562 consensus_meta.cc:287] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.956s	user 0.006s	sys 0.006s
I20260709 13:18:49.049059 15546 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:49.032023 15496 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:49.050074 15667 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:49.130748 15557 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:49.131371 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 267648us
I20260709 13:18:49.133796 15648 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:49.134450 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:49.145852 15667 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:49.146245 12704 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:18:49.147367 15667 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:49.148808 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:49.149977 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:49.150516 12706 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:49.151299 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:49.104867 15597 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.152462 15597 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:48.968163 15670 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:49.143550 15651 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:49.147403 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:49.173640 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:49.184309 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:49.185009 12630 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
W20260709 13:18:49.141323 15498 consensus_meta.cc:287] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.698s	user 0.006s	sys 0.002s
I20260709 13:18:49.175925 12633 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:18:49.176540 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:49.186573 12625 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:49.177104 12629 raft_consensus.cc:1275] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:18:49.177820 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:49.036696 15456 consensus_queue.cc:1059] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:18:49.194885 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:49.196007 12704 raft_consensus.cc:2410] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:18:49.197189 12629 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:49.205112 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:49.112254 15468 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.196902 12706 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:18:49.137305 15254 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:49.212311 12704 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:49.220044 12705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1404) took 1302 ms. Trace:
I20260709 13:18:49.220520 12705 rpcz_store.cc:276] 0709 13:18:47.917684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.917863 (+   179us) service_pool.cc:224] Handling call
0709 13:18:47.980126 (+ 62263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:47.987285 (+  7159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.219852 (+1232567us) spinlock_profiling.cc:243] Waited 61.9 ms on lock 0x7b54002495d0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:49.219991 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":142368128}
I20260709 13:18:49.224738 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:49.228375 15651 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:18:49.226738 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 198202us
I20260709 13:18:49.229704 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:49.230363 12626 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
I20260709 13:18:49.231612 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:49.232024 13348 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:49.233934 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:49.234553 13376 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:49.235580 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:49.236241 13381 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:18:49.236805 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:49.237232 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:49.237406 12625 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:18:49.239313 12704 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:18:49.211866 15670 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:49.240618 15456 consensus_queue.cc:1243] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer 522fd2ba0e9f402691b5ef29af8579cd log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:18:49.233213 15651 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:18:49.256048 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:49.259012 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:49.267988 12597 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:49.245359 15461 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:49.218186 15468 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:49.271486 15670 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:18:49.272959 15438 consensus_meta.cc:287] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.611s	user 0.008s	sys 0.004s
I20260709 13:18:49.267992 15456 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:49.276970 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:49.274986 15651 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:49.276100 15670 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:49.343794 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:49.278199 15456 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:49.279731 15461 raft_consensus.cc:697] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:49.302778 15549 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.348342 15456 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:49.312733 15536 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.302840 15458 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:49.357856 15458 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:49.358304 15458 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:49.302790 15597 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:49.346071 15386 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:49.344187 15670 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:49.285835 15438 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:49.363718 15438 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:49.364452 12707 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:18:49.312942 15651 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:49.366061 15651 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:49.373937 12701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1704) took 1343 ms. Trace:
I20260709 13:18:49.374629 12701 rpcz_store.cc:276] 0709 13:18:48.030924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.031055 (+   131us) service_pool.cc:224] Handling call
0709 13:18:48.050350 (+ 19295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:48.058975 (+  8625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.373480 (+1314505us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b54001086d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:49.373854 (+   374us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3851008}
I20260709 13:18:49.378573 13385 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:49.379223 13385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 513) took 1073 ms. Trace:
I20260709 13:18:49.379525 13385 rpcz_store.cc:276] 0709 13:18:48.305934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.306049 (+   115us) service_pool.cc:224] Handling call
0709 13:18:49.377946 (+1071897us) spinlock_profiling.cc:243] Waited 4.75 ms on lock 0x7b5800095198. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c7238bfc0 00007f4c72376929 00007f4c723763d2 00007f4c6b8edbd4
0709 13:18:49.378168 (+   222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:49.378973 (+   805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.379168 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10931072}
I20260709 13:18:49.382696 15468 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:49.352754 15549 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:49.385492 15536 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:49.386659 15670 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:49.387835 15254 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:49.346418 15648 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:49.388624 15648 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:49.343891 15557 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:49.383044 15651 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:49.510486 15496 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:49.514608 13385 raft_consensus.cc:1275] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:18:49.519204 15667 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:49.519730 15667 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:49.520172 15648 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:49.522670 15461 consensus_queue.cc:237] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:49.511091 15496 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:49.524256 15542 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:49.520987 15648 consensus_queue.cc:1048] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:49.526104 15681 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:49.506228 15557 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:49.577318 15254 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.577989 15254 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:49.579695 15546 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:49.580283 15546 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:49.580443 15254 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:49.580564 15688 log.cc:927] Time spent T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.237s	user 0.002s	sys 0.000s
I20260709 13:18:49.582538 15634 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:49.583729 15546 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:49.584484 15549 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:49.574047 15681 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:49.588191 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:49.591410 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:49.594013 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:49.595885 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:49.598258 15689 log.cc:927] Time spent T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:18:49.601347 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:49.602478 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:49.603276 12634 raft_consensus.cc:2410] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:18:49.603971 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:49.604502 12634 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:18:49.605511 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:49.606655 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:49.609438 12673 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:49.596747 12629 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:18:49.615736 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:49.616776 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:49.617257 12629 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:18:49.610224 12625 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:49.618906 12625 raft_consensus.cc:1275] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:49.621024 12632 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:18:49.561184 15667 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:49.609932 12634 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:49.610980 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:49.607843 15670 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:49.621218 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.630312 12634 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:49.630354 12673 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:49.574107 15542 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:49.622274 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:49.642047 12673 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:49.644820 12673 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:49.624572 15099 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:49.679230 12673 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:49.643344 15664 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:49.647527 15651 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:49.668763 15634 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:49.668473 15099 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:49.684053 15468 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:49.646728 15370 consensus_queue.cc:1048] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:18:49.646605 15554 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:18:49.683921 15438 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:18:49.700636 15697 log.cc:927] Time spent T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:18:49.712551 15438 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:49.713155 15438 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:49.691479 15664 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:49.713915 15664 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:49.708577 12632 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
W20260709 13:18:49.721963 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 88328.4us
I20260709 13:18:49.703821 15099 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:49.726047 12778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2075) took 1399 ms. Trace:
I20260709 13:18:49.731509 12778 rpcz_store.cc:276] 0709 13:18:48.327084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.327328 (+   244us) service_pool.cc:224] Handling call
0709 13:18:48.338806 (+ 11478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:48.636469 (+297663us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:48.761910 (+125441us) log.cc:844] Serialized 14 byte log entry
0709 13:18:48.794615 (+ 32705us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:48.794666 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:48.794708 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.725826 (+931118us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b540034a654. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:49.725869 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:49.725981 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3431,"prepare.queue_time_us":84562,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":4004,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"spinlock_wait_cycles":684040448,"thread_start_us":115139,"threads_started":2,"wal-append.queue_time_us":108836}
I20260709 13:18:49.732936 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1698) took 1703 ms. Trace:
I20260709 13:18:49.733475 12702 rpcz_store.cc:276] 0709 13:18:48.029015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.029143 (+   128us) service_pool.cc:224] Handling call
0709 13:18:48.029447 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:48.143195 (+113748us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:48.143227 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:48.146634 (+  3407us) log.cc:844] Serialized 26 byte log entry
0709 13:18:48.251956 (+105322us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:48.251999 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.728876 (+1476877us) spinlock_profiling.cc:243] Waited 34.7 ms on lock 0x7b54001ccad4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:49.728925 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:49.728976 (+    51us) raft_consensus.cc:1685] finished
0709 13:18:49.728990 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:49.732833 (+  3843us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65184,"raft.queue_time_us":81031,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":6342,"spinlock_wait_cycles":81082496,"thread_start_us":96056,"threads_started":1,"wal-append.queue_time_us":207455,"wal-append.run_cpu_time_us":3272,"wal-append.run_wall_time_us":1351948}
I20260709 13:18:49.708312 15468 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:49.708652 15634 raft_consensus.cc:697] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:49.723623 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:49.725921 15681 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:49.745025 15683 raft_consensus.cc:3055] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:49.745815 15683 raft_consensus.cc:740] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:49.703186 15458 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:49.741732 15312 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:49.739344 15408 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.739876 15468 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:49.738032 15496 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:49.794148 15408 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:49.800688 15691 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:49.800952 15634 consensus_queue.cc:237] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:49.801263 15691 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:49.802119 15408 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:49.804121 15691 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:49.794612 13377 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:18:49.805728 13377 pending_rounds.cc:85] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd: Aborting all ops after (but not including) 7
I20260709 13:18:49.805994 13377 pending_rounds.cc:107] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:18:49.806218 13377 raft_consensus.cc:2889] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:49.806854 13377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 511) took 1524 ms. Trace:
I20260709 13:18:49.807153 13377 rpcz_store.cc:276] 0709 13:18:48.282170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.283020 (+   850us) service_pool.cc:224] Handling call
0709 13:18:49.793913 (+1510893us) spinlock_profiling.cc:243] Waited 2.13 ms on lock 0x7b5800095198. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c7238bfc0 00007f4c72376929 00007f4c723763d2 00007f4c6b8edbd4
0709 13:18:49.794158 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:49.806620 (+ 12462us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.806802 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4896640}
I20260709 13:18:49.809413 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:49.810097 12627 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:49.811412 12673 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:49.811616 12627 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:18:49.812846 15495 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:49.813269 15495 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:49.813149 15673 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:49.813750 15495 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:49.817800 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:49.818562 12628 raft_consensus.cc:2393] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 10: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:49.721678 15557 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:49.821347 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:49.739560 15596 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.079s
I20260709 13:18:49.772249 15312 raft_consensus.cc:697] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 5 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:49.820734 15554 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.832463 15408 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.838917 15437 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:49.853899 15462 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:49.852262 15597 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:49.948024 12527 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54792 (request call id 225) took 23590 ms (client timeout 15000 ms). Trace:
I20260709 13:18:49.872105 15451 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:49.873135 15554 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:50.110018 15462 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:50.133050 15703 log.cc:927] Time spent T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.345s	user 0.000s	sys 0.001s
I20260709 13:18:50.141227 12628 consensus_queue.cc:237] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.145373 15451 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:50.147437 15554 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:49.949330 15437 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:49.890373 15557 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:50.132174 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 226190us
W20260709 13:18:50.154048 13348 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.548s (SENT) [suppressed 27 similar messages]
I20260709 13:18:50.160522 12672 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:49.894616 15408 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:49.990408 13376 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:18:50.159634 15451 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:49.846539 15650 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.879s	user 0.016s	sys 0.008s
W20260709 13:18:50.132830 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:49.987866 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 92374.9us
I20260709 13:18:50.179652 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:50.163838 15408 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:50.181068 15408 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:50.181910 15408 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.181196 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:50.146057 15462 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.189965 15462 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:50.111943 15597 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:50.156059 15557 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:50.214416 15312 consensus_queue.cc:237] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:50.196398 15650 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.274s	user 0.022s	sys 0.009s
I20260709 13:18:49.840198 15683 consensus_queue.cc:260] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.151079 15437 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:50.244745 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:50.225173 15542 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:50.239487 15634 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:50.264534 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:50.344192 12629 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:50.241209 15596 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:50.246061 15496 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:50.245510 15465 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:18:50.289076 15664 consensus_meta.cc:287] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.575s	user 0.007s	sys 0.002s
I20260709 13:18:50.276815 15458 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:50.293850 15536 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:50.298884 12709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1413) took 1660 ms. Trace:
I20260709 13:18:50.359647 12709 rpcz_store.cc:276] 0709 13:18:48.638218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.642694 (+  4476us) service_pool.cc:224] Handling call
0709 13:18:48.643112 (+   418us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:48.854469 (+211357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:48.854517 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:49.578458 (+723941us) spinlock_profiling.cc:243] Waited 26.1 ms on lock 0x7b2c00011ed8. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6bcaba11 00007f4c6bcbb372 00007f4c6d3eda01 00007f4c6d3b243a 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2
0709 13:18:49.579013 (+   555us) log.cc:844] Serialized 52 byte log entry
0709 13:18:49.684426 (+105413us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:49.684479 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.298462 (+613983us) spinlock_profiling.cc:243] Waited 31.5 ms on lock 0x7b540006e854. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:50.298530 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:50.298591 (+    61us) raft_consensus.cc:1685] finished
0709 13:18:50.298611 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:50.298803 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":488,"raft.queue_time_us":83808,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":11941,"spinlock_wait_cycles":132676096,"thread_start_us":284158,"threads_started":2,"wal-append.queue_time_us":130899}
I20260709 13:18:50.301160 12707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1412) took 1663 ms. Trace:
I20260709 13:18:50.361939 12707 rpcz_store.cc:276] 0709 13:18:48.637980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.640517 (+  2537us) service_pool.cc:224] Handling call
0709 13:18:48.640871 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:49.364915 (+724044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.300868 (+935953us) spinlock_profiling.cc:243] Waited 723 ms on lock 0x7b54003b38d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:50.301091 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1663551232}
W20260709 13:18:50.313159 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 150791us
W20260709 13:18:49.991050 12527 rpcz_store.cc:269] 0709 13:18:26.321331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.321504 (+   173us) service_pool.cc:224] Handling call
0709 13:18:49.911647 (+23590143us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:44.071631 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:44.090398 (+ 18767us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:44.090450 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:18:44.090486 (+    36us) tablet.cc:662] Decoding operations
0709 13:18:44.091579 (+  1093us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:44.091709 (+   130us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:44.091741 (+    32us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:18:44.092705 (+   964us) tablet.cc:701] Row locks acquired
0709 13:18:44.092757 (+    52us) write_op.cc:260] PREPARE: finished
0709 13:18:44.092965 (+   208us) write_op.cc:270] Start()
0709 13:18:44.093112 (+   147us) write_op.cc:276] Timestamp: P: 1783603124092888 usec, L: 0
0709 13:18:44.093137 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:18:44.134770 (+ 41633us) log.cc:844] Serialized 3999 byte log entry
0709 13:18:44.468579 (+333809us) op_driver.cc:464] REPLICATION: finished
0709 13:18:44.833448 (+364869us) write_op.cc:301] APPLY: starting
0709 13:18:44.855329 (+ 21881us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:44.883872 (+ 28543us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:44.961171 (+ 77299us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:44.983531 (+ 22360us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:44.983839 (+   308us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 13:18:44.983881 (+    42us) write_op.cc:312] APPLY: finished
0709 13:18:44.984225 (+   344us) log.cc:844] Serialized 77 byte log entry
0709 13:18:44.984448 (+   223us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:44.984586 (+   138us) write_op.cc:454] Released schema lock
0709 13:18:44.984648 (+    62us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:44.985009 (+   361us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:44.985037 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":20600,"reported_tablets":41,"child_traces":[["op",{"apply.queue_time_us":218443,"apply.run_cpu_time_us":3892,"apply.run_wall_time_us":205092,"num_ops":10,"prepare.queue_time_us":312530,"prepare.run_cpu_time_us":39440,"prepare.run_wall_time_us":198566,"raft.queue_time_us":35232,"raft.run_cpu_time_us":17002,"raft.run_wall_time_us":209747,"replication_time_us":345847,"thread_start_us":371359,"threads_started":4,"wal-append.queue_time_us":144692,"wal-append.run_cpu_time_us":4826,"wal-append.run_wall_time_us":1668631}]]}
I20260709 13:18:50.300531 15495 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:50.365108 15495 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:50.293874 12517 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 981fb2f435e24d06a81fc02cf4591cc5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 19cd864c053744bebd96093a7982537e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:18:50.366158 12517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 85524.8us
I20260709 13:18:50.335201 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:50.302037 15099 consensus_meta.cc:287] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.562s	user 0.009s	sys 0.002s
I20260709 13:18:50.347078 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:50.185504 15408 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:50.378510 15465 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.367297 15596 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:50.381734 15468 consensus_meta.cc:287] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.550s	user 0.012s	sys 0.004s
I20260709 13:18:50.390760 15468 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:50.392042 15596 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:50.393445 15468 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:50.367111 15496 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:50.382603 12628 raft_consensus.cc:2410] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:18:50.394574 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:50.395627 15496 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:50.381781 15495 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.396338 15495 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.394217 15468 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:50.399189 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:50.399605 15468 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.399699 15664 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:50.401252 15495 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:50.401770 15664 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:50.402372 15664 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.402823 15664 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.406045 15557 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:50.409714 15708 log.cc:927] Time spent T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:18:50.411610 15468 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:50.412329 15468 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.413031 15465 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:50.412793 15468 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.399314 15597 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:50.415335 15468 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:50.415716 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:50.416564 13376 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:50.401777 15495 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:50.422580 15495 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.371454 12631 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:18:50.388955 15566 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:18:50.389750 15438 consensus_meta.cc:287] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.676s	user 0.010s	sys 0.000s
I20260709 13:18:50.336793 15650 raft_consensus.cc:697] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:50.356348 15683 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:50.379997 15634 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:50.434808 15715 log.cc:927] Time spent T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.151s	user 0.002s	sys 0.000s
W20260709 13:18:50.387542 15456 consensus_meta.cc:287] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.903s	user 0.010s	sys 0.000s
I20260709 13:18:50.389870 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:50.453892 13377 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:50.454821 12631 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
I20260709 13:18:50.456542 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:50.438812 15597 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:50.440192 15664 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:50.461613 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:50.464198 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:50.459079 15634 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:50.467438 15495 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:50.468638 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:50.471489 12628 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:50.473335 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 319305us
I20260709 13:18:50.476142 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:50.479755 13377 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:18:50.479983 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:50.485358 12625 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:50.486156 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:50.486729 12631 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:50.461114 15718 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.438191 15099 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:50.434960 15566 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.439877 15438 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:50.461301 15456 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.478770 12634 raft_consensus.cc:2393] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 10: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:50.497432 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:50.465126 15536 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:50.500999 12631 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:50.487188 12628 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:18:50.517724 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:50.519208 12634 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:18:50.519325 12673 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:50.521402 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:50.522190 13384 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:50.525417 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:50.525956 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:50.526618 12628 raft_consensus.cc:2410] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:18:50.505218 15717 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:50.473313 15597 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.507735 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:50.551581 12625 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:18:50.555473 15597 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:50.557837 15456 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:50.550153 15664 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:50.559156 13376 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:50.559427 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:50.560254 13377 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:50.538681 15717 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.562628 12779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2074) took 2303 ms. Trace:
I20260709 13:18:50.564184 15717 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:50.562999 12779 rpcz_store.cc:276] 0709 13:18:48.259055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.259205 (+   150us) service_pool.cc:224] Handling call
0709 13:18:48.293088 (+ 33883us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:48.859105 (+566017us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:48.859153 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:48.871148 (+ 11995us) log.cc:844] Serialized 26 byte log entry
0709 13:18:48.992926 (+121778us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:49.044801 (+ 51875us) log.cc:844] Serialized 14 byte log entry
0709 13:18:50.397813 (+1353012us) spinlock_profiling.cc:243] Waited 34.9 ms on lock 0x7b18003c7120. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c53f3 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:18:50.562091 (+164278us) spinlock_profiling.cc:243] Waited 2 ms on lock 0x7b18003c7120. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a23fc0a 00007f4c7397f219 00007f4c6a2c7a40 00007f4c6a2c24e4 00007f4c6a2c2455 00007f4c6d3aa060 00007f4c6d3bc716 00007f4c6d3cd287 00007f4c6d3cd208
0709 13:18:50.562280 (+   189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.562349 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:50.562391 (+    42us) raft_consensus.cc:1685] finished
0709 13:18:50.562414 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:50.562555 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47499,"prepare.queue_time_us":57362,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":46092,"raft.queue_time_us":1569737,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":9447,"spinlock_wait_cycles":84832896,"thread_start_us":650108,"threads_started":2,"wal-append.queue_time_us":239423}
I20260709 13:18:50.569836 12673 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:50.579203 13377 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:50.581113 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:50.581820 15436 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.584052 15659 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:50.584290 15436 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.584797 15659 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.586475 15436 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:50.582012 15478 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:50.587718 15456 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:50.588913 15456 raft_consensus.cc:697] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:50.590173 15456 consensus_queue.cc:237] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.591741 12627 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:18:50.592450 15664 raft_consensus.cc:697] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:50.592975 12673 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:50.593456 15480 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:50.593736 15468 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:50.593334 15664 consensus_queue.cc:237] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:50.594172 15468 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.594015 15480 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:50.594565 15468 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:50.596051 15480 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:50.596577 15480 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:18:50.597175 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 894414us
I20260709 13:18:50.585250 13376 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:18:50.530311 15496 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:50.538746 13381 raft_consensus.cc:1275] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:18:50.503963 15718 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.620121 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:50.617159 13377 raft_consensus.cc:1275] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:18:50.599532 15668 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.619040 12633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2202) took 1032 ms. Trace:
I20260709 13:18:50.536120 15536 raft_consensus.cc:697] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:50.623389 12673 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:50.621883 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:50.622728 12633 rpcz_store.cc:276] 0709 13:18:49.586741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.590495 (+  3754us) service_pool.cc:224] Handling call
0709 13:18:49.590843 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:49.608665 (+ 17822us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:49.608699 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:49.608739 (+    40us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:49.608770 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.618785 (+1010015us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b54000e1ad4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:50.618828 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:50.618966 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40545664}
I20260709 13:18:50.625972 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:50.621313 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:50.634313 15557 consensus_queue.cc:1048] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:50.634506 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:50.641516 12709 raft_consensus.cc:1275] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:18:50.635128 15436 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:50.636036 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:50.601060 15441 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:50.636561 12703 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:18:50.636749 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:50.621695 15523 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:18:50.637439 15554 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.637704 15474 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:50.618798 15683 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.638676 15566 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:50.637759 15724 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.639307 12672 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:50.640060 15099 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:50.640473 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:50.630805 15496 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:50.690121 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 50898.9us
I20260709 13:18:50.692371 12672 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:50.641225 15438 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:50.593348 15597 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:50.625541 15717 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:50.660939 12673 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:50.661489 12777 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: [6.6-8.7]
I20260709 13:18:50.662334 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:50.655452 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:50.710130 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:50.651304 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:50.658030 12705 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:50.664062 12597 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:18:50.711983 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 48001.4us
I20260709 13:18:50.712222 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:50.619194 15659 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:50.507264 15650 consensus_queue.cc:237] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.667450 12707 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:18:50.651262 12702 raft_consensus.cc:2410] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:18:50.680819 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:50.647472 15668 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.601716 15478 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:50.662386 15451 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:50.682430 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:50.695392 15562 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:50.743014 12703 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:18:50.745359 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:50.648499 15519 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:50.648612 15658 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:50.658285 15572 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:50.620926 15480 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:50.648792 15675 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:18:50.681315 15647 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.846s	user 0.034s	sys 0.014s
I20260709 13:18:50.688962 12704 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:18:50.697302 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:50.664988 15441 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:50.763917 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:50.713485 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 52634.8us
I20260709 13:18:50.715611 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:50.717716 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:50.721666 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:50.773156 12774 raft_consensus.cc:2393] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 8: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:50.774076 12704 raft_consensus.cc:1275] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 13:18:50.774281 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 77074.5us
I20260709 13:18:50.775035 12672 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:50.716867 15566 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:50.780267 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:50.719480 15557 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:50.786341 12778 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
W20260709 13:18:50.763640 15647 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.958s	user 0.035s	sys 0.015s
I20260709 13:18:50.729651 15662 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:50.728869 15596 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.754050 15495 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:50.689204 15683 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:50.686067 15554 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.707530 15724 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.754527 15648 consensus_queue.cc:1048] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.135s
I20260709 13:18:50.719856 15717 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.755596 15720 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:50.787452 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:50.802371 12633 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:50.886229 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:50.729310 15659 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.799806 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:51.035645 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 193946us
I20260709 13:18:50.754686 15536 consensus_queue.cc:237] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:50.782619 15688 log.cc:927] Time spent T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:18:51.027431 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:51.039535 12703 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
W20260709 13:18:50.784126 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.052477 12706 raft_consensus.cc:2393] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 5: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:51.042606 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:50.755514 15597 raft_consensus.cc:697] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:18:51.045269 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.036440 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:50.765282 15652 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.058441 12631 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:51.062656 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:51.062451 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:50.977315 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:50.773854 15549 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:51.033042 15496 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:51.067051 12701 raft_consensus.cc:2410] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:18:50.856637 15370 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:51.067806 12781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2080) took 2153 ms. Trace:
I20260709 13:18:51.067544 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:50.763211 15658 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.933504 15557 raft_consensus.cc:697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:51.068542 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:51.068749 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:50.779886 15451 raft_consensus.cc:697] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:51.025741 15726 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.045166 15436 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.956965 15722 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:50.774022 15572 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:51.047887 15596 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:50.758276 15519 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:50.781502 15675 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:51.080474 12708 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:18:51.164867 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:51.165544 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 53943.9us
I20260709 13:18:50.797974 15728 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:51.057847 15720 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:51.127151 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 238106us
I20260709 13:18:51.231498 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:51.250891 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:51.249699 12633 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:51.251369 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:51.232834 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:51.252187 12774 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:51.080221 12781 rpcz_store.cc:276] 0709 13:18:48.914379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.918676 (+  4297us) service_pool.cc:224] Handling call
0709 13:18:48.967770 (+ 49094us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:48.987186 (+ 19416us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:49.023634 (+ 36448us) log.cc:844] Serialized 14 byte log entry
0709 13:18:49.579989 (+556355us) spinlock_profiling.cc:243] Waited 1.07 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:18:49.713054 (+133065us) spinlock_profiling.cc:243] Waited 6.59 ms on lock 0x7b4800042258. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c722dbaa1 00007f4c722d2b37 00007f4c722d29ca 00007f4c723ca26f 00007f4c723c9f37 00007f4c723de55c 00007f4c723de508 00007f4c723de488 00007f4c723de443 00007f4c723dd6ec 00007f4c71307a96
0709 13:18:50.563153 (+850099us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:50.563798 (+   645us) log.cc:844] Serialized 26 byte log entry
0709 13:18:50.563993 (+   195us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:50.685141 (+121148us) log.cc:844] Serialized 14 byte log entry
0709 13:18:51.066475 (+381334us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:51.067546 (+  1071us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:51.067601 (+    55us) raft_consensus.cc:1685] finished
0709 13:18:51.067614 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:51.067738 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2346122,"prepare.queue_time_us":433072,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":46055,"raft.queue_time_us":1797799,"raft.run_cpu_time_us":36076,"raft.run_wall_time_us":735422,"spinlock_wait_cycles":17623296,"thread_start_us":547346,"threads_started":3,"wal-append.queue_time_us":222023}
I20260709 13:18:51.254527 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:51.249673 12705 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:51.265656 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:51.266481 12701 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
I20260709 13:18:51.267588 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:51.268383 12701 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:51.256824 12704 raft_consensus.cc:2393] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 5: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:51.179437 12631 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:18:51.271701 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:51.259588 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:51.064746 15723 log.cc:927] Time spent T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.422s	user 0.001s	sys 0.000s
I20260709 13:18:51.050021 15312 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:51.187701 15730 log.cc:927] Time spent T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.497s	user 0.000s	sys 0.001s
I20260709 13:18:51.277937 12704 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:18:51.068387 15739 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.279163 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:51.270578 15728 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:51.256910 15720 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:51.294262 12708 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:51.051227 15495 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:51.058010 15659 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:51.162672 15370 raft_consensus.cc:697] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:51.188740 15718 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:51.297556 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:51.207978 15731 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:51.300799 13379 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:18:51.260571 15648 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.303272 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:51.308019 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:51.308393 12708 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
W20260709 13:18:51.304505 12779 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.008s	sys 0.007s
I20260709 13:18:51.074637 15652 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:51.303910 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:51.069552 15597 consensus_queue.cc:237] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:51.310034 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:51.295550 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:51.310773 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:51.324707 12708 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:51.324713 12701 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:51.051370 15662 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:51.307890 15648 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:51.320485 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:51.335176 12777 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:51.323963 12673 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:51.336334 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:51.262398 15726 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:51.279776 15523 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:18:51.225636 15480 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:51.297086 15478 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:51.255964 15722 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:51.332945 12782 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.004s	sys 0.016s
I20260709 13:18:51.295871 15496 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:51.337137 12777 raft_consensus.cc:2393] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 8: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:51.335321 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:51.309621 12779 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.008s	sys 0.007s
I20260709 13:18:51.086151 15549 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:51.320029 15727 log.cc:927] Time spent T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.435s	user 0.001s	sys 0.001s
I20260709 13:18:51.297993 15728 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:51.300173 15596 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:51.331739 12633 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:18:51.447180 15592 consensus_queue.cc:1048] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:18:51.447569 15724 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:51.344125 15451 consensus_queue.cc:237] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:51.243122 15436 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:51.463791 13382 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:51.464811 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:51.465404 15436 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:51.388716 15438 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:51.368912 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:51.466230 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:51.466190 15436 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:51.233834 15557 consensus_queue.cc:237] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:51.466923 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:51.467506 15722 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:51.446897 15480 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:18:51.370303 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:51.468524 12706 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:51.461784 15496 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:51.293711 15312 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:51.321530 15441 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:51.375469 12625 raft_consensus.cc:2410] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:18:51.378924 12708 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
I20260709 13:18:51.305660 15721 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:18:51.304123 15739 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:51.453429 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:51.455708 15737 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:18:51.487387 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 235890us
W20260709 13:18:51.323078 15437 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.003s	sys 0.013s
I20260709 13:18:51.467998 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:51.333495 15668 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:51.470651 12632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2229) took 1015 ms. Trace:
I20260709 13:18:51.439879 15647 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:18:51.393829 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 56656.8us
W20260709 13:18:51.445775 12782 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.004s	sys 0.023s
I20260709 13:18:51.319537 15731 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:51.485910 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:51.461859 15716 log.cc:927] Time spent T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:18:51.502668 12632 rpcz_store.cc:276] 0709 13:18:50.455655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.457872 (+  2217us) service_pool.cc:224] Handling call
0709 13:18:50.458293 (+   421us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:50.462280 (+  3987us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:50.462327 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:50.467008 (+  4681us) log.cc:844] Serialized 52 byte log entry
0709 13:18:50.535305 (+ 68297us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:50.535345 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.535823 (+   478us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:51.470390 (+934567us) raft_consensus.cc:1685] finished
0709 13:18:51.470417 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:51.470583 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":123,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":2969,"thread_start_us":67897,"threads_started":1,"wal-append.queue_time_us":131745}
W20260709 13:18:51.481464 15710 log.cc:927] Time spent T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 1.134s	user 0.006s	sys 0.000s
I20260709 13:18:51.470494 15659 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:51.469496 15480 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.506747 12777 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 4.
I20260709 13:18:51.517282 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:51.508641 12598 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:51.515691 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:51.518035 12777 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
W20260709 13:18:51.493076 15437 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.003s	sys 0.014s
I20260709 13:18:51.487095 15717 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:51.524991 12598 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:51.550938 12780 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.006s	sys 0.019s
W20260709 13:18:51.557525 12780 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.006s	sys 0.019s
I20260709 13:18:51.557354 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:18:51.522600 12700 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.003s	sys 0.019s
I20260709 13:18:51.488775 13378 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
W20260709 13:18:51.558655 12700 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.003s	sys 0.020s
I20260709 13:18:51.502288 15438 raft_consensus.cc:697] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:51.566344 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:51.567656 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:51.568794 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:51.555027 15683 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:51.569622 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:51.580895 15658 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:51.506131 15750 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:51.555363 15734 log.cc:927] Time spent T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.336s	user 0.000s	sys 0.002s
I20260709 13:18:51.580366 15312 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:51.515589 15441 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.573382 12627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 394) took 1399 ms. Trace:
I20260709 13:18:51.572036 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:51.617174 12627 rpcz_store.cc:276] 0709 13:18:50.174290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.174464 (+   174us) service_pool.cc:224] Handling call
0709 13:18:50.219992 (+ 45528us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:50.592166 (+372174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:51.573092 (+980926us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b54002db9d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:51.573320 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":854017792}
I20260709 13:18:51.582734 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:51.582036 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:51.602042 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 63954.5us
W20260709 13:18:51.575083 15738 log.cc:927] Time spent T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:18:51.608655 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:51.616134 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:51.569747 15759 consensus_queue.cc:1048] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.011s
W20260709 13:18:51.574359 15735 log.cc:927] Time spent T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:18:51.575788 15731 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:51.626133 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:51.556104 15480 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:51.651203 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:51.652225 15468 consensus_meta.cc:287] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 1.057s	user 0.012s	sys 0.000s
I20260709 13:18:51.658244 15468 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:51.523283 15647 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:51.665095 15647 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:51.652278 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:51.583364 15675 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:51.569801 15496 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:51.629168 13349 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:51.666164 12780 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:18:51.665904 15728 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:51.667127 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:51.667256 13349 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:51.668300 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:51.668483 13349 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:51.668943 15572 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:51.652333 15480 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:51.669358 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:51.669718 15651 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:51.613842 15750 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:51.670441 15651 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:51.670442 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:51.671029 15651 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:51.671440 15651 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:51.671711 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:51.671870 15750 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:51.672294 12780 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:51.620921 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:51.625660 15441 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:51.613204 15668 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:51.620316 15737 consensus_queue.cc:1059] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:18:51.569834 15659 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.103s
I20260709 13:18:51.632594 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:51.635423 12746 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:51.502532 15370 consensus_queue.cc:237] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:51.636197 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:51.636778 15753 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:18:51.638201 15752 log.cc:927] Time spent T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:18:51.515915 15650 consensus_queue.cc:1059] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.103s
I20260709 13:18:51.643123 15495 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:51.644409 15726 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:51.644974 12707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1709) took 1025 ms. Trace:
I20260709 13:18:51.645139 15652 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:51.645874 15662 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:51.646282 15519 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:51.647008 15648 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:51.647084 15549 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:51.647886 15312 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:51.649959 15739 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:51.630898 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:51.648916 15438 consensus_queue.cc:237] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:51.673012 12673 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:51.783325 13349 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:51.762965 12783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1801) took 1108 ms. Trace:
W20260709 13:18:51.852150 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 68963.2us
I20260709 13:18:51.803489 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:51.853171 12627 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:51.654618 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:51.799913 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:51.758045 13377 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:18:51.855595 12632 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:51.734812 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:51.856979 12707 rpcz_store.cc:276] 0709 13:18:50.619584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.619768 (+   184us) service_pool.cc:224] Handling call
0709 13:18:51.644634 (+1024866us) spinlock_profiling.cc:243] Waited 46.4 ms on lock 0x7b540006e354. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:51.644751 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":106637568}
I20260709 13:18:51.857800 12780 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:51.686309 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:51.858801 12625 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:51.676169 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:51.816908 12779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1793) took 1182 ms. Trace:
I20260709 13:18:51.862619 12630 raft_consensus.cc:2410] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:18:51.852005 12783 rpcz_store.cc:276] 0709 13:18:50.654739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.684537 (+ 29798us) service_pool.cc:224] Handling call
0709 13:18:50.684891 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:50.786474 (+101583us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:50.786507 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:50.786557 (+    50us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:50.786591 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:51.762669 (+976078us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b540019d2d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:51.762731 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:51.762893 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":233147776}
I20260709 13:18:51.862398 12779 rpcz_store.cc:276] 0709 13:18:50.634955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.651966 (+ 17011us) service_pool.cc:224] Handling call
0709 13:18:50.668826 (+ 16860us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:51.451244 (+782418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:51.451290 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:51.451915 (+   625us) log.cc:844] Serialized 26 byte log entry
0709 13:18:51.816364 (+364449us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:51.816432 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:51.816539 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:51.816604 (+    65us) raft_consensus.cc:1685] finished
0709 13:18:51.816623 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:51.816807 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5549,"raft.queue_time_us":309530,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":19433,"thread_start_us":1128131,"threads_started":2,"wal-append.queue_time_us":289037}
W20260709 13:18:51.851868 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 178915us
W20260709 13:18:51.876329 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 421647us
I20260709 13:18:51.877490 12672 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:51.853017 13349 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:51.902565 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 266380us
I20260709 13:18:51.735024 15653 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:51.918164 15653 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:51.918617 15653 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:51.921862 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:51.922545 12633 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:51.922539 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:51.933969 15458 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:51.934342 15458 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:51.934695 15458 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:51.934937 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:51.871507 15566 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:51.937110 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:51.755853 15728 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:51.943404 15519 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:51.943882 15519 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:51.944290 15519 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:51.945957 12776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1797) took 1292 ms. Trace:
W20260709 13:18:51.947042 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 311689us
I20260709 13:18:51.946352 12776 rpcz_store.cc:276] 0709 13:18:50.653309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.653430 (+   121us) service_pool.cc:224] Handling call
0709 13:18:50.653787 (+   357us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:51.123459 (+469672us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:51.123505 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:51.162860 (+ 39355us) log.cc:844] Serialized 52 byte log entry
0709 13:18:51.682666 (+519806us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:51.682725 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:51.682827 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:51.945747 (+262920us) raft_consensus.cc:1685] finished
0709 13:18:51.945768 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:51.945899 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68997,"raft.queue_time_us":129215,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":72592,"thread_start_us":908642,"threads_started":2,"wal-append.queue_time_us":646321}
W20260709 13:18:51.998492 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 49958.7us
I20260709 13:18:51.999879 12780 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:51.712198 15468 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:51.947861 15659 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:51.656772 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:51.861177 15668 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:52.006995 12775 raft_consensus.cc:1275] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:18:52.022027 12775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1806) took 1323 ms. Trace:
I20260709 13:18:51.760279 15463 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.245s
I20260709 13:18:51.889813 15525 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:18:52.016145 15456 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.007s	user 0.011s	sys 0.023s
I20260709 13:18:52.009711 12779 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:52.023193 12781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2088) took 1242 ms. Trace:
I20260709 13:18:52.022580 12775 rpcz_store.cc:276] 0709 13:18:50.698771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.874996 (+1176225us) service_pool.cc:224] Handling call
0709 13:18:52.006607 (+131611us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:52.021760 (+ 15153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:52.021965 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:52.036841 12781 rpcz_store.cc:276] 0709 13:18:50.780339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.904353 (+1124014us) service_pool.cc:224] Handling call
0709 13:18:52.022586 (+118233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:52.022807 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:52.022842 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:52.022895 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:52.022937 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:52.023004 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:52.023125 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:52.038188 12633 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:18:52.022027 15474 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:52.042003 15474 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:52.043545 15634 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:52.020426 15759 consensus_queue.cc:1059] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:18:52.018950 15659 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:52.025951 15525 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:52.022993 15668 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:51.755497 15720 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:52.047322 12782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1798) took 1393 ms. Trace:
W20260709 13:18:52.026765 15752 log.cc:927] Time spent T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
W20260709 13:18:52.035881 15456 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.099s	user 0.011s	sys 0.024s
I20260709 13:18:52.024150 15737 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
W20260709 13:18:52.234437 13348 leader_election.cc:336] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.548s (SENT)
I20260709 13:18:52.127604 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:52.054380 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:52.226115 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:52.226016 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 348635us
I20260709 13:18:52.238333 12747 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:52.117139 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:52.122573 12778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36488 (request call id 1805) took 1424 ms. Trace:
I20260709 13:18:52.242805 12672 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:52.137333 12632 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:52.120997 12703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1731) took 1482 ms. Trace:
I20260709 13:18:52.243398 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:52.258658 12625 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:52.092640 13385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 549) took 1609 ms. Trace:
I20260709 13:18:52.258028 12782 rpcz_store.cc:276] 0709 13:18:50.653668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.654304 (+   636us) service_pool.cc:224] Handling call
0709 13:18:50.654564 (+   260us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:51.507007 (+852443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:51.507041 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:51.537318 (+ 30277us) log.cc:844] Serialized 26 byte log entry
0709 13:18:52.020550 (+483232us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:52.020591 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:52.020686 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:52.047094 (+ 26408us) raft_consensus.cc:1685] finished
0709 13:18:52.047113 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:52.047263 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14397,"raft.queue_time_us":297632,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":42532,"thread_start_us":1282689,"threads_started":2,"wal-append.queue_time_us":463590}
I20260709 13:18:52.243080 13384 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:18:52.087574 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:52.255796 12778 rpcz_store.cc:276] 0709 13:18:50.698444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.708586 (+ 10142us) service_pool.cc:224] Handling call
0709 13:18:52.122320 (+1413734us) spinlock_profiling.cc:243] Waited 21.6 ms on lock 0x7b54002cc254. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:52.122479 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49751808}
W20260709 13:18:52.269019 13383 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51754 (request call id 553) took 1753 ms (client timeout 2032 ms). Trace:
I20260709 13:18:52.050923 15659 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:52.269519 13383 rpcz_store.cc:269] 0709 13:18:50.497542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.502061 (+  4519us) service_pool.cc:224] Handling call
0709 13:18:52.127456 (+1625395us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b5800095198. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28de9c 00007f4c6a28da5b 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c723780a0 00007f4c72377984 00007f4c6b8f26d4 00007f4c6b8f2662
0709 13:18:52.251004 (+123548us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12631808}
I20260709 13:18:52.257011 12703 rpcz_store.cc:276] 0709 13:18:50.638775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.744981 (+106206us) service_pool.cc:224] Handling call
0709 13:18:52.120721 (+1375740us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b54001ccfd4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:52.120909 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":674826496}
I20260709 13:18:52.051895 15566 raft_consensus.cc:697] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:52.259521 13385 rpcz_store.cc:276] 0709 13:18:50.482758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.482902 (+   144us) service_pool.cc:224] Handling call
0709 13:18:51.664248 (+1181346us) spinlock_profiling.cc:243] Waited 5.37 ms on lock 0x7b5800095198. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c7238bfc0 00007f4c72376929 00007f4c723763d2 00007f4c6b8edbd4
0709 13:18:51.664434 (+   186us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:51.664803 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:51.664836 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:51.665315 (+   479us) log.cc:844] Serialized 26 byte log entry
0709 13:18:52.034396 (+369081us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:52.034452 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:52.034552 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:52.053117 (+ 18565us) raft_consensus.cc:1685] finished
0709 13:18:52.053139 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:52.092573 (+ 39434us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1084,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":76,"spinlock_wait_cycles":12361600,"thread_start_us":368722,"threads_started":1,"wal-append.queue_time_us":374418}
I20260709 13:18:52.085547 15451 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:52.048373 15474 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:52.275934 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 47401.6us
I20260709 13:18:52.289397 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:52.291139 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 48411.3us
I20260709 13:18:52.292290 12747 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:52.115541 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:52.293860 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:52.298971 12707 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:52.252183 15759 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:52.051709 15525 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:52.219959 15750 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.085907 15675 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.212s
I20260709 13:18:52.301437 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:52.156168 15757 consensus_queue.cc:1048] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.140s
I20260709 13:18:52.063081 15753 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.238s
I20260709 13:18:52.162178 15480 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.139s
I20260709 13:18:52.325486 13384 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:18:52.296267 15441 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:52.293572 15438 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:18:52.335546 13348 leader_election.cc:336] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.569s (SENT)
W20260709 13:18:52.404995 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 69517.3us
I20260709 13:18:52.294613 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:52.357164 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 60788.7us
W20260709 13:18:52.310626 15651 consensus_meta.cc:287] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.639s	user 0.006s	sys 0.014s
I20260709 13:18:52.358681 15555 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:52.324374 15750 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:52.364166 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:52.303630 15451 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:52.393059 15753 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:52.394052 15783 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:52.401144 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 112614us
I20260709 13:18:52.136811 15658 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.190s
I20260709 13:18:52.420277 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:52.418579 15498 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.428550 12703 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:52.433617 13383 raft_consensus.cc:2393] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 10: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:52.286166 15566 consensus_queue.cc:237] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:52.444743 13384 raft_consensus.cc:2410] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
W20260709 13:18:52.450960 15772 log.cc:927] Time spent T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:18:52.453893 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:52.455399 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:52.458364 12672 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:52.418236 15312 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.024s
W20260709 13:18:52.438891 15778 log.cc:927] Time spent T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:18:52.442862 15658 consensus_queue.cc:1243] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 639ab2db265f4747bb7afb93e833f530 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:18:52.422240 15651 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:52.427022 15555 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:52.478055 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:52.438184 15451 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:52.467983 15312 consensus_queue.cc:1243] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:18:52.444195 15441 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:18:52.487407 15784 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:18:52.458915 15498 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:52.467180 15753 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:18:52.446194 15783 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:52.545765 15777 log.cc:927] Time spent T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
W20260709 13:18:52.551806 15458 consensus_meta.cc:287] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.617s	user 0.001s	sys 0.009s
W20260709 13:18:52.557396 15519 consensus_meta.cc:287] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.613s	user 0.012s	sys 0.005s
I20260709 13:18:52.589160 15783 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:52.591187 15555 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:52.599206 15653 consensus_meta.cc:287] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.680s	user 0.010s	sys 0.001s
I20260709 13:18:52.600337 15653 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:52.602782 15498 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:52.568960 15668 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:52.629019 15668 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:52.572880 15458 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:52.637264 15451 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:52.620132 15519 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:52.641434 15519 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:52.646869 15525 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:52.623536 15659 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:52.621814 15750 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:52.681301 15659 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:52.749645 12627 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.001s	sys 0.000s
I20260709 13:18:52.757171 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:52.997080 12627 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.002s	sys 0.000s
I20260709 13:18:52.997529 12630 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:18:52.758940 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:52.759090 12633 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:18:52.808204 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 366519us
I20260709 13:18:52.809974 12783 raft_consensus.cc:1240] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:52.813854 13377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 524) took 1514 ms. Trace:
I20260709 13:18:52.815356 13378 raft_consensus.cc:2393] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 6: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:53.002645 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:52.850301 15793 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:18:52.867409 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 409005us
I20260709 13:18:52.470611 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:52.883567 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:52.891541 13384 raft_consensus.cc:2393] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 9: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:52.896730 15794 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:52.904424 15796 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:52.960310 15525 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:52.997795 15659 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:53.014132 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 46515us
I20260709 13:18:53.011026 12630 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:53.032001 15794 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:53.036077 15796 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:53.036718 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:53.038405 13381 raft_consensus.cc:1813] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Handling vote request from an unknown peer 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:53.039460 13381 raft_consensus.cc:2410] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:18:53.040129 13381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51768 (request call id 529) took 1034 ms. Trace:
I20260709 13:18:53.040594 13381 rpcz_store.cc:276] 0709 13:18:52.005246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.439999 (+434753us) service_pool.cc:224] Handling call
0709 13:18:53.039993 (+599994us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:53.042241 13383 raft_consensus.cc:2393] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:53.044341 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:53.047205 13383 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:53.045840 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:53.050395 12633 raft_consensus.cc:2410] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:18:53.045389 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:52.999472 13377 rpcz_store.cc:276] 0709 13:18:51.299596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.873334 (+573738us) service_pool.cc:224] Handling call
0709 13:18:51.873836 (+   502us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:51.874388 (+   552us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:51.874442 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:51.994131 (+119689us) log.cc:844] Serialized 26 byte log entry
0709 13:18:52.050078 (+ 55947us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:52.050136 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:52.050229 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:52.813116 (+762887us) raft_consensus.cc:1685] finished
0709 13:18:52.813177 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:52.813593 (+   416us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":671,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":100984,"thread_start_us":55534,"threads_started":1,"wal-append.queue_time_us":298769}
I20260709 13:18:53.052922 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:53.053094 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:53.045336 12632 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:18:53.051599 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:53.053812 13377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51768 (request call id 530) took 1048 ms. Trace:
I20260709 13:18:53.055100 13377 rpcz_store.cc:276] 0709 13:18:52.005604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.052534 (+1046930us) service_pool.cc:224] Handling call
0709 13:18:53.053731 (+  1197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:53.061342 13381 raft_consensus.cc:2393] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 8: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:53.057083 15794 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:53.067111 13383 raft_consensus.cc:2410] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:18:53.068909 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:53.070324 12672 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:53.067891 15659 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:53.073016 12672 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:53.076000 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:53.078369 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.076476 15794 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:53.085393 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.077101 13383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51768 (request call id 531) took 1069 ms. Trace:
I20260709 13:18:53.088729 13383 rpcz_store.cc:276] 0709 13:18:52.006011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.050214 (+1044203us) service_pool.cc:224] Handling call
0709 13:18:53.075520 (+ 25306us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:53.075178 13381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51762 (request call id 549) took 1188 ms. Trace:
I20260709 13:18:53.094360 12778 raft_consensus.cc:3055] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:53.094825 12778 raft_consensus.cc:740] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:53.094169 13381 rpcz_store.cc:276] 0709 13:18:51.880030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.052639 (+1172609us) service_pool.cc:224] Handling call
0709 13:18:53.067861 (+ 15222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:53.097930 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:53.099789 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 92131.9us
I20260709 13:18:53.103145 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.104801 12632 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:53.113955 15785 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:53.114179 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.114500 15785 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:53.115044 15785 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:53.105846 15750 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:53.106977 15796 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:53.110697 12634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2245) took 2310 ms. Trace:
I20260709 13:18:53.096159 12778 consensus_queue.cc:260] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:53.073438 15659 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:53.116909 12634 rpcz_store.cc:276] 0709 13:18:50.800551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.804002 (+  3451us) service_pool.cc:224] Handling call
0709 13:18:53.109081 (+2305079us) spinlock_profiling.cc:243] Waited 7.06 ms on lock 0x7b5800001298. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c7238bfc0 00007f4c72376929 00007f4c723763d2 00007f4c6b8edbd4
0709 13:18:53.109510 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:53.110081 (+   571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:53.110112 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:53.110174 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:53.110212 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.110280 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:53.110470 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16230272}
I20260709 13:18:53.133432 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:53.141374 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:53.116564 15750 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.143252 15750 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:53.145902 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.146731 12778 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:53.087497 15787 consensus_queue.cc:1059] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:18:53.153693 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:53.154697 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1726) took 2517 ms. Trace:
I20260709 13:18:53.155123 12702 rpcz_store.cc:276] 0709 13:18:50.637472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.768555 (+131083us) service_pool.cc:224] Handling call
0709 13:18:50.785778 (+ 17223us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:51.063286 (+277508us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:51.063316 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:51.063361 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:51.063397 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.154142 (+2090745us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b5400405454. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:53.154216 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:53.154488 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":637745536}
I20260709 13:18:53.156088 12672 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:53.184953 15437 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:53.185446 15437 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:53.185871 15437 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:53.186517 12780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2085) took 2407 ms. Trace:
I20260709 13:18:53.187036 12780 rpcz_store.cc:276] 0709 13:18:50.779180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.875123 (+1095943us) service_pool.cc:224] Handling call
0709 13:18:51.875506 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:52.000421 (+124915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.185851 (+1185430us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b540019d054. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:53.186252 (+   401us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":285139712}
W20260709 13:18:53.188481 13348 leader_election.cc:336] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.989s (SENT)
I20260709 13:18:53.195622 12778 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:18:53.196091 12778 pending_rounds.cc:85] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 2
I20260709 13:18:53.202527 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:53.203522 12776 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:53.202569 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.206948 12779 raft_consensus.cc:2393] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 8: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:53.205257 15785 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:53.198702 12778 pending_rounds.cc:107] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:18:53.210242 12778 raft_consensus.cc:2889] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:53.212508 13381 raft_consensus.cc:1240] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:18:53.202651 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.213392 15796 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:53.217724 15785 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:53.218039 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.219084 12775 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
W20260709 13:18:53.224160 12626 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39590 (request call id 381) took 3927 ms (client timeout 1898 ms). Trace:
I20260709 13:18:53.224741 15312 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.225813 15312 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:53.200830 13348 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:53.229080 15312 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:53.230872 15462 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:18:53.231601 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 75722.6us
W20260709 13:18:53.224820 12626 rpcz_store.cc:269] 0709 13:18:49.296988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.297164 (+   176us) service_pool.cc:224] Handling call
0709 13:18:53.223585 (+3926421us) spinlock_profiling.cc:243] Waited 2.15 s on lock 0x7b540004bad4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:53.223885 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4937091328}
I20260709 13:18:53.235462 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:53.236758 12672 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:53.237898 15749 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:53.239046 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:53.240077 12634 raft_consensus.cc:2376] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9. Re-sending same reply.
W20260709 13:18:53.240940 13349 leader_election.cc:336] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.548s (SENT)
I20260709 13:18:53.241469 15554 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:53.241544 13349 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:53.242477 13380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 552) took 2745 ms. Trace:
I20260709 13:18:53.242978 13380 rpcz_store.cc:276] 0709 13:18:50.497134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.502506 (+  5372us) service_pool.cc:224] Handling call
0709 13:18:51.147161 (+644655us) spinlock_profiling.cc:243] Waited 3.05 ms on lock 0x7b5800095198. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c7238bfc0 00007f4c72376929 00007f4c723763d2 00007f4c6b8edbd4
0709 13:18:51.147439 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:51.470476 (+323037us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:51.470505 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:51.470546 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:51.470578 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.241847 (+1771269us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b54005415d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:53.241932 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:53.242187 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":749475840}
W20260709 13:18:53.244498 13348 leader_election.cc:336] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.898s (SENT)
I20260709 13:18:53.247123 15651 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:53.249607 15653 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:53.250090 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:53.251220 12705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38496 (request call id 321) took 2464 ms (client timeout 1548 ms). Trace:
W20260709 13:18:53.251629 12705 rpcz_store.cc:269] 0709 13:18:50.786980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.787189 (+   209us) service_pool.cc:224] Handling call
0709 13:18:53.250850 (+2463661us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b54001081d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:53.251021 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":481919872}
I20260709 13:18:53.252141 12702 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:18:53.254143 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:53.255002 12706 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:53.255085 15437 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:53.256424 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:53.257870 15651 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:53.258189 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:53.266662 12706 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 10: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:53.260421 12705 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:18:53.261592 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:53.267827 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:53.268381 12709 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:53.268635 12706 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:53.262221 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:53.269481 12707 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:53.256455 12775 raft_consensus.cc:3055] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:53.272425 12775 raft_consensus.cc:740] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:53.274560 15796 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:53.275185 15739 raft_consensus.cc:3055] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:53.275766 15739 raft_consensus.cc:740] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:53.277032 12775 consensus_queue.cc:260] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:53.280171 15776 log.cc:927] Time spent T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.886s	user 0.009s	sys 0.015s
I20260709 13:18:53.281735 12775 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:53.283514 12783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2102) took 1959 ms. Trace:
I20260709 13:18:53.283929 12783 rpcz_store.cc:276] 0709 13:18:51.323622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.865361 (+541739us) service_pool.cc:224] Handling call
0709 13:18:51.865611 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:53.026077 (+1160466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.283230 (+257153us) spinlock_profiling.cc:243] Waited 944 ms on lock 0x7b54001fde54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:53.283449 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2170201984}
I20260709 13:18:53.286127 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:53.286902 12700 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:53.224900 15750 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:53.289029 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.289642 15750 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:53.289834 12782 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:53.292842 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.298019 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.299357 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:53.300083 12705 raft_consensus.cc:2376] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10. Re-sending same reply.
I20260709 13:18:53.300356 12778 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:18:53.301537 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:53.302088 12779 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:18:53.302211 12705 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
I20260709 13:18:53.304060 12700 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:18:53.257475 15437 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:53.305966 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:53.277056 15739 consensus_queue.cc:260] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:53.306915 12705 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:53.308463 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:53.309216 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.310173 12777 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:18:53.314333 12706 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:18:53.315055 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:53.315824 15739 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:53.317139 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:53.237669 12776 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:18:53.334213 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.335610 12779 raft_consensus.cc:2393] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 8: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:53.336148 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:53.337177 15462 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:53.338713 13349 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:53.339607 15462 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:53.340145 15462 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:53.340745 15462 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:53.341513 15653 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:53.346653 12707 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:18:53.347414 12775 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:18:53.347798 12775 pending_rounds.cc:85] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 4
I20260709 13:18:53.338726 15592 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:53.354681 12775 pending_rounds.cc:107] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:18:53.355072 12775 raft_consensus.cc:2889] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:53.356213 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.357075 12776 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:53.376677 13383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 525) took 2063 ms. Trace:
I20260709 13:18:53.390713 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.391705 12775 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
W20260709 13:18:53.394343 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:53.395572 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.399777 15456 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:53.400516 15456 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:53.401120 15456 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:53.401719 15549 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:53.354417 15592 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:53.386250 13383 rpcz_store.cc:276] 0709 13:18:51.300966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.146420 (+1845454us) service_pool.cc:224] Handling call
0709 13:18:53.285970 (+139550us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:53.286627 (+   657us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:53.286657 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:53.287227 (+   570us) log.cc:844] Serialized 26 byte log entry
0709 13:18:53.363815 (+ 76588us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:53.364189 (+   374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.364283 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:53.364331 (+    48us) raft_consensus.cc:1685] finished
0709 13:18:53.364343 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:53.364466 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5437,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":243,"thread_start_us":76139,"threads_started":1,"wal-append.queue_time_us":73752}
I20260709 13:18:53.420449 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.423667 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:53.424787 12597 connection.cc:256] RPC call timeout handler was delayed by -2.2785121780002555s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16910000000000003s for a response
I20260709 13:18:53.406725 15592 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:53.427173 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 85727.6us
W20260709 13:18:53.437840 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.426434 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:53.425544 12597 connection.cc:256] RPC call timeout handler was delayed by -2.2224041350000334s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2178s for a response
I20260709 13:18:53.442904 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.446550 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.449610 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.450309 12780 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:18:53.450594 15496 consensus_queue.cc:1048] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:53.444777 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.454064 12782 raft_consensus.cc:2393] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 9: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:53.454270 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.456354 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.465102 13378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51762 (request call id 552) took 1584 ms. Trace:
I20260709 13:18:53.467909 12783 raft_consensus.cc:3055] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:53.468107 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.469167 12776 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:18:53.468292 12783 raft_consensus.cc:740] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:53.465934 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:53.472074 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.473217 12777 raft_consensus.cc:2393] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 8: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:53.474400 12778 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:18:53.478497 13379 raft_consensus.cc:1240] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:18:53.478902 15650 raft_consensus.cc:3055] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:53.479373 15650 raft_consensus.cc:740] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:53.480625 15650 consensus_queue.cc:260] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:53.484045 12627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2265) took 1310 ms. Trace:
I20260709 13:18:53.484516 12627 rpcz_store.cc:276] 0709 13:18:52.174130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.174347 (+   217us) service_pool.cc:224] Handling call
0709 13:18:52.174722 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:52.174982 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:52.997825 (+822843us) log.cc:844] Serialized 14 byte log entry
0709 13:18:53.483305 (+485480us) log.cc:844] Serialized 14 byte log entry
0709 13:18:53.483534 (+   229us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:53.483599 (+    65us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:53.483682 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.483783 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:53.483951 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":417653,"prepare.queue_time_us":410620,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":64,"thread_start_us":834415,"threads_started":2,"wal-append.queue_time_us":83593}
I20260709 13:18:53.488031 15648 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:53.488782 15648 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:53.489679 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:53.490136 12597 connection.cc:256] RPC call timeout handler was delayed by -2.1535170850002032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1845s for a response
I20260709 13:18:53.491320 13381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 526) took 2190 ms. Trace:
I20260709 13:18:53.491926 13381 rpcz_store.cc:276] 0709 13:18:51.301250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.097473 (+1796223us) service_pool.cc:224] Handling call
0709 13:18:53.212028 (+114555us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:53.213053 (+  1025us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.491064 (+278011us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54005d7d50. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:53.491244 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":261741312}
I20260709 13:18:53.492548 15557 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:53.472815 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.493247 15557 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:53.493873 13382 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51768 (request call id 518) took 2196 ms (client timeout 1634 ms). Trace:
W20260709 13:18:53.494336 13382 rpcz_store.cc:269] 0709 13:18:51.297248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.297357 (+   109us) service_pool.cc:224] Handling call
0709 13:18:53.493419 (+2196062us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b5400465354. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:53.493633 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":363161984}
I20260709 13:18:53.495102 13377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51762 (request call id 562) took 1072 ms. Trace:
I20260709 13:18:53.495477 13377 rpcz_store.cc:276] 0709 13:18:52.422596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.142948 (+720352us) service_pool.cc:224] Handling call
0709 13:18:53.493881 (+350933us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:53.494193 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:53.494224 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:53.494281 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:53.494313 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.494363 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:53.494947 (+   584us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b5400465350. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:53.495043 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":671498496}
I20260709 13:18:53.496181 15557 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:53.497165 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 58165us
I20260709 13:18:53.497881 15557 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.497933 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:53.498798 15557 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:53.499677 13349 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:53.500949 15720 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:53.501714 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:53.502460 12631 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:53.502537 12627 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:18:53.510543 12775 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:53.477133 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.514802 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:53.515084 12780 raft_consensus.cc:3055] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:53.515585 12780 raft_consensus.cc:740] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:53.518155 12782 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:53.518262 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.467211 13378 rpcz_store.cc:276] 0709 13:18:51.881040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.936823 (+ 55783us) service_pool.cc:224] Handling call
0709 13:18:53.464743 (+1527920us) spinlock_profiling.cc:243] Waited 875 ms on lock 0x7b5400690854. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:53.464991 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2011509120}
I20260709 13:18:53.522550 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:53.525379 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.491271 15648 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:53.521142 13349 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:53.501426 15557 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:53.536448 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.538551 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.543330 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.543977 12778 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:53.545324 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.541797 12626 pending_rounds.cc:85] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 6
I20260709 13:18:53.548748 12626 pending_rounds.cc:107] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:18:53.546211 12779 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:53.551110 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.548992 12626 raft_consensus.cc:2889] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:53.533088 15651 raft_consensus.cc:2764] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 13:18:53.557495 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 66682.2us
I20260709 13:18:53.548357 15792 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:53.545537 12783 consensus_queue.cc:260] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:53.560495 15650 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:53.561235 12672 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:53.564352 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.554924 15653 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:53.560552 15792 raft_consensus.cc:697] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:53.560529 15465 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:53.570722 15792 consensus_queue.cc:237] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:53.571393 15465 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:53.563506 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 1.89176e+06us
I20260709 13:18:53.568184 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:53.576112 15664 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:53.579521 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:53.580363 12625 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:53.584633 15741 raft_consensus.cc:3055] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:53.585150 15741 raft_consensus.cc:740] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:53.585212 12780 consensus_queue.cc:260] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:53.586510 12780 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:53.586453 15741 consensus_queue.cc:260] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:53.588107 15741 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:53.555316 15462 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:53.581499 15536 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:53.605158 12783 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:53.608685 12631 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:53.612206 12598 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:53.613633 12517 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 981fb2f435e24d06a81fc02cf4591cc5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 19cd864c053744bebd96093a7982537e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:53.631966 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:53.637944 12531 catalog_manager.cc:5697] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 1 to 4, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 4 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:53.644477 15812 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:53.624567 15536 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:53.659174 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 58080.7us
I20260709 13:18:53.661204 13349 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:53.661818 15312 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:53.664018 15312 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:53.665715 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:53.666410 12673 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:53.667766 12673 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:53.668398 15664 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:53.669117 15664 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.669535 15664 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:53.674288 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:53.680217 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 50495.9us
W20260709 13:18:53.680740 12597 connection.cc:256] RPC call timeout handler was delayed by -1.987908133000019s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1946s for a response
I20260709 13:18:53.661204 15536 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:53.668344 12531 catalog_manager.cc:5697] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 4 to 5, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 5 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
W20260709 13:18:53.699492 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 138363us
W20260709 13:18:53.770800 12597 connection.cc:256] RPC call timeout handler was delayed by -1.9105081330001212s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20320000000000002s for a response
W20260709 13:18:53.771231 12597 connection.cc:256] RPC call timeout handler was delayed by -1.9105081330001212s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20320000000000002s for a response
W20260709 13:18:53.771517 12597 connection.cc:256] RPC call timeout handler was delayed by -1.8720419200001288s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2093s for a response
W20260709 13:18:53.771812 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.768630 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:53.695319 15812 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:18:53.771848 12597 connection.cc:256] RPC call timeout handler was delayed by -1.7791081330001361s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2178s for a response
I20260709 13:18:53.701647 15648 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:53.775471 12597 connection.cc:256] RPC call timeout handler was delayed by -1.5690443650000816s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1507s for a response
W20260709 13:18:53.772797 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 78162.3us
W20260709 13:18:53.777720 12597 connection.cc:256] RPC call timeout handler was delayed by -1.4620370850002473s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1646s for a response
W20260709 13:18:53.778128 12597 connection.cc:256] RPC call timeout handler was delayed by -1.3327138940001078s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.329s for a response
I20260709 13:18:53.751130 12531 catalog_manager.cc:5697] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 4 to 5, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 5 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:53.754184 15549 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:53.784446 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:53.788465 12709 raft_consensus.cc:2393] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
W20260709 13:18:53.790163 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 132375us
I20260709 13:18:53.806100 15437 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:53.809867 15720 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:53.789532 15814 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.818229 15814 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:53.820001 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:18:53.820823 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.822495 15462 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:53.781657 15370 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:53.829784 15557 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:53.830168 15370 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:18:53.831101 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:53.844897 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 181511us
I20260709 13:18:53.849247 12780 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
W20260709 13:18:53.852408 12703 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1733) took 1559 ms (client timeout 1815 ms). Trace:
I20260709 13:18:53.853935 12783 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:18:53.830698 15370 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:53.854651 12783 pending_rounds.cc:85] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 5
W20260709 13:18:53.846966 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 47755.9us
I20260709 13:18:53.855953 12783 pending_rounds.cc:107] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:18:53.821430 15814 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:53.856511 12783 raft_consensus.cc:2889] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:53.853142 12703 rpcz_store.cc:269] 0709 13:18:52.293086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.294366 (+  1280us) service_pool.cc:224] Handling call
0709 13:18:53.851734 (+1557368us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b540045f6d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:53.852188 (+   454us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49100160}
I20260709 13:18:53.849714 12780 pending_rounds.cc:85] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 3
W20260709 13:18:53.845491 15739 consensus_meta.cc:287] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.006s
I20260709 13:18:53.830701 15557 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:53.855625 12598 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:53.859094 12780 pending_rounds.cc:107] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:18:53.859416 13349 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:53.845276 12531 catalog_manager.cc:5697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 5 to 6, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 6 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:53.864971 12780 raft_consensus.cc:2889] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:53.865075 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 112645us
I20260709 13:18:53.796818 15648 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:53.876422 15536 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:53.845902 15592 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:53.829771 15812 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:53.880846 12597 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:53.877411 12531 catalog_manager.cc:5697] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 1 to 4, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 4 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:53.886354 12625 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:18:53.887255 15775 log.cc:927] Time spent T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 1.484s	user 0.011s	sys 0.014s
I20260709 13:18:53.890913 15592 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:53.893700 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:53.878527 15814 raft_consensus.cc:2764] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:18:53.903666 12598 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:53.906864 13383 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
W20260709 13:18:53.908310 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 100493us
W20260709 13:18:53.910207 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.910979 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:53.916200 15592 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:53.917086 15592 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:53.917542 15592 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.918004 15592 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:53.920289 12673 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:53.914225 12531 catalog_manager.cc:5697] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 2 to 6, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 6 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:53.930132 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:53.906136 15664 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:53.942386 12673 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:53.942725 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 61284.5us
I20260709 13:18:53.945153 15495 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:18:53.945611 15822 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:53.946099 15822 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:53.946475 15822 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:53.954957 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:53.955339 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 83723.9us
I20260709 13:18:53.962262 15451 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:53.971773 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:53.975500 15557 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:53.981372 15458 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:53.980641 15465 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:53.973356 12597 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:53.989729 12531 catalog_manager.cc:5697] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 1 to 5, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 5 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
W20260709 13:18:54.036980 15821 log.cc:927] Time spent T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:18:54.039578 15664 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:54.040431 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 79882.1us
I20260709 13:18:54.042122 12628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2258) took 2242 ms. Trace:
I20260709 13:18:54.042927 15822 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:53.965999 15648 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:54.042582 12628 rpcz_store.cc:276] 0709 13:18:51.799274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.799380 (+   106us) service_pool.cc:224] Handling call
0709 13:18:51.799762 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:53.569426 (+1769664us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:53.575149 (+  5723us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:53.655550 (+ 80401us) log.cc:844] Serialized 26 byte log entry
0709 13:18:53.879669 (+224119us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:53.879733 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.883690 (+  3957us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:54.041698 (+158008us) raft_consensus.cc:1685] finished
0709 13:18:54.041719 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:54.041928 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1549075,"raft.queue_time_us":1664924,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":186,"thread_start_us":235029,"threads_started":2,"wal-append.queue_time_us":254277}
I20260709 13:18:53.982362 15653 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:54.050222 15653 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:54.050227 15456 consensus_meta.cc:287] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.632s	user 0.010s	sys 0.003s
I20260709 13:18:54.051621 15456 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:54.054021 15456 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:54.057760 15822 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:54.066008 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:54.067278 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:54.067658 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:54.067965 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:54.066721 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 100547us
I20260709 13:18:54.064990 12672 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:54.072091 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 105652us
I20260709 13:18:54.077203 15814 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:54.079933 15554 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:54.080037 15814 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:54.080693 15814 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:54.082144 15817 log.cc:927] Time spent T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.384s	user 0.000s	sys 0.002s
I20260709 13:18:54.099766 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:54.100911 12531 catalog_manager.cc:5697] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:54.108310 15554 raft_consensus.cc:2764] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 13:18:54.121187 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 148022us
I20260709 13:18:54.120641 12531 catalog_manager.cc:5697] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:54.099267 15462 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:18:54.135156 12672 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:54.136291 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:54.137874 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:54.139341 12709 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:54.140915 15451 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:54.141373 15451 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:54.145977 15664 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:54.157423 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:54.157578 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:54.161253 12706 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:54.129309 12531 catalog_manager.cc:5697] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 2 to 4, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 4 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:54.133066 15814 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:54.174984 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:54.131006 15592 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:54.184273 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:54.185047 12706 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:54.188931 12672 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:54.193440 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:54.198784 15797 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:54.199318 15797 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:54.199842 15797 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:54.202095 12531 catalog_manager.cc:5697] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 2 to 5, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 5 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:54.205129 15812 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:54.213800 12597 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:54.206444 12531 catalog_manager.cc:5697] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 1 to 5, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 5 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:54.222330 15456 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:54.223351 15456 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:54.223804 15456 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:54.224330 12597 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:54.225237 15814 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:54.232219 15456 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:54.241940 15456 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:54.236750 15812 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:18:54.237923 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.238014 12597 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:54.238219 12598 connection.cc:256] RPC call timeout handler was delayed by -1.4241150200000448s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1815s for a response
I20260709 13:18:54.238315 15650 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:54.258178 15650 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:54.259788 12597 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:54.260514 15650 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:54.261086 15650 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:54.261319 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:54.261775 15648 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:54.262322 15648 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:54.262552 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:54.263366 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:54.242774 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:54.234499 12531 catalog_manager.cc:5697] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 3 to 4, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 4 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:54.264215 12709 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 4.
I20260709 13:18:54.238659 15465 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:54.262655 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:54.266194 12631 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:54.262712 15648 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:54.266561 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:54.277704 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:54.278419 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:54.279981 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:54.283602 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:54.285508 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:54.298847 12626 raft_consensus.cc:2410] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:18:54.300606 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:54.301513 12632 raft_consensus.cc:2393] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:54.301836 15465 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:54.302975 12625 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:18:54.304211 15465 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:54.309978 15456 consensus_queue.cc:1048] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:54.311930 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:54.312677 12706 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:54.313601 12707 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:18:54.314957 12531 catalog_manager.cc:5697] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 3 to 4, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 4 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:54.317163 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:54.265784 15370 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:54.330498 13348 leader_election.cc:336] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.567s (SENT)
I20260709 13:18:54.331456 15370 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:54.331734 13348 leader_election.cc:336] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 2.009s (SENT)
I20260709 13:18:54.332235 13348 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:54.332777 15653 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:54.336869 13348 leader_election.cc:336] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 2.014s (SENT)
I20260709 13:18:54.273849 15741 consensus_queue.cc:1059] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:18:54.338989 12531 catalog_manager.cc:5697] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:54.350293 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:54.274997 15554 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:18:54.335490 15653 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:54.358179 12709 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:18:54.358621 12703 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 13:18:54.359609 12709 raft_consensus.cc:1240] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:18:54.359992 12706 raft_consensus.cc:1240] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:18:54.360569 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:54.360913 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:54.361311 12709 raft_consensus.cc:2393] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:54.358615 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:54.362439 12707 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:18:54.366982 12706 raft_consensus.cc:2393] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:54.367427 12672 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:54.374020 12705 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:18:54.379124 12597 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:18:54.380172 13348 leader_election.cc:336] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 2.174s (SENT)
I20260709 13:18:54.380990 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:54.383905 12700 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:54.331749 15648 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:54.392370 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.382386 15664 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:54.393841 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:54.394972 12625 raft_consensus.cc:2410] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:18:54.404055 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:54.408911 15474 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:54.410358 15474 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:54.410991 12627 raft_consensus.cc:2393] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 5: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:54.368130 15462 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:54.417719 12632 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:18:54.416211 12705 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 8. Current term is 9. Ops: []
W20260709 13:18:54.418835 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 80261us
I20260709 13:18:54.423629 12703 raft_consensus.cc:1240] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:18:54.424297 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:54.423486 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:54.435460 12702 raft_consensus.cc:2393] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:54.435210 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:54.439399 12597 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:18:54.441787 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:54.443419 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.446633 13379 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:54.402159 15456 raft_consensus.cc:2764] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:18:54.412616 15474 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:54.455060 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:54.456282 15648 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:54.458750 15664 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:54.460062 15664 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:54.460172 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:54.461717 15822 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:54.462049 15664 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:54.466475 12705 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8. Ops: []
W20260709 13:18:54.466729 15831 log.cc:927] Time spent T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.052s	user 0.002s	sys 0.000s
I20260709 13:18:54.467689 12701 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8. Ops: []
W20260709 13:18:54.469892 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 404068us
I20260709 13:18:54.470793 12673 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:54.479913 15437 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:54.480461 15437 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:54.480891 15437 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:54.480504 15554 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:18:54.483697 15832 log.cc:927] Time spent T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:18:54.488023 12634 raft_consensus.cc:1240] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: []
W20260709 13:18:54.489825 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:54.490543 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.473207 12673 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:54.438687 15811 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:18:54.506361 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.437338 15592 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:54.522496 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:54.513680 12632 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8. Ops: [5.3-7.4]
I20260709 13:18:54.513800 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:54.446511 15812 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:54.511193 12597 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:54.524523 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:54.527016 12626 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:54.530821 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:54.529031 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 80349.2us
I20260709 13:18:54.527254 12633 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:18:54.536947 12631 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:18:54.534868 12628 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:18:54.447144 15451 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:54.512460 15811 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:54.514405 15370 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:54.540953 15741 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:54.543543 15474 raft_consensus.cc:3055] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:18:54.545552 15830 log.cc:927] Time spent T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:18:54.567432 15474 raft_consensus.cc:740] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:54.548085 12597 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:54.575956 15474 consensus_queue.cc:260] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:54.525629 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 65539.8us
I20260709 13:18:54.512887 15651 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:18:54.579761 15592 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:54.580179 12700 raft_consensus.cc:2393] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:54.580330 15451 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:54.504840 15792 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:18:54.561229 15496 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:54.582422 15741 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:54.584004 12780 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:18:54.584359 12780 pending_rounds.cc:85] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 6
I20260709 13:18:54.566385 15811 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:54.584830 12780 pending_rounds.cc:107] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:18:54.584928 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:54.559571 15814 raft_consensus.cc:2955] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 4 to 5, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:54.574163 15797 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:54.586376 15437 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:54.587733 15797 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:54.585884 15496 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:54.585143 12780 raft_consensus.cc:2889] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:54.526225 15812 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:54.589304 15437 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:54.613395 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.617717 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:54.618901 12774 raft_consensus.cc:2410] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 9 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:18:54.620333 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:54.577406 15474 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:54.621979 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:54.652822 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:54.653450 12672 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:54.653815 12778 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:54.655786 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:54.594017 15496 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:54.591920 15370 raft_consensus.cc:697] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:54.594272 15812 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:54.658598 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.666083 12700 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:54.666322 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:54.661293 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:54.668401 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:54.630513 15651 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.584527 15741 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:54.669865 12781 raft_consensus.cc:2393] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 8: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:54.589715 15557 consensus_queue.cc:1059] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:18:54.668707 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:18:54.670769 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 164374us
I20260709 13:18:54.671547 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:54.671307 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:54.672278 12782 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:18:54.672858 12673 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:54.674690 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:54.675329 12775 raft_consensus.cc:2393] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 8: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:54.673005 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:54.677418 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:54.676614 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 187250us
I20260709 13:18:54.678427 12774 raft_consensus.cc:2410] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 10 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:18:54.679220 12627 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 8. Current term is 9. Ops: [4.3-8.5]
I20260709 13:18:54.683476 12747 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:54.670476 15437 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:54.686230 12779 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:54.687438 12673 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:54.687603 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:54.669137 15835 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:54.692325 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:54.694844 15739 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:54.695041 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.695380 15739 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:54.686879 12634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:18:54.697057 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:54.697638 15739 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:54.698321 15838 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:54.677762 15465 raft_consensus.cc:3055] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:54.715552 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:54.699273 15822 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:54.717347 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:54.675740 15651 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:54.690569 15437 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:54.740768 15437 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:54.741757 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:54.726361 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 96053.2us
I20260709 13:18:54.736922 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:54.726683 13380 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:54.687069 15797 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:54.737358 13379 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:18:54.736879 13382 raft_consensus.cc:2955] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:54.772076 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:54.775314 12777 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:18:54.729211 15465 raft_consensus.cc:740] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:54.788031 12625 raft_consensus.cc:1240] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:54.763423 15797 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:54.774652 15838 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:54.793774 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:54.765532 15822 raft_consensus.cc:697] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:18:54.795130 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 122525us
I20260709 13:18:54.795286 12625 raft_consensus.cc:1240] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:18:54.798187 15797 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:54.801779 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:54.805655 15837 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.812234 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:54.814469 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:54.818259 12625 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:18:54.814706 12629 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:54.821821 12631 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:54.694414 15835 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:54.828577 12597 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:54.850797 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:54.841914 15650 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:54.859071 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.859033 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:54.790076 15370 consensus_queue.cc:237] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:54.861608 12778 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
W20260709 13:18:54.867427 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.879994 12631 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
W20260709 13:18:54.880911 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 102900us
I20260709 13:18:54.883427 15664 consensus_queue.cc:1059] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:54.831739 15837 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:54.887382 15837 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:54.887251 15496 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:54.888353 12626 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:54.888532 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:54.889165 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:54.865423 15741 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:18:54.890237 15741 consensus_queue.cc:1243] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 639ab2db265f4747bb7afb93e833f530 log is divergent from this leader: its last log entry 9.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:18:54.890777 15741 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:54.891075 15741 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:54.891369 15741 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:54.893795 15847 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:54.894460 15847 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:54.894869 15847 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:54.849686 15456 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:18:54.889745 12778 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:54.893864 15834 raft_consensus.cc:3055] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:18:54.899883 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 63208.6us
I20260709 13:18:54.901880 12629 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:18:54.837139 15465 consensus_queue.cc:260] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:54.907923 15465 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:54.909546 12776 raft_consensus.cc:2410] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:18:54.855440 15557 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:18:54.917253 12625 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:18:54.820807 15838 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:54.925691 12747 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:54.925279 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:54.930267 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 47334.5us
I20260709 13:18:54.930511 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:54.931875 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:54.932494 12631 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:18:54.933869 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:54.934702 12777 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:54.900110 15834 raft_consensus.cc:740] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:54.930943 12747 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:54.950148 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:54.938228 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:54.939287 12629 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:54.940002 15835 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:54.941054 15843 log.cc:927] Time spent T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:18:54.941684 15474 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:54.945111 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:54.945459 12778 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:18:54.949096 15837 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.965603 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:54.951038 12631 raft_consensus.cc:2410] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:18:54.951236 12630 raft_consensus.cc:2393] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 5: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:54.940029 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:54.922112 15811 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:54.960168 15834 consensus_queue.cc:260] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:54.978163 15834 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:54.979884 13380 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:54.962859 15797 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:54.963331 15838 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:54.965440 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.965950 12629 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:18:54.983155 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:54.984138 12631 raft_consensus.cc:2393] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:54.985020 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:54.974924 15451 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:54.985711 15451 consensus_queue.cc:1243] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer 3d539f8d95d343d8b1696803f18b30d2 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:18:54.976107 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:54.987603 12632 raft_consensus.cc:2393] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:54.988534 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:54.989823 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:54.990511 12632 raft_consensus.cc:2393] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 8: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:54.990377 15437 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:54.976706 15496 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:54.976634 12780 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:54.983320 15797 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:54.968026 12625 raft_consensus.cc:2393] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:54.992574 15437 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:54.969415 15837 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:54.995566 15837 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:54.914578 15822 consensus_queue.cc:237] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:54.998926 15651 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:54.999261 15811 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:54.999644 12634 raft_consensus.cc:606] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:55.000113 12634 raft_consensus.cc:614] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:55.005190 12633 raft_consensus.cc:1240] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:55.008113 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:55.009171 12778 raft_consensus.cc:2368] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 8. Current term is 9.
I20260709 13:18:55.009577 13349 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:55.010154 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:55.012493 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:55.013373 12781 raft_consensus.cc:2393] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 8: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:55.034442 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:55.042999 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:55.046669 13349 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:55.047338 12629 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:55.047665 12775 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:18:55.048012 12775 pending_rounds.cc:85] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 6
I20260709 13:18:55.048237 12775 pending_rounds.cc:107] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:18:55.048496 12775 raft_consensus.cc:2889] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:55.049785 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:55.051733 13382 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:18:55.052459 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:55.052675 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:55.053046 12780 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:55.053167 12776 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:55.054381 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:55.057044 12627 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:55.058528 12778 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:18:55.059240 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:55.044487 15739 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:55.058802 15842 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:55.074893 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:55.058471 15651 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:55.044245 15835 consensus_queue.cc:1059] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:18:55.077135 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:55.078697 12779 raft_consensus.cc:2393] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 9: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:55.048406 15592 consensus_queue.cc:1059] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:18:55.080421 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:55.069662 15739 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:55.090404 12776 raft_consensus.cc:2410] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:18:55.095760 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:18:55.137043 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.096329 15838 raft_consensus.cc:3055] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:55.095512 15856 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:18:55.089845 15739 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:55.159934 12627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
W20260709 13:18:55.169199 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.169333 12627 raft_consensus.cc:606] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:55.169780 12627 raft_consensus.cc:614] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:55.057377 15860 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:18:55.072767 15458 consensus_queue.cc:1059] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:18:55.184383 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:55.185004 12782 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:55.184043 15651 raft_consensus.cc:697] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:55.188128 15651 consensus_queue.cc:237] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:55.190949 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:55.192442 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:55.161464 15838 raft_consensus.cc:740] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:55.195230 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:55.205053 15554 raft_consensus.cc:3055] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:55.205628 15554 raft_consensus.cc:740] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:55.207122 15554 consensus_queue.cc:260] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:55.208319 15554 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:55.213973 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:18:55.214991 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.214785 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:55.137018 15812 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:55.223927 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 167430us
I20260709 13:18:55.237058 12629 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 9. Current term is 10. Ops: []
W20260709 13:18:55.237643 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 266617us
W20260709 13:18:55.243398 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.252677 15847 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:55.254233 15648 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.255119 15648 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:55.256155 15847 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:55.258112 15814 consensus_meta.cc:287] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.671s	user 0.008s	sys 0.004s
I20260709 13:18:55.258009 15648 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:55.223691 15741 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:55.221055 15858 consensus_queue.cc:1059] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.159s
I20260709 13:18:55.261386 12631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:18:55.270367 12631 raft_consensus.cc:606] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:55.270956 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.270821 12631 raft_consensus.cc:614] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:55.296334 12779 raft_consensus.cc:2393] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 8: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
W20260709 13:18:55.303694 15864 log.cc:927] Time spent T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:18:55.311131 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:55.300907 12747 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:55.338063 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:55.339059 13383 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:55.341495 13380 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7. Ops: []
W20260709 13:18:55.348379 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 270600us
I20260709 13:18:55.344045 15496 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:18:55.356340 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:55.356853 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:55.357276 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.276398 15838 consensus_queue.cc:260] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:55.358337 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.354063 12673 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:55.365263 12777 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:55.351285 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:55.367125 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:55.366806 12776 raft_consensus.cc:1240] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:55.385154 12673 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:55.358323 15474 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.408386 12673 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:55.384256 12777 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:55.383999 12672 connection.cc:256] RPC call timeout handler was delayed by -1.1953822729999501s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27740000000000004s for a response
W20260709 13:18:55.378530 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:55.460122 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 159355us
W20260709 13:18:55.386344 15873 log.cc:927] Time spent T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 13:18:55.388139 15865 log.cc:927] Time spent T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.253s	user 0.000s	sys 0.003s
W20260709 13:18:55.390409 15869 log.cc:927] Time spent T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:18:55.384280 15797 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:55.392549 15811 raft_consensus.cc:3055] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:18:55.393318 15868 log.cc:927] Time spent T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb: Append to log took a long time: real 0.089s	user 0.000s	sys 0.002s
I20260709 13:18:55.396893 15812 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:55.359791 15814 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:56830 (request call id 79) took 5214 ms. Trace:
I20260709 13:18:55.492592 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:55.388228 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:55.402637 15741 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:55.457890 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 96774.4us
I20260709 13:18:55.497038 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:55.497967 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:55.496197 12777 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:55.368400 15862 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:18:55.466905 12747 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:55.479668 12779 pending_rounds.cc:85] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 3
I20260709 13:18:55.484460 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:55.505419 12779 pending_rounds.cc:107] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:18:55.469794 15474 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:55.460618 15886 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:55.409027 12597 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:18:55.522398 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 113413us
I20260709 13:18:55.523574 12597 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:55.524627 12597 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:55.525776 15456 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:55.526252 15456 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:55.526692 15456 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:55.528264 15592 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:55.528789 15592 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:55.529239 15592 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:55.530066 15812 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:55.530764 15812 raft_consensus.cc:697] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:55.531970 15812 consensus_queue.cc:237] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:55.493739 12780 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:55.492188 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.505788 12779 raft_consensus.cc:2889] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:55.505784 12706 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:55.536813 12777 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:18:55.537689 15870 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:55.538478 15844 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:55.539044 15844 raft_consensus.cc:697] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:55.540020 15844 consensus_queue.cc:237] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:55.540889 15885 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.541682 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:55.541829 15885 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:55.543455 15877 log.cc:927] Time spent T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:18:55.543740 15867 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:55.544381 15867 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:55.546304 15879 log.cc:927] Time spent T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 13:18:55.486163 15720 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.466113 15834 consensus_queue.cc:1059] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.038s
W20260709 13:18:55.547071 15884 log.cc:927] Time spent T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:18:55.548036 15882 log.cc:927] Time spent T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:18:55.547689 15720 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:55.548872 15875 log.cc:927] Time spent T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:18:55.491844 15842 raft_consensus.cc:697] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:18:55.552572 15881 log.cc:927] Time spent T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:18:55.555284 15860 raft_consensus.cc:3055] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:55.555823 15860 raft_consensus.cc:740] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:55.559538 15886 raft_consensus.cc:697] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:18:55.560971 15863 log.cc:927] Time spent T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.427s	user 0.002s	sys 0.002s
I20260709 13:18:55.560753 15886 consensus_queue.cc:237] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:55.560519 15887 consensus_queue.cc:1059] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:18:55.586833 15887 raft_consensus.cc:3055] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:55.587226 15887 raft_consensus.cc:740] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:55.587752 12630 raft_consensus.cc:1217] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Deduplicated request from leader. Original: 6.4->[8.5-9.6]   Dedup: 8.5->[9.6-9.6]
I20260709 13:18:55.588304 12630 raft_consensus.cc:1240] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 9. Current term is 10. Ops: [8.5-9.6]
I20260709 13:18:55.588162 15887 consensus_queue.cc:260] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:55.498378 15437 consensus_queue.cc:1059] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:18:55.493113 15814 rpcz_store.cc:276] 0709 13:18:50.139078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.139218 (+   140us) service_pool.cc:224] Handling call
0709 13:18:50.148073 (+  8855us) log.cc:844] Serialized 494 byte log entry
0709 13:18:50.400170 (+252097us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b5400278ba0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d32b142 00007f4c6d30beaa 00007f4c6d312d53 00007f4c6d312cd9 00007f4c6d312c69 00007f4c6d312c31 00007f4c6d311dfd 000000000038f354 000000000038f199 00007f4c6a2c5475
0709 13:18:50.613115 (+212945us) spinlock_profiling.cc:243] Waited 2.5 ms on lock 0x7b180042c760. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2932a6 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d332b26 00007f4c6d32c49a 00007f4c6d32b714
0709 13:18:50.614122 (+  1007us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b5400278ba0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d332b26 00007f4c6d32c49a 00007f4c6d32b714 00007f4c6d30beaa 00007f4c6d312d53 00007f4c6d312cd9 00007f4c6d312c69 00007f4c6d312c31 00007f4c6d311dfd 000000000038f354
0709 13:18:55.351852 (+4737730us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4192,"raft.queue_time_us":3841,"raft.run_cpu_time_us":4890,"raft.run_wall_time_us":376706,"spinlock_wait_cycles":35883392,"thread_start_us":166920,"threads_started":1,"wal-append.queue_time_us":275320,"wal-append.run_cpu_time_us":11631,"wal-append.run_wall_time_us":1024748}
I20260709 13:18:55.603025 15437 raft_consensus.cc:3055] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:55.603256 15838 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:55.603685 15437 raft_consensus.cc:740] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:55.604682 15437 consensus_queue.cc:260] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:55.566812 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 105916us
I20260709 13:18:55.613479 15847 consensus_queue.cc:1059] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:55.515632 15837 consensus_queue.cc:1059] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:18:55.614661 15892 consensus_queue.cc:1059] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:55.486194 15797 raft_consensus.cc:697] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:55.615631 15892 raft_consensus.cc:3055] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:55.616279 15892 raft_consensus.cc:740] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:55.616602 15797 consensus_queue.cc:237] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:55.617406 15892 consensus_queue.cc:260] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:55.618917 15892 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:55.567044 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:55.582827 15870 raft_consensus.cc:3055] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:55.619693 12708 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:18:55.590042 15741 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:55.618968 15536 raft_consensus.cc:3055] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:55.623574 15536 raft_consensus.cc:740] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:55.624643 15536 consensus_queue.cc:260] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:55.625965 15536 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:55.489554 15811 raft_consensus.cc:740] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:55.615624 15837 raft_consensus.cc:3055] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:55.630051 15837 raft_consensus.cc:740] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:55.630702 15811 consensus_queue.cc:260] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:55.631273 15837 consensus_queue.cc:260] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:55.632450 15811 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:55.633137 15837 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:55.634930 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 110373us
W20260709 13:18:55.620349 13349 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.739s (SENT) [suppressed 34 similar messages]
I20260709 13:18:55.635835 12597 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:18:55.649163 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.619956 15870 raft_consensus.cc:740] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:55.639365 15741 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:55.657210 15847 consensus_queue.cc:1059] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:18:55.660710 15741 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:55.552587 15842 consensus_queue.cc:237] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:55.661108 15835 consensus_queue.cc:1059] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:18:55.674533 15835 raft_consensus.cc:3055] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:55.675014 15835 raft_consensus.cc:740] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:55.676519 15835 consensus_queue.cc:260] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:55.661406 15870 consensus_queue.cc:260] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:55.556731 15860 consensus_queue.cc:260] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:55.561491 15458 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.589526 15887 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:55.728853 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.741899 15860 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:55.733675 15458 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:55.859774 13348 leader_election.cc:336] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.654s (SENT)
I20260709 13:18:55.862105 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:55.863472 12598 leader_election.cc:336] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.691s (SENT)
I20260709 13:18:55.865410 12598 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:55.863993 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 228229us
I20260709 13:18:55.865484 13348 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:55.866650 12597 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:55.871388 12625 raft_consensus.cc:1240] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:55.875062 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:55.876104 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:55.876892 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:55.877470 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:55.877874 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:55.878521 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 258180us
I20260709 13:18:55.879232 15899 consensus_queue.cc:1059] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:55.883205 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:55.884593 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:55.885413 12707 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:55.882717 15862 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:55.889526 12597 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:55.870718 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6609d5f93ab048558ea46e511ef3b2ec with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:18:55.891641 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 210283us
I20260709 13:18:55.895306 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:55.861696 15835 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:55.896310 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:55.903232 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:55.901607 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.861992 15437 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:55.912319 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 48879.1us
W20260709 13:18:55.913919 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:55.914538 12598 leader_election.cc:336] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.795s (SENT)
I20260709 13:18:55.862053 15878 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:18:55.919500 12598 leader_election.cc:336] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.807s (SENT)
I20260709 13:18:55.921357 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:55.922196 12700 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:55.888373 15862 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:55.893155 15557 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:55.870621 15496 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:18:55.925388 12517 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 981fb2f435e24d06a81fc02cf4591cc5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 19cd864c053744bebd96093a7982537e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:18:55.916471 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.896306 15841 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:55.861786 15856 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:55.914325 15474 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:55.896425 15648 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:55.895802 15870 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:55.918897 15554 consensus_meta.cc:287] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.710s	user 0.038s	sys 0.032s
W20260709 13:18:55.928123 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 437025us
I20260709 13:18:55.902793 15465 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:18:55.926199 15822 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.829s	user 0.005s	sys 0.048s
I20260709 13:18:55.936038 12780 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:18:55.939455 15465 raft_consensus.cc:697] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:55.941128 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:55.928682 15862 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:55.957041 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:55.937332 15822 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.846s	user 0.005s	sys 0.048s
I20260709 13:18:55.962961 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:55.934324 15648 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:55.977124 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:55.977521 12628 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:18:55.941852 15814 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:55.926671 15878 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:56.002384 12782 consensus_meta.cc:287] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.780s	user 0.016s	sys 0.008s
I20260709 13:18:56.002825 12782 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:55.934561 15496 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:56.022811 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.978673 15648 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:56.014113 12705 consensus_meta.cc:287] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.517s	user 0.017s	sys 0.029s
W20260709 13:18:56.014042 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 94578.5us
I20260709 13:18:55.966022 15720 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:56.016943 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:55.966192 15867 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:56.018203 15739 consensus_meta.cc:287] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.822s	user 0.009s	sys 0.012s
I20260709 13:18:55.934523 15856 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:56.026510 12705 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:18:55.944650 15465 consensus_queue.cc:237] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:56.014967 15885 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:56.030292 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 135141us
I20260709 13:18:56.040689 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:56.051108 12598 leader_election.cc:336] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.912s (SENT)
I20260709 13:18:56.041081 12625 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:56.051592 12598 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:56.043527 15834 raft_consensus.cc:3055] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:56.053454 12747 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:56.038479 15867 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:56.074107 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:56.054525 15834 raft_consensus.cc:740] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:56.048763 15592 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:56.063961 15812 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:56.080013 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.049501 15739 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:56.077878 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 182682us
I20260709 13:18:56.083191 12597 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:56.084259 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:56.085065 12705 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:56.086524 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 181645us
I20260709 13:18:56.087687 13348 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:56.091370 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:56.091683 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:56.092741 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:56.098062 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:56.099789 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:56.107434 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:56.109920 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:56.111550 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:56.113648 12783 raft_consensus.cc:2393] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 8: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:56.114660 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:56.080317 15900 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:56.073479 15867 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:56.130990 15907 log.cc:927] Time spent T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:18:56.134618 15812 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:56.193917 15812 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:56.194141 15496 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:56.155949 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.165552 15911 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:18:56.174510 15908 log.cc:927] Time spent T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:18:56.178238 15896 log.cc:927] Time spent T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:18:56.194612 15812 raft_consensus.cc:697] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:18:56.135982 12747 connection.cc:256] RPC call timeout handler was delayed by -1.1675267299999632s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
W20260709 13:18:56.219489 12747 connection.cc:256] RPC call timeout handler was delayed by -1.1405267299999196s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1557s for a response
I20260709 13:18:56.123227 15900 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:56.182829 15864 log.cc:927] Time spent T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:18:56.203653 12705 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
W20260709 13:18:56.219903 12747 connection.cc:256] RPC call timeout handler was delayed by -1.101826729999857s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16000000000000003s for a response
W20260709 13:18:56.221225 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 133596us
I20260709 13:18:56.222905 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:56.224982 12747 connection.cc:256] RPC call timeout handler was delayed by -1.0559267300000101s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16510000000000002s for a response
I20260709 13:18:56.225687 12700 raft_consensus.cc:2393] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:56.226411 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:56.231906 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 148808us
I20260709 13:18:56.233017 12597 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:56.206408 15911 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:56.236660 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 148353us
I20260709 13:18:56.239295 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:56.241838 13385 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:56.226112 15900 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:56.253989 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:18:56.254561 12775 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:56.255411 15651 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:56.256188 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:56.256804 15651 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:56.232298 15814 raft_consensus.cc:697] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:56.258312 15814 consensus_queue.cc:237] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:56.247437 15842 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.260561 15842 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:56.247902 15496 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:56.262885 12700 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:56.263934 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:56.264995 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:56.227746 15878 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:56.269153 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:56.274524 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:56.270205 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:56.275269 12782 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:18:56.275390 12780 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:56.198150 15834 consensus_queue.cc:260] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:56.278858 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.237483 15911 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:56.286235 13384 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:56.268613 15904 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.270790 15496 raft_consensus.cc:697] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:56.296365 12780 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
W20260709 13:18:56.288489 15536 consensus_meta.cc:287] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.662s	user 0.019s	sys 0.015s
I20260709 13:18:56.257367 15651 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:56.279181 15835 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.306406 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:56.320472 12707 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:56.321694 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:56.322902 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:56.323827 15887 consensus_meta.cc:287] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.595s	user 0.016s	sys 0.000s
I20260709 13:18:56.323915 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:56.324900 12707 raft_consensus.cc:2393] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 8: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:56.307467 15592 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:56.307673 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:56.327831 13382 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
W20260709 13:18:56.313768 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.329672 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:56.302250 15834 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:56.334141 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:56.306217 15856 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:56.342648 15592 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:18:56.343477 15741 consensus_meta.cc:287] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.674s	user 0.014s	sys 0.018s
I20260709 13:18:56.343339 15592 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:56.344033 15835 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:56.331802 15887 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:56.345332 15887 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:56.346393 15835 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:56.343936 15741 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:56.347536 15887 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:18:56.348961 15838 consensus_meta.cc:287] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.745s	user 0.018s	sys 0.021s
I20260709 13:18:56.350210 12775 raft_consensus.cc:1240] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 5. Current term is 6. Ops: []
W20260709 13:18:56.351155 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 102914us
I20260709 13:18:56.354561 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:56.355777 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:56.359184 12704 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:56.352017 15739 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:56.357745 12701 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:18:56.357542 15496 consensus_queue.cc:237] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:56.360270 12672 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:56.361027 15437 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:56.361055 15862 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:56.362347 12627 raft_consensus.cc:1240] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8. Ops: []
W20260709 13:18:56.358193 15860 consensus_meta.cc:287] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.615s	user 0.015s	sys 0.003s
I20260709 13:18:56.363564 15862 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:56.365859 15592 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:56.356775 15651 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:56.305960 15812 consensus_queue.cc:237] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:56.319363 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:18:56.351742 15837 consensus_meta.cc:287] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.718s	user 0.023s	sys 0.023s
I20260709 13:18:56.362505 12672 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:56.372681 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:56.358486 15456 consensus_meta.cc:287] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.831s	user 0.018s	sys 0.003s
I20260709 13:18:56.376237 15456 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:56.377720 15909 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:56.373961 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:56.349392 15741 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:56.376540 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:56.378388 12672 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:56.371842 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:56.379166 15451 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:56.379647 12672 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:56.379961 12632 raft_consensus.cc:2410] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:18:56.380443 15909 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:56.380946 12672 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:56.381731 15909 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:56.381222 12704 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:56.382418 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:56.381834 15456 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:56.383195 15878 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:56.383394 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:56.383716 15909 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:56.386960 12782 raft_consensus.cc:1240] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:18:56.388162 12776 raft_consensus.cc:1240] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:56.388548 12782 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:56.392153 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:56.393539 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:56.394886 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:56.398005 12780 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:56.399082 12704 raft_consensus.cc:2393] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 5: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:56.404731 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:56.404083 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:56.406430 12707 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:18:56.395256 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:56.406716 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:56.406972 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:56.416739 12672 leader_election.cc:400] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 8. Current term is 9.
I20260709 13:18:56.425988 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:56.432003 12706 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
I20260709 13:18:56.409031 15909 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.436724 12782 raft_consensus.cc:1275] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:56.436887 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:56.436218 15867 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:56.449857 13378 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
W20260709 13:18:56.457927 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 52796.8us
I20260709 13:18:56.303993 15904 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:56.343890 15900 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:56.464882 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:56.463544 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 133069us
I20260709 13:18:56.473985 12672 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:56.474938 12706 raft_consensus.cc:1240] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:56.463675 15909 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:56.475521 15921 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:56.476064 15921 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:56.476974 15909 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:56.477087 15458 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:56.478264 15921 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:56.478350 12672 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:56.479812 12709 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:18:56.480302 12672 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:56.481096 15856 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:56.481284 15885 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:56.481621 15856 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:56.482074 15856 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:56.483011 15878 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:18:56.468453 15867 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:56.494602 15867 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:56.495471 13385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 543) took 1184 ms. Trace:
I20260709 13:18:56.495994 13385 rpcz_store.cc:276] 0709 13:18:55.311232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.311371 (+   139us) service_pool.cc:224] Handling call
0709 13:18:55.336693 (+ 25322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.245804 (+909111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.495060 (+249256us) spinlock_profiling.cc:243] Waited 905 ms on lock 0x7b54000596d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:56.495350 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2081130368}
I20260709 13:18:56.497992 15867 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:56.502657 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:56.503405 12704 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:56.505900 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:56.473109 12597 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:56.511551 12597 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:56.514353 15592 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:56.518596 15592 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:56.519045 15592 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:56.521087 15811 consensus_meta.cc:287] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.888s	user 0.009s	sys 0.011s
I20260709 13:18:56.510166 15886 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:18:56.526189 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.542222 15474 raft_consensus.cc:3055] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:56.542740 15474 raft_consensus.cc:740] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:56.543642 15474 consensus_queue.cc:260] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:56.544641 15474 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:56.546723 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1739) took 1064 ms. Trace:
I20260709 13:18:56.547204 12702 rpcz_store.cc:276] 0709 13:18:55.482793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.489267 (+  6474us) service_pool.cc:224] Handling call
0709 13:18:56.546530 (+1057263us) spinlock_profiling.cc:243] Waited 586 ms on lock 0x7b540034f154. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:56.546661 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1347449216}
I20260709 13:18:56.547020 12776 raft_consensus.cc:1275] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 4. (term mismatch)
I20260709 13:18:56.549068 12776 pending_rounds.cc:85] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 3
I20260709 13:18:56.549335 12776 pending_rounds.cc:107] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 13:18:56.549688 12776 raft_consensus.cc:2889] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:56.550418 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 70163.9us
I20260709 13:18:56.552589 12672 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:56.549185 15922 log.cc:927] Time spent T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
W20260709 13:18:56.557451 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:56.561695 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.563905 12627 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:56.566973 12627 raft_consensus.cc:1240] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:56.571743 12627 raft_consensus.cc:1275] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:56.568521 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:56.575719 12632 raft_consensus.cc:2393] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 10: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:18:56.577178 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.573210 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:56.587239 12672 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:56.582518 12627 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
W20260709 13:18:56.614830 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:56.628767 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.629325 12776 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:56.628638 12627 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:18:56.630213 12707 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:18:56.950254 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 320602us
I20260709 13:18:56.563159 15437 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:56.651729 15911 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:56.561975 15811 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:56.601729 15496 consensus_queue.cc:1059] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 13:18:56.594702 15878 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.085s
I20260709 13:18:56.632107 15451 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:56.594282 15834 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:18:56.713204 15930 log.cc:927] Time spent T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:18:56.950523 12704 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
W20260709 13:18:56.951298 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:56.957010 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.951187 12672 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:56.963721 12627 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:56.986768 12779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2153) took 1825 ms. Trace:
W20260709 13:18:56.958142 15892 consensus_meta.cc:287] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 1.322s	user 0.030s	sys 0.036s
W20260709 13:18:56.963701 15870 consensus_meta.cc:287] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 1.027s	user 0.016s	sys 0.003s
I20260709 13:18:56.996137 15911 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:56.944958 15549 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:57.001770 13384 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:57.001544 15549 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:57.002192 15904 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:57.002874 15720 consensus_queue.cc:1059] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:57.004856 15549 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:56.945228 15814 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:57.006750 15814 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:57.007843 15648 consensus_meta.cc:287] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.977s	user 0.014s	sys 0.017s
I20260709 13:18:57.008378 15648 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:56.953758 15917 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.009930 12629 raft_consensus.cc:1240] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:56.995957 15834 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:56.996557 12702 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:18:56.986037 15856 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:56.997783 12627 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:56.958381 15909 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:56.962894 15651 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:56.986414 15924 log.cc:927] Time spent T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.327s	user 0.000s	sys 0.008s
I20260709 13:18:56.992277 12779 rpcz_store.cc:276] 0709 13:18:55.161187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.471118 (+309931us) service_pool.cc:224] Handling call
0709 13:18:55.479359 (+  8241us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:55.582010 (+102651us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:55.582042 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:55.667243 (+ 85201us) spinlock_profiling.cc:243] Waited 57.7 ms on lock 0x7b2c00160c88. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6bcaba11 00007f4c6bcbb372 00007f4c6d3eda01 00007f4c6d3b243a 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2
0709 13:18:55.862809 (+195566us) log.cc:844] Serialized 52 byte log entry
0709 13:18:56.236035 (+373226us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:56.236081 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.236143 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:56.970320 (+734177us) raft_consensus.cc:1685] finished
0709 13:18:56.970344 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:56.986706 (+ 16362us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46236,"raft.queue_time_us":46874,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":120,"spinlock_wait_cycles":132808960,"thread_start_us":318133,"threads_started":1,"wal-append.queue_time_us":459860}
W20260709 13:18:57.048789 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 97686.2us
I20260709 13:18:57.010994 15814 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:56.971257 15370 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.078s
I20260709 13:18:57.041085 15904 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.987186 15592 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:57.072023 15900 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:57.069921 15904 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:57.074748 15904 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:57.077069 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:57.079103 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:57.079526 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:57.079890 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:57.080245 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.080902 15904 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:57.081521 15904 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:57.083570 15904 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:57.083636 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:57.084123 15904 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.084618 15904 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:57.085897 12633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2306) took 1129 ms. Trace:
I20260709 13:18:57.086414 12633 rpcz_store.cc:276] 0709 13:18:55.956908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.957025 (+   117us) service_pool.cc:224] Handling call
0709 13:18:55.977234 (+ 20209us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.031582 (+ 54348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:56.031629 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:56.031705 (+    76us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:56.031758 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.085538 (+1053780us) spinlock_profiling.cc:243] Waited 54.1 ms on lock 0x7b5400159fd4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:57.085628 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.085818 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124438784}
I20260709 13:18:57.018298 15917 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:57.057696 15870 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:57.090945 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.097244 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:57.054726 15885 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:57.093176 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:57.094830 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:57.099829 12779 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:57.099895 12780 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:57.096000 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:57.106341 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:57.107116 12776 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:57.112534 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:57.113152 13382 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:18:57.049431 15651 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:57.058383 15878 consensus_queue.cc:1059] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:18:57.118251 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:57.127089 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.097713 15856 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:57.132580 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:57.066381 15797 consensus_queue.cc:1059] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:18:57.144846 12776 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
W20260709 13:18:57.155624 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.129963 15648 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:57.159010 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:57.175240 12776 raft_consensus.cc:2393] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 9: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:57.161062 12780 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
W20260709 13:18:57.176077 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 231510us
I20260709 13:18:57.162333 12746 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:57.123980 15885 consensus_queue.cc:1048] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:18:57.134048 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:57.179842 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:57.111589 15870 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:57.187525 15797 consensus_queue.cc:1059] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:57.190620 12780 raft_consensus.cc:1275] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:18:57.192672 12780 raft_consensus.cc:1275] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:18:57.195371 12780 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:18:57.173936 15856 raft_consensus.cc:3055] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:57.214531 15856 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:57.215606 15856 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:57.216915 15856 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:57.196290 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:57.218633 12627 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 8: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:57.218894 15904 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:57.218940 12747 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:57.220333 12625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2303) took 1265 ms. Trace:
I20260709 13:18:57.220954 15904 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.221324 12747 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:57.217072 15834 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:57.222630 12781 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:57.196388 12673 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:57.199795 13378 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:18:57.206524 15946 consensus_queue.cc:1048] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:57.207500 15917 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:57.208266 15592 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:57.198427 15474 consensus_meta.cc:287] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.653s	user 0.003s	sys 0.014s
I20260709 13:18:57.185986 15900 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:57.228780 12779 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:18:57.220746 12625 rpcz_store.cc:276] 0709 13:18:55.955363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.958462 (+  3099us) service_pool.cc:224] Handling call
0709 13:18:55.958801 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.052029 (+ 93228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.220106 (+1168077us) spinlock_profiling.cc:243] Waited 82 ms on lock 0x7b540004a1d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:57.220275 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":188574080}
I20260709 13:18:57.230881 12673 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:57.232995 12783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36478 (request call id 2154) took 2061 ms (client timeout 2211 ms). Trace:
I20260709 13:18:57.221796 15904 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:57.237329 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:57.233047 15892 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:57.233922 15834 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:18:57.244340 12783 rpcz_store.cc:269] 0709 13:18:55.161479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.193498 (+ 32019us) service_pool.cc:224] Handling call
0709 13:18:57.222254 (+2028756us) spinlock_profiling.cc:243] Waited 909 ms on lock 0x7b54001fd954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:57.222415 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2090378624}
I20260709 13:18:57.200460 15811 raft_consensus.cc:697] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:57.286623 15870 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:57.281279 15878 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 13:18:57.301843 15878 consensus_queue.cc:1059] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:18:57.242125 15937 log.cc:927] Time spent T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.186s	user 0.003s	sys 0.000s
I20260709 13:18:57.233508 15886 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:57.299443 15904 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:57.241150 15900 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:57.293208 15946 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:18:57.282569 13382 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:57.281064 15797 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:57.242635 15651 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:57.311401 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 248180us
I20260709 13:18:57.290930 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:18:57.320138 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.335071 15911 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:57.322876 15900 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:57.357321 15911 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:57.335631 15856 raft_consensus.cc:3055] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:57.364164 15856 raft_consensus.cc:740] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:57.368055 12632 raft_consensus.cc:1275] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:18:57.336465 15797 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:57.365502 15856 consensus_queue.cc:260] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:57.373821 15797 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:57.390007 15946 consensus_queue.cc:1048] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:57.391593 15946 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:57.393625 15811 consensus_queue.cc:237] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:57.398398 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 145165us
I20260709 13:18:57.386366 15856 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:57.409390 12630 raft_consensus.cc:1240] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:57.410645 12778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1854) took 1141 ms. Trace:
W20260709 13:18:57.410946 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 72584.8us
I20260709 13:18:57.411151 12778 rpcz_store.cc:276] 0709 13:18:56.269061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.269640 (+   579us) service_pool.cc:224] Handling call
0709 13:18:56.286262 (+ 16622us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:56.292439 (+  6177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:56.293162 (+   723us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:56.293653 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 13:18:56.466202 (+172549us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:57.407890 (+941688us) log.cc:844] Serialized 14 byte log entry
0709 13:18:57.408513 (+   623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.408647 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:57.408711 (+    64us) raft_consensus.cc:1685] finished
0709 13:18:57.408732 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.410555 (+  1823us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":745795,"prepare.queue_time_us":753259,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":6435,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":6329,"thread_start_us":345929,"threads_started":2,"wal-append.queue_time_us":245809}
W20260709 13:18:57.438207 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 206545us
I20260709 13:18:57.438555 12775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1853) took 1171 ms. Trace:
W20260709 13:18:57.438982 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.439142 12775 rpcz_store.cc:276] 0709 13:18:56.267056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.267230 (+   174us) service_pool.cc:224] Handling call
0709 13:18:56.279798 (+ 12568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.350733 (+ 70935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.438212 (+1087479us) spinlock_profiling.cc:243] Waited 70.2 ms on lock 0x7b54002cbd54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:57.438477 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161476480}
I20260709 13:18:57.444231 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:57.445048 12704 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 8: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:57.451076 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:57.451915 12779 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:18:57.453686 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:57.454365 12780 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 8: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:57.456162 12780 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:57.451413 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:57.464916 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.476828 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:57.477106 12708 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:57.477921 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:57.478693 12709 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
I20260709 13:18:57.480191 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:57.486150 12778 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:18:57.486544 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:57.488034 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:57.488775 12707 raft_consensus.cc:2376] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7. Re-sending same reply.
I20260709 13:18:57.490022 12706 raft_consensus.cc:2393] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 8: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:57.484863 13349 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:57.491890 12777 raft_consensus.cc:1240] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8. Ops: []
W20260709 13:18:57.492187 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 270925us
I20260709 13:18:57.492094 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:57.492728 12702 raft_consensus.cc:2393] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:57.492970 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:57.509057 15842 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:57.512485 15931 raft_consensus.cc:3055] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:57.512987 15931 raft_consensus.cc:740] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:57.513901 15931 consensus_queue.cc:260] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:57.515228 15931 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:57.494056 15651 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:57.502465 15841 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:57.505577 15910 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:18:57.528607 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.535275 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:57.522742 15886 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.516955 15797 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:57.530471 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:57.518517 15965 consensus_queue.cc:1048] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:18:57.532815 15904 raft_consensus.cc:2764] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:18:57.532863 15910 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:57.520378 15948 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:18:57.570367 12631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2311) took 1606 ms. Trace:
I20260709 13:18:57.532320 15458 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:18:57.572710 12747 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:57.574841 12631 rpcz_store.cc:276] 0709 13:18:55.963645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.963779 (+   134us) service_pool.cc:224] Handling call
0709 13:18:55.964056 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:55.964258 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:56.092808 (+128550us) log.cc:844] Serialized 14 byte log entry
0709 13:18:57.570051 (+1477243us) spinlock_profiling.cc:243] Waited 6.01 ms on lock 0x7b180042c1c0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6cf5a4bc 00007f4c6a2a848b 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c2455 00007f4c6d3aa060 00007f4c6d3bc716 00007f4c6d3cd287
0709 13:18:57.570120 (+    69us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:57.570152 (+    32us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:57.570180 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.570234 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.570324 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":538,"prepare.queue_time_us":288334,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":113520,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":2505,"spinlock_wait_cycles":13821824,"thread_start_us":257618,"threads_started":2}
I20260709 13:18:57.569885 15910 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:57.553248 15886 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:57.587379 15886 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:57.554908 15962 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:57.594676 12631 raft_consensus.cc:1240] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:57.596689 15973 consensus_queue.cc:1059] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:57.625387 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:57.582823 15904 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:57.628850 12747 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:57.603417 15900 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:57.635381 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:57.639053 12774 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1856) took 1367 ms. Trace:
I20260709 13:18:57.639585 12774 rpcz_store.cc:276] 0709 13:18:56.271564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.271855 (+   291us) service_pool.cc:224] Handling call
0709 13:18:56.287266 (+ 15411us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:56.299750 (+ 12484us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:57.210249 (+910499us) spinlock_profiling.cc:243] Waited 1.75 ms on lock 0x7b1800317ce0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a23fc0a 00007f4c7397f219 00007f4c6a2c7a40 00007f4c6a2c24e4 00007f4c6a2c2455 00007f4c70daa3c7 00007f4c6d3bc9a7 00007f4c6d3cc0bf 00007f4c6d3cc038
0709 13:18:57.377946 (+167697us) log.cc:844] Serialized 14 byte log entry
0709 13:18:57.504413 (+126467us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:57.524388 (+ 19975us) log.cc:844] Serialized 52 byte log entry
0709 13:18:57.527712 (+  3324us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:57.527767 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.527868 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:57.638723 (+110855us) raft_consensus.cc:1685] finished
0709 13:18:57.638833 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.638988 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12128,"prepare.queue_time_us":919518,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":18303,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":18548,"spinlock_wait_cycles":10076288,"thread_start_us":226141,"threads_started":2,"wal-append.queue_time_us":158822}
I20260709 13:18:57.633841 15898 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:57.644038 15797 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:57.644384 15900 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:57.634479 15911 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:57.650197 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:18:57.650606 12703 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1742) took 2166 ms (client timeout 1912 ms). Trace:
I20260709 13:18:57.634335 15892 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:57.651603 15892 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:57.652002 15892 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:57.655334 15910 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:57.355787 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:57.651095 12631 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:57.662652 12747 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:57.664395 15962 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:57.665017 15962 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:57.665607 15962 consensus_queue.cc:1059] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:57.666568 15962 raft_consensus.cc:3055] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:57.666903 15962 raft_consensus.cc:740] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:57.667662 15962 consensus_queue.cc:260] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:57.668967 15962 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:57.651065 12703 rpcz_store.cc:269] 0709 13:18:55.483734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.483899 (+   165us) service_pool.cc:224] Handling call
0709 13:18:57.650399 (+2166500us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b540006e854. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:57.650535 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1329733376}
I20260709 13:18:57.671941 12631 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:57.677716 12673 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:57.679163 15370 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:57.679730 15370 raft_consensus.cc:697] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:18:57.680924 15925 log.cc:927] Time spent T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.712s	user 0.000s	sys 0.003s
I20260709 13:18:57.680725 15370 consensus_queue.cc:237] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:57.686195 12628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2314) took 1120 ms. Trace:
I20260709 13:18:57.686596 12628 rpcz_store.cc:276] 0709 13:18:56.565797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.565942 (+   145us) service_pool.cc:224] Handling call
0709 13:18:56.627996 (+ 62054us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.628201 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:56.660822 (+ 32621us) log.cc:844] Serialized 14 byte log entry
0709 13:18:57.107056 (+446234us) log.cc:844] Serialized 14 byte log entry
0709 13:18:57.685526 (+578470us) spinlock_profiling.cc:243] Waited 8.32 ms on lock 0x7b18002140c0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a23fc0a 00007f4c7397f219 00007f4c6a2c7a40 00007f4c6a2c24e4 00007f4c6a2c2455 00007f4c6d3aa060 00007f4c6d3bc716 00007f4c6d3cd287 00007f4c6d3cd208
0709 13:18:57.685692 (+   166us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:57.685746 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:57.685790 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.686021 (+   231us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.686138 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43454,"prepare.queue_time_us":39746,"prepare.run_cpu_time_us":7806,"prepare.run_wall_time_us":266082,"raft.queue_time_us":42636,"raft.run_wall_time_us":2785,"spinlock_wait_cycles":69570688,"thread_start_us":127089,"threads_started":1,"wal-append.queue_time_us":176810}
W20260709 13:18:57.688971 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 579131us
I20260709 13:18:57.684376 15910 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:57.689781 12597 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:57.690305 13384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 582) took 1393 ms. Trace:
I20260709 13:18:57.690665 13384 rpcz_store.cc:276] 0709 13:18:56.296824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.302490 (+  5666us) service_pool.cc:224] Handling call
0709 13:18:56.303457 (+   967us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.002215 (+698758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.690061 (+687846us) spinlock_profiling.cc:243] Waited 698 ms on lock 0x7b54004bfd54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:57.690248 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1605485440}
I20260709 13:18:57.691685 12597 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:57.692512 12597 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:57.721776 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:57.728583 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 64586us
I20260709 13:18:57.731142 12747 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:57.724392 15917 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:57.724575 15949 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:57.707917 15892 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:57.732929 15465 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.737082 15900 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:57.740137 15370 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:57.745710 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:57.751699 15943 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:18:57.752656 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.754845 15906 consensus_queue.cc:1048] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:57.755415 12598 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:57.755709 15953 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.756045 15917 raft_consensus.cc:697] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:57.756309 15931 consensus_queue.cc:1048] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:57.757139 12597 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:18:57.757618 15949 raft_consensus.cc:697] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:57.757620 15942 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:57.770597 15860 raft_consensus.cc:3055] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:57.769886 15465 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:57.777504 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:57.778115 12672 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:57.778955 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:57.801981 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:57.782119 12626 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:57.783542 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:57.802805 12704 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:18:57.803625 12708 raft_consensus.cc:2393] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 6: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:57.803910 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:57.794243 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:57.795084 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:57.776623 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:57.797109 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:57.807435 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 76375.2us
I20260709 13:18:57.807494 12702 raft_consensus.cc:2376] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10. Re-sending same reply.
I20260709 13:18:57.809306 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:57.817137 15465 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:57.784242 15965 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:57.822093 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.820736 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:57.793887 15860 raft_consensus.cc:740] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:57.829516 12747 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:57.793502 15983 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:57.811770 15814 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:57.813171 15437 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:57.784698 15943 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:57.838338 15949 consensus_queue.cc:237] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:57.842366 15917 consensus_queue.cc:237] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:57.843538 15892 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:57.793758 15953 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:57.846764 15860 consensus_queue.cc:260] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:57.847606 15953 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:57.784649 15370 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:57.848693 15860 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:57.825987 15966 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:57.850939 15370 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:57.830668 15965 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:57.854591 15965 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:57.859428 12707 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:18:57.834872 12598 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:57.835206 13349 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:57.835695 15834 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.831926 15465 consensus_queue.cc:1059] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:18:57.831494 15981 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:57.872164 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:57.862701 12782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2165) took 1475 ms. Trace:
I20260709 13:18:57.869422 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:57.871091 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:57.871170 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:57.895422 13349 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:57.872949 12782 rpcz_store.cc:276] 0709 13:18:56.387314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.466614 (+ 79300us) service_pool.cc:224] Handling call
0709 13:18:56.467842 (+  1228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.998725 (+530883us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:18:56.998763 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:56.998810 (+    47us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:56.998855 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.862349 (+863494us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b54001aefd4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:57.862415 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.862614 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":483,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"spinlock_wait_cycles":1220077312}
I20260709 13:18:57.899421 12598 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:57.895592 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:57.901727 15465 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:57.901808 15948 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:57.902661 15948 raft_consensus.cc:697] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:18:57.904000 15948 consensus_queue.cc:237] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:57.867266 15983 raft_consensus.cc:697] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:57.908016 15451 raft_consensus.cc:3055] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:57.907127 15983 consensus_queue.cc:237] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:57.908448 15451 raft_consensus.cc:740] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:57.822321 15980 consensus_queue.cc:1059] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 13:18:57.910203 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:57.900486 12626 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:18:57.872538 15648 raft_consensus.cc:2764] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:18:57.914448 13349 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:57.915225 12598 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:57.834051 15900 consensus_queue.cc:1059] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 13:18:57.897933 15458 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:18:57.918902 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.919694 12598 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:57.897269 15911 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:57.898471 15811 raft_consensus.cc:3055] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:57.955016 12598 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:57.928037 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:57.958545 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.959205 15878 raft_consensus.cc:3055] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:57.917989 15910 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:57.959592 15878 raft_consensus.cc:740] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:57.960489 15878 consensus_queue.cc:260] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:57.917091 15981 raft_consensus.cc:2764] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 13:18:57.926105 12705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1772) took 1660 ms (client timeout 1546 ms). Trace:
I20260709 13:18:57.894672 15834 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:57.964915 12705 rpcz_store.cc:269] 0709 13:18:56.266414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.925885 (+1659471us) service_pool.cc:211] Skipping call since client already timed out
0709 13:18:57.926018 (+   133us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:18:57.963562 12598 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:57.942293 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 71185.9us
I20260709 13:18:57.919857 15906 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:57.967378 12779 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:18:57.969954 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:58.150840 12777 raft_consensus.cc:2393] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 10: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:57.979229 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:58.151803 12782 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:18:57.931300 15458 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:58.154567 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.963167 15878 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:57.964550 15465 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:57.909397 15451 consensus_queue.cc:260] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:58.142479 12776 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:58.142901 12673 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:18:57.919497 15867 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:57.927628 15648 consensus_queue.cc:1059] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.243s
I20260709 13:18:58.146174 12629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2308) took 2188 ms. Trace:
I20260709 13:18:58.147616 12778 raft_consensus.cc:1275] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:18:57.965981 15950 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:18:58.148417 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:58.154958 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 191488us
W20260709 13:18:58.212606 15962 consensus_meta.cc:287] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.543s	user 0.016s	sys 0.004s
I20260709 13:18:57.957490 15811 raft_consensus.cc:740] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:18:58.224115 12517 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 981fb2f435e24d06a81fc02cf4591cc5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 19cd864c053744bebd96093a7982537e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:57.898720 15928 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.065s
W20260709 13:18:58.295123 15988 log.cc:927] Time spent T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:18:58.295907 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:58.172315 15458 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:58.199918 12776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1862) took 1103 ms. Trace:
W20260709 13:18:58.143069 12701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1737) took 2661 ms (client timeout 1642 ms). Trace:
W20260709 13:18:58.298362 15990 log.cc:927] Time spent T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
W20260709 13:18:58.298521 12701 rpcz_store.cc:269] 0709 13:18:55.482071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.482195 (+   124us) service_pool.cc:224] Handling call
0709 13:18:57.971831 (+2489636us) spinlock_profiling.cc:243] Waited 491 ms on lock 0x7b5400359154. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:58.142995 (+171164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1130401024}
W20260709 13:18:58.210234 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 343681us
I20260709 13:18:58.299307 15956 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.299966 15956 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:58.300748 15860 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.148401 15904 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.301851 15904 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:58.302796 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:58.303356 12709 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:58.304464 15904 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:58.305756 13384 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:18:58.146519 15909 raft_consensus.cc:3055] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:18:58.213446 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 70606.7us
I20260709 13:18:58.195497 15797 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:58.308081 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:58.045624 15814 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:58.298005 12776 rpcz_store.cc:276] 0709 13:18:57.096445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.177997 (+ 81552us) service_pool.cc:224] Handling call
0709 13:18:57.971263 (+793266us) spinlock_profiling.cc:243] Waited 3.57 ms on lock 0x7b5800193e98. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c7238bfc0 00007f4c72376929 00007f4c723763d2 00007f4c6b8edbd4
0709 13:18:58.142282 (+171019us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.199639 (+ 57357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.199854 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8205568}
I20260709 13:18:58.188853 15557 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:58.171172 12781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1846) took 2917 ms. Trace:
W20260709 13:18:58.260850 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.292208 12707 raft_consensus.cc:1240] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:58.328800 12781 rpcz_store.cc:276] 0709 13:18:55.228131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.470872 (+242741us) service_pool.cc:224] Handling call
0709 13:18:57.222441 (+1751569us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.223003 (+   562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.052258 (+829255us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b54001fd950. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:58.145766 (+ 93508us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4027424640}
I20260709 13:18:58.216894 15950 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:58.343655 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:58.290645 15886 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:58.352646 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 66000.8us
I20260709 13:18:58.193149 15841 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:58.294323 12517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 70272.3us
I20260709 13:18:58.214425 12629 rpcz_store.cc:276] 0709 13:18:55.959312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.959431 (+   119us) service_pool.cc:224] Handling call
0709 13:18:55.991407 (+ 31976us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.010322 (+1018915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.969138 (+958816us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540027a454. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:58.146101 (+176963us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2341915392}
I20260709 13:18:58.205549 15465 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:18:58.326766 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 111035us
I20260709 13:18:58.329452 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:58.284633 15961 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:58.301236 15860 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:58.188476 15496 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.354794 12701 raft_consensus.cc:2410] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:18:58.387444 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:58.329044 15814 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:58.311376 15909 raft_consensus.cc:740] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:58.392884 12598 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:58.344800 15451 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:58.394596 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:58.396997 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:58.346717 15950 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:58.397868 12746 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:58.398419 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:58.398710 12746 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:58.399263 13380 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 11: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:58.351289 15862 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:58.399178 13385 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:58.383953 15465 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:58.400586 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:58.415937 12598 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:58.421088 15992 log.cc:927] Time spent T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 13:18:58.422504 15915 log.cc:927] Time spent T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:18:58.425094 15943 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:58.402050 15811 consensus_queue.cc:260] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:58.426381 15989 log.cc:927] Time spent T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.228s	user 0.000s	sys 0.000s
I20260709 13:18:58.427131 15811 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:58.429908 15909 consensus_queue.cc:260] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:58.431885 15956 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:58.353606 15953 consensus_queue.cc:1059] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:18:58.404834 15892 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:58.437955 15892 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:58.442167 15892 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:18:58.447770 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:58.448523 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:58.448859 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.453754 15920 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:58.455304 12630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2304) took 2499 ms. Trace:
I20260709 13:18:58.455686 12630 rpcz_store.cc:276] 0709 13:18:55.956109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.956248 (+   139us) service_pool.cc:224] Handling call
0709 13:18:55.978481 (+ 22233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.418256 (+1439775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.455052 (+1036796us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54000df554. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:58.455238 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3290359680}
I20260709 13:18:58.458331 15834 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:58.461311 15860 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:58.466756 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:58.470655 15991 log.cc:927] Time spent T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:18:58.333379 15797 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:58.475178 15797 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:18:58.478653 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 62810.3us
I20260709 13:18:58.480127 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:58.482510 12779 raft_consensus.cc:1240] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:58.384130 15961 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:58.488370 12709 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:18:58.416910 15847 raft_consensus.cc:3055] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:58.395999 12747 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:58.496114 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 188104us
I20260709 13:18:58.413599 15900 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:58.501549 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:58.503459 12778 raft_consensus.cc:2410] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:18:58.402251 15942 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.088s
I20260709 13:18:58.383850 15886 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:58.505806 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.482335 12598 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:58.382681 15841 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:58.489737 15949 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:58.490684 15961 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:58.526268 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:58.515309 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:58.521317 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 213309us
W20260709 13:18:58.519856 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.487547 13378 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:18:58.400995 15496 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:58.529608 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.529654 12672 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:58.534157 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 138252us
I20260709 13:18:58.535768 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:58.499886 15847 raft_consensus.cc:740] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:58.503963 15900 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:58.504539 15834 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:58.600031 12597 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:18:58.601533 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 119303us
I20260709 13:18:58.602535 12598 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:58.504801 15962 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:18:58.631708 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 96045.3us
I20260709 13:18:58.525544 15949 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:58.520274 15885 consensus_queue.cc:1059] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.096s
W20260709 13:18:58.632431 15988 log.cc:927] Time spent T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:18:58.664566 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:58.663228 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:58.603696 15900 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:58.524981 15814 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:58.599401 15931 consensus_queue.cc:1059] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 13:18:58.660928 15949 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:58.721956 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.739847 12632 raft_consensus.cc:1240] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:58.605078 15921 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.088s
W20260709 13:18:58.779289 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 116153us
I20260709 13:18:58.780432 12747 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:58.716632 15944 log.cc:927] Time spent T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 1.243s	user 0.011s	sys 0.000s
I20260709 13:18:58.631933 15834 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:58.712662 15943 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:58.689806 15956 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.799098 12775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2177) took 1343 ms. Trace:
I20260709 13:18:58.698789 15920 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:58.818650 12775 rpcz_store.cc:276] 0709 13:18:57.455730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.456593 (+   863us) service_pool.cc:224] Handling call
0709 13:18:57.495974 (+ 39381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:57.531805 (+ 35831us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:57.531837 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:57.532342 (+   505us) log.cc:844] Serialized 26 byte log entry
0709 13:18:57.930275 (+397933us) spinlock_profiling.cc:243] Waited 922 us on lock 0x7b18001d4ee0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf520d1 00007f4c6d36da0f
0709 13:18:58.312057 (+381782us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:58.701340 (+389283us) log.cc:844] Serialized 14 byte log entry
0709 13:18:58.781945 (+ 80605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.782049 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:58.782126 (+    77us) raft_consensus.cc:1685] finished
0709 13:18:58.798548 (+ 16422us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.798738 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39634,"raft.queue_time_us":113797,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":20448,"spinlock_wait_cycles":2120448,"thread_start_us":739651,"threads_started":2,"wal-append.queue_time_us":909265}
I20260709 13:18:58.699347 15370 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:58.783928 15962 raft_consensus.cc:2764] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 13:18:58.832909 12702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.010s	sys 0.028s
W20260709 13:18:58.836314 12702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.010s	sys 0.028s
W20260709 13:18:58.845901 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 65458.1us
I20260709 13:18:58.847023 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:58.847214 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 370385us
W20260709 13:18:58.825706 15993 log.cc:927] Time spent T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:18:58.799497 15998 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:58.807420 16002 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:58.867769 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:58.799873 16001 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:18:58.868835 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:58.870200 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.869879 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:58.883723 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:58.860677 15944 log.cc:927] Time spent T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:18:58.740335 15847 consensus_queue.cc:260] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:58.870167 15867 consensus_meta.cc:287] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.657s	user 0.008s	sys 0.002s
I20260709 13:18:58.890843 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:58.865644 15437 raft_consensus.cc:3055] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:58.868942 15998 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:58.880992 15458 consensus_meta.cc:287] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.584s	user 0.010s	sys 0.000s
I20260709 13:18:58.864729 15981 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:58.896667 15847 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:58.824039 15956 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:58.843564 15370 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:58.890422 15962 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:58.931334 15886 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:58.898588 15867 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:58.931357 15370 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:58.934820 12774 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:18:58.902602 15998 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:58.932885 15956 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:58.933437 15834 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:58.897255 15898 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:58.894783 15437 raft_consensus.cc:740] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:58.930089 12775 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:18:58.936772 15878 consensus_meta.cc:287] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.726s	user 0.010s	sys 0.000s
W20260709 13:18:58.955655 15996 log.cc:927] Time spent T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.481s	user 0.001s	sys 0.001s
I20260709 13:18:58.892686 12747 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:58.968859 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 368859us
I20260709 13:18:58.975682 13378 raft_consensus.cc:1240] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: []
W20260709 13:18:58.964248 15451 consensus_meta.cc:287] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.555s	user 0.009s	sys 0.002s
I20260709 13:18:58.915192 15458 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:58.997995 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 395527us
I20260709 13:18:59.000270 12598 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:59.004947 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:59.005865 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.974238 15886 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.049844 15886 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.052428 15886 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:58.982344 16018 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:59.059157 16018 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:59.015504 16009 log.cc:927] Time spent T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:18:58.998361 15814 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:59.062589 16019 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
W20260709 13:18:59.015930 15911 consensus_meta.cc:287] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 1.052s	user 0.003s	sys 0.007s
I20260709 13:18:59.002995 15898 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:59.003223 15841 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:59.019518 15950 consensus_meta.cc:287] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.606s	user 0.009s	sys 0.000s
W20260709 13:18:59.020897 15465 consensus_meta.cc:287] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.545s	user 0.010s	sys 0.001s
I20260709 13:18:59.030582 15961 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:59.035262 15994 log.cc:927] Time spent T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.495s	user 0.001s	sys 0.000s
I20260709 13:18:59.058457 15921 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:59.058499 12625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2323) took 1880 ms. Trace:
I20260709 13:18:59.010697 15437 consensus_queue.cc:260] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.006796 12597 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:18:59.008972 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 116356us
W20260709 13:18:59.150866 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.151023 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:59.068089 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 67925.1us
I20260709 13:18:59.074396 16018 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:59.082157 15496 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:59.082815 15465 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:59.085503 15892 consensus_meta.cc:287] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.643s	user 0.007s	sys 0.004s
I20260709 13:18:59.087688 15458 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:59.088289 15953 raft_consensus.cc:3055] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:59.090356 15900 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:59.046830 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.110273 15898 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:59.126098 15956 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.124718 12625 rpcz_store.cc:276] 0709 13:18:57.177771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.308951 (+131180us) service_pool.cc:224] Handling call
0709 13:18:57.309413 (+   462us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:57.724767 (+415354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:57.724807 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:57.726058 (+  1251us) log.cc:844] Serialized 52 byte log entry
0709 13:18:58.049152 (+323094us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b180046d080. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf520d1 00007f4c6d36da0f
0709 13:18:58.350184 (+301032us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:58.350389 (+   205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.350493 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:58.940922 (+590429us) spinlock_profiling.cc:243] Waited 9.44 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:18:59.058253 (+117331us) raft_consensus.cc:1685] finished
0709 13:18:59.058277 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:59.058438 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":413419,"raft.queue_time_us":415792,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":4402,"spinlock_wait_cycles":27314432,"thread_start_us":300782,"threads_started":1,"wal-append.queue_time_us":678816}
I20260709 13:18:59.124853 15554 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:59.170055 15554 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:59.171542 15822 consensus_queue.cc:1059] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:59.172662 15822 consensus_queue.cc:1243] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:18:59.173156 15465 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:59.124558 15720 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:59.139935 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.174187 12634 raft_consensus.cc:1240] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 7. Current term is 8. Ops: []
W20260709 13:18:59.174463 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.141955 15945 raft_consensus.cc:3055] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:59.152340 12598 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:59.152609 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 145909us
I20260709 13:18:59.153798 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:59.072945 15451 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.073462 15911 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:59.162777 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:59.092656 15950 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:59.159287 15953 raft_consensus.cc:740] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:59.161929 15892 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:59.177206 12631 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:18:59.153995 15942 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.155871 15814 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:59.186623 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:59.175886 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:59.158555 16001 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:59.182085 12598 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:59.173918 15860 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.004s	sys 0.021s
I20260709 13:18:59.186928 13384 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
W20260709 13:18:59.184242 15797 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.009s	sys 0.008s
I20260709 13:18:59.173659 16006 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.210589 13382 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:18:59.214677 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:59.217893 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:59.204231 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:59.204900 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:59.174261 15898 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:59.208803 12747 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:18:59.208998 12633 raft_consensus.cc:1240] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:59.173152 15554 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:59.227404 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:59.186491 15496 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:18:59.209860 15797 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.009s	sys 0.008s
I20260709 13:18:59.228596 15592 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:59.226646 16006 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:59.229547 16010 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.207473 15942 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.229277 16020 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:18:59.215221 13382 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 10. Current term is 11. Ops: []
I20260709 13:18:59.232578 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:59.233286 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:59.201086 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 315803us
I20260709 13:18:59.211771 15465 consensus_queue.cc:1059] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:18:59.176538 15956 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:59.196067 15814 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.298599 15814 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:59.300071 15956 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:59.247258 15953 consensus_queue.cc:260] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:59.301426 15953 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:59.303117 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:59.274310 15949 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.306528 15814 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.308099 12626 raft_consensus.cc:2393] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 9: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:59.308151 15949 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:59.309083 15961 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:59.275105 15981 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:59.278107 16010 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.313748 16010 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:59.238896 12630 raft_consensus.cc:2393] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:18:59.314255 15931 raft_consensus.cc:3055] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:59.314687 15931 raft_consensus.cc:740] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:59.313751 15844 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:59.314927 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:59.315582 15931 consensus_queue.cc:260] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.316288 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:59.316970 12630 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:18:59.317076 15931 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:59.314952 12631 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:59.186064 15878 consensus_queue.cc:1059] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:18:59.184039 15451 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.185952 16018 consensus_queue.cc:1048] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:18:59.250572 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:18:59.181386 15998 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:59.279184 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.176440 15945 raft_consensus.cc:740] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:59.281760 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:59.285082 12531 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54800 (request call id 229) took 26865 ms (client timeout 15000 ms). Trace:
I20260709 13:18:59.284860 15456 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:18:59.288738 12528 catalog_manager.cc:5697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 1 to 7. New cstate: current_term: 7 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:59.291083 15942 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:59.182760 15720 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.314970 12777 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 9. Current term is 10. Ops: [6.6-9.8]
I20260709 13:18:59.320284 12673 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:18:59.324469 12672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.004s	sys 0.027s
I20260709 13:18:59.328166 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:18:59.316401 16024 log.cc:927] Time spent T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:18:59.273268 15496 raft_consensus.cc:697] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:59.287326 15900 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:59.284606 15458 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:18:59.340890 12672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.004s	sys 0.027s
I20260709 13:18:59.333635 15966 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.333986 12528 catalog_manager.cc:5697] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 2 to 5, term changed from 3 to 7, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 7 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:59.344368 12627 raft_consensus.cc:2368] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8.
I20260709 13:18:59.341853 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:59.352492 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 822882us
I20260709 13:18:59.358768 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:59.361584 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:59.362288 12778 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
W20260709 13:18:59.377907 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 57702us
I20260709 13:18:59.361652 12528 catalog_manager.cc:5697] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:59.383091 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:59.385624 12776 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.889s	user 0.009s	sys 0.011s
W20260709 13:18:59.390020 12776 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.893s	user 0.009s	sys 0.011s
I20260709 13:18:59.392421 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:59.352730 16006 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:59.340314 15456 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:59.395933 12778 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 11: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:59.360011 15814 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:59.402631 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:59.378690 15900 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.406190 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:59.406955 12778 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:18:59.404660 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:59.405294 12779 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
I20260709 13:18:59.409689 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:59.410612 12779 raft_consensus.cc:2393] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 11: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:59.384531 15886 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:18:59.331060 12531 rpcz_store.cc:269] 0709 13:18:32.382227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:32.382864 (+   637us) service_pool.cc:224] Handling call
0709 13:18:59.247852 (+26864988us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:54.697948 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:54.698584 (+   636us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:54.698635 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:18:54.698677 (+    42us) tablet.cc:662] Decoding operations
0709 13:18:54.856016 (+157339us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:54.871754 (+ 15738us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:54.871795 (+    41us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:18:54.907239 (+ 35444us) tablet.cc:701] Row locks acquired
0709 13:18:54.907295 (+    56us) write_op.cc:260] PREPARE: finished
0709 13:18:54.907496 (+   201us) write_op.cc:270] Start()
0709 13:18:54.907647 (+   151us) write_op.cc:276] Timestamp: P: 1783603134907428 usec, L: 0
0709 13:18:54.907670 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:18:54.908453 (+   783us) log.cc:844] Serialized 5962 byte log entry
0709 13:18:55.881904 (+973451us) op_driver.cc:464] REPLICATION: finished
0709 13:18:56.501308 (+619404us) spinlock_profiling.cc:243] Waited 3.7 ms on lock 0x7b180013d740. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c53f3 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:18:56.501484 (+   176us) write_op.cc:301] APPLY: starting
0709 13:18:56.501588 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:56.501906 (+   318us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:56.501935 (+    29us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:56.834434 (+332499us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:56.999909 (+165475us) spinlock_profiling.cc:243] Waited 3.4 ms on lock 0x7b180013d740. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c5557 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:18:57.072036 (+ 72127us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=15
0709 13:18:57.072098 (+    62us) write_op.cc:312] APPLY: finished
0709 13:18:57.072576 (+   478us) log.cc:844] Serialized 107 byte log entry
0709 13:18:57.199181 (+126605us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:57.200924 (+  1743us) write_op.cc:454] Released schema lock
0709 13:18:57.201008 (+    84us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:57.206761 (+  5753us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:57.206805 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":8952,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":160673,"apply.run_cpu_time_us":26800,"apply.run_wall_time_us":705148,"num_ops":15,"prepare.queue_time_us":156367,"prepare.run_cpu_time_us":30222,"prepare.run_wall_time_us":550671,"raft.queue_time_us":383480,"raft.run_cpu_time_us":16295,"raft.run_wall_time_us":158568,"replication_time_us":974028,"spinlock_wait_cycles":16328320,"thread_start_us":1113901,"threads_started":5,"wal-append.queue_time_us":648154,"wal-append.run_cpu_time_us":43607,"wal-append.run_wall_time_us":2680874}]]}
I20260709 13:18:59.417268 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:59.418495 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:59.421200 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:59.421921 12777 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:59.422449 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.419463 12781 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:18:59.413972 12528 catalog_manager.cc:5697] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 5 to 8, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 8 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:59.427945 12781 raft_consensus.cc:1240] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:59.418238 15900 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:59.433393 12528 catalog_manager.cc:5697] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 3 to 8. New cstate: current_term: 8 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:59.363734 15966 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.362970 15458 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:59.446658 12778 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 11: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:59.413187 15950 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:59.419673 15886 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.428534 15892 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:59.471791 12528 catalog_manager.cc:5697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:59.430720 16001 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:59.504812 16001 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:59.506902 15451 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:59.507515 16001 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:59.477445 15814 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.478350 12528 catalog_manager.cc:5697] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 3 to 6, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 6 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:59.480338 12775 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:18:59.519797 12775 pending_rounds.cc:85] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 7
I20260709 13:18:59.520138 12775 pending_rounds.cc:107] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:18:59.520426 12775 raft_consensus.cc:2889] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:59.521852 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:59.522655 12775 raft_consensus.cc:2393] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 6: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:59.482465 16032 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.085s
I20260709 13:18:59.525512 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:59.482712 15720 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:59.483084 16000 log.cc:927] Time spent T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.544s	user 0.005s	sys 0.000s
I20260709 13:18:59.428411 15496 consensus_queue.cc:237] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:59.485037 16031 log.cc:927] Time spent T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:18:59.485685 16027 log.cc:927] Time spent T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
W20260709 13:18:59.485996 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:59.486545 16029 log.cc:927] Time spent T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 13:18:59.486231 13349 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:59.547019 12777 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
W20260709 13:18:59.547363 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 61240.6us
I20260709 13:18:59.503048 15998 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:59.512909 12673 leader_election.cc:400] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8.
I20260709 13:18:59.548193 12673 leader_election.cc:403] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:18:59.401428 15945 consensus_queue.cc:260] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:59.459802 15886 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:59.531235 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:59.540714 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:59.549094 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:59.531260 12528 catalog_manager.cc:5697] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 4 to 6, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 6 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:59.543888 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:59.549824 12633 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:59.550375 12775 raft_consensus.cc:2410] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:18:59.548095 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:59.519860 16026 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.518241 15451 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.553854 12781 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 11: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:59.555621 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:59.556391 13385 raft_consensus.cc:2393] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 7: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
W20260709 13:18:59.556893 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 45318.3us
W20260709 13:18:59.561177 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.542184 15898 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:59.609108 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:59.610086 12706 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:59.611248 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:59.611512 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:59.611999 12700 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:59.565412 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:59.554795 16032 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:59.567853 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:59.615593 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:59.616071 12775 raft_consensus.cc:2376] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8. Re-sending same reply.
I20260709 13:18:59.616497 12701 raft_consensus.cc:2393] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:18:59.617033 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:59.617229 15554 raft_consensus.cc:3055] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:59.617671 12775 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:18:59.617660 15554 raft_consensus.cc:740] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:18:59.618778 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:59.618844 15554 consensus_queue.cc:260] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.619905 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:59.620141 15554 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:59.621023 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:59.568070 12528 catalog_manager.cc:5697] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:18:59.621865 15996 log.cc:927] Time spent T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:18:59.621933 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 379) took 1321 ms. Trace:
I20260709 13:18:59.553508 16030 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:59.569141 12782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2185) took 1084 ms. Trace:
I20260709 13:18:59.623245 12782 rpcz_store.cc:276] 0709 13:18:58.484781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.490347 (+  5566us) service_pool.cc:224] Handling call
0709 13:18:58.494076 (+  3729us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:58.498529 (+  4453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:58.498573 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:58.505653 (+  7080us) log.cc:844] Serialized 26 byte log entry
0709 13:18:59.110203 (+604550us) spinlock_profiling.cc:243] Waited 1.68 ms on lock 0x7b18002f5a20. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf520d1 00007f4c6d36da0f
0709 13:18:59.343764 (+233561us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:59.343809 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.343868 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:59.568833 (+224965us) raft_consensus.cc:1685] finished
0709 13:18:59.568851 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:59.569082 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4402,"raft.queue_time_us":8006,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":18136,"spinlock_wait_cycles":3857152,"thread_start_us":233372,"threads_started":1,"wal-append.queue_time_us":894507}
I20260709 13:18:59.624701 15945 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:59.569823 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:59.570390 16038 log.cc:927] Time spent T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:18:59.532272 15814 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:18:59.571465 16034 log.cc:927] Time spent T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:18:59.571561 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:18:59.668640 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 97185.7us
I20260709 13:18:59.571774 15906 raft_consensus.cc:3055] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:59.573830 16037 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:59.574450 15878 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:59.574903 15860 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:18:59.575344 16010 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.575390 15797 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.588371 15966 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:59.561417 15451 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:59.593333 15911 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:18:59.573859 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.595552 15456 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:59.597182 12783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1871) took 1630 ms. Trace:
I20260709 13:18:59.606388 12705 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:59.606648 12708 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:59.607460 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:59.693254 12783 rpcz_store.cc:276] 0709 13:18:57.967188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.967295 (+   107us) service_pool.cc:224] Handling call
0709 13:18:57.967564 (+   269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:58.148655 (+181091us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:58.148685 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:58.149160 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 13:18:58.392600 (+243440us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:58.392636 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.392704 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:58.605251 (+212547us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b4800042258. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c722dbaa1 00007f4c722d2b37 00007f4c722d29ca 00007f4c723ca26f 00007f4c723c9f37 00007f4c723de55c 00007f4c723de508 00007f4c723de488 00007f4c723de443 00007f4c723dd6ec 00007f4c71307a96
0709 13:18:59.199913 (+594662us) spinlock_profiling.cc:243] Waited 9.24 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:18:59.596869 (+396956us) raft_consensus.cc:1685] finished
0709 13:18:59.596911 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:59.597099 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":178811,"raft.queue_time_us":181504,"raft.run_cpu_time_us":1553,"raft.run_wall_time_us":480719,"spinlock_wait_cycles":72817280,"thread_start_us":108500,"threads_started":1,"wal-append.queue_time_us":453784}
I20260709 13:18:59.696201 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:59.611176 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:59.614929 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:59.698875 12707 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 11: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:59.615545 13385 raft_consensus.cc:2410] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:18:59.625284 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:59.629948 12781 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:18:59.633217 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:59.701730 12774 raft_consensus.cc:2410] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:18:59.623117 16042 log.cc:927] Time spent T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.128s	user 0.001s	sys 0.001s
I20260709 13:18:59.647720 13348 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:59.652041 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:59.705703 12701 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:18:59.653390 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:59.654562 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:59.570554 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:59.707737 12777 raft_consensus.cc:2393] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 8: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:59.662441 12528 catalog_manager.cc:5697] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:59.622385 12702 rpcz_store.cc:276] 0709 13:18:58.300814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.304987 (+  4173us) service_pool.cc:224] Handling call
0709 13:18:58.305302 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:58.838190 (+532888us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:58.838239 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:58.838838 (+   599us) log.cc:844] Serialized 26 byte log entry
0709 13:18:59.061492 (+222654us) spinlock_profiling.cc:243] Waited 3.73 ms on lock 0x7b48001b0258. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c722dbaa1 00007f4c722d2b37 00007f4c722d29ca 00007f4c723ca26f 00007f4c723c9f37 00007f4c723de55c 00007f4c723de508 00007f4c723de488 00007f4c723de443 00007f4c723dd6ec 00007f4c71307a96
0709 13:18:59.285914 (+224422us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:59.286144 (+   230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.286665 (+   521us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:59.621617 (+334952us) raft_consensus.cc:1685] finished
0709 13:18:59.621637 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:59.621854 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2014,"raft.queue_time_us":28005,"raft.run_cpu_time_us":3055,"raft.run_wall_time_us":720679,"spinlock_wait_cycles":8579456,"thread_start_us":913877,"threads_started":2,"wal-append.queue_time_us":488809}
I20260709 13:18:59.673722 12672 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:59.581288 16023 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.557789 16026 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.703905 12597 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:18:59.676209 16003 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:59.711035 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:59.713117 12700 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:18:59.719161 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:59.719563 12781 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:59.719831 12700 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 11: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:59.675055 15906 raft_consensus.cc:740] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:59.697067 12783 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:18:59.714620 12672 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:18:59.720810 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.721714 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:18:59.721248 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 73654us
I20260709 13:18:59.728185 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:59.728579 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:59.729821 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:59.687371 15860 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.729342 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:59.734660 12777 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:59.684700 16032 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8.
I20260709 13:18:59.738586 12528 catalog_manager.cc:5697] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 5 to 6, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 6 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:59.733150 15860 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:59.740916 12783 raft_consensus.cc:3055] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:59.741282 12783 raft_consensus.cc:740] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:18:59.739873 12672 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:59.742762 12672 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:59.744407 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:59.745929 12781 raft_consensus.cc:1275] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:18:59.746001 13348 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:18:59.747071 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:59.752233 12780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1873) took 1784 ms. Trace:
I20260709 13:18:59.716449 15942 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:59.749902 15456 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:59.761169 15456 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.761605 15456 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:59.771965 12634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 471) took 1243 ms. Trace:
I20260709 13:18:59.772588 12634 rpcz_store.cc:276] 0709 13:18:58.528809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.528910 (+   101us) service_pool.cc:224] Handling call
0709 13:18:58.529168 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.177909 (+648741us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.771596 (+593687us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b54000dd754. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:59.771871 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1482313600}
I20260709 13:18:59.773701 15557 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:18:59.775163 16028 log.cc:927] Time spent T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.273s	user 0.002s	sys 0.004s
I20260709 13:18:59.777271 15999 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:59.777887 15999 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.778348 15999 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:59.681090 16037 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.783802 16037 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:59.786458 16026 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:59.787263 15860 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:59.788393 13384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51768 (request call id 551) took 1408 ms. Trace:
I20260709 13:18:59.788944 15451 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:59.735718 15910 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:59.708938 15797 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.793931 15797 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:59.690702 16010 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:59.796290 16010 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:59.797997 15906 consensus_queue.cc:260] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.799960 15906 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:59.799787 15592 consensus_queue.cc:1059] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:18:59.801203 15592 raft_consensus.cc:3055] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:18:59.721213 16023 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:59.801735 15592 raft_consensus.cc:740] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:18:59.803519 16023 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:59.803027 15592 consensus_queue.cc:260] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:59.804157 12704 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:59.804790 15592 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:59.807075 15886 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:59.807821 12706 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:18:59.809113 15886 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:59.752795 12780 rpcz_store.cc:276] 0709 13:18:57.967825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.967946 (+   121us) service_pool.cc:224] Handling call
0709 13:18:58.145864 (+177918us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:58.195646 (+ 49782us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:58.195701 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:59.064118 (+868417us) spinlock_profiling.cc:243] Waited 7.57 ms on lock 0x7b180041d7c0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2932a6 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6bcaba11 00007f4c6bcbb372 00007f4c6d3eda01 00007f4c6d3b243a
0709 13:18:59.064576 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 13:18:59.386201 (+321625us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:59.386257 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.490724 (+104467us) spinlock_profiling.cc:243] Waited 47.8 ms on lock 0x7b54003700d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:18:59.490779 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:59.751951 (+261172us) raft_consensus.cc:1685] finished
0709 13:18:59.751986 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:59.752163 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4120,"raft.queue_time_us":2227,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":9040,"spinlock_wait_cycles":129226880,"thread_start_us":309873,"threads_started":1,"wal-append.queue_time_us":333542}
I20260709 13:18:59.755918 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:59.756458 12528 catalog_manager.cc:5697] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 1 to 5, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 5 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:59.797662 16032 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:59.811033 12528 catalog_manager.cc:5697] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:18:59.811842 16032 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.812268 16032 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:59.683648 15878 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:59.812690 12528 catalog_manager.cc:5697] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 5 to 8, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 8 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:59.814604 12528 catalog_manager.cc:5697] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:59.815713 15878 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:59.816196 12528 catalog_manager.cc:5697] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 1 to 7, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 7 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:59.817044 15942 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.818120 12528 catalog_manager.cc:5697] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:59.819602 15942 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:18:59.820847 15931 consensus_meta.cc:287] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.503s	user 0.008s	sys 0.003s
I20260709 13:18:59.788944 13384 rpcz_store.cc:276] 0709 13:18:58.379667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.387104 (+  7437us) service_pool.cc:224] Handling call
0709 13:18:59.788129 (+1401025us) spinlock_profiling.cc:243] Waited 799 ms on lock 0x7b5400486054. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:18:59.788305 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1836758784}
I20260709 13:18:59.753897 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:59.828078 15458 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:59.832283 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:59.843052 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:59.844014 12700 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:18:59.740063 15822 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:59.848750 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:59.850543 12598 connection.cc:256] RPC call timeout handler was delayed by -2.1441793300000427s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
I20260709 13:18:59.853497 12783 consensus_queue.cc:260] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.855576 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:18:59.750725 15961 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:59.717435 15911 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:18:59.860858 12598 leader_election.cc:336] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.546s (SENT)
I20260709 13:18:59.861641 15557 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:59.863328 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:18:59.863605 15557 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:18:59.864277 12700 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:59.866250 12783 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:59.856201 13380 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:18:59.862536 12598 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:59.869957 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:18:59.843727 15900 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:18:59.871551 13384 raft_consensus.cc:2393] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 7: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:18:59.878306 16032 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.882604 16032 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:18:59.898146 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:18:59.898819 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.899178 12701 raft_consensus.cc:2410] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:18:59.900538 15437 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:18:59.901445 15953 consensus_meta.cc:287] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.600s	user 0.004s	sys 0.005s
I20260709 13:18:59.851773 15931 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:59.902307 15931 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.873176 15949 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:59.907131 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:18:59.907711 12705 raft_consensus.cc:1813] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Handling vote request from an unknown peer 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:18:59.908238 12705 raft_consensus.cc:2410] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:18:59.913918 16001 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:59.914155 12747 connection.cc:256] RPC call timeout handler was delayed by -1.9395352120000098s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
I20260709 13:18:59.914485 16001 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.915072 12700 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:18:59.915356 15834 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.915804 15834 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.916841 16001 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:59.918998 15962 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:18:59.919636 15962 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.921904 15847 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:18:59.922246 12780 raft_consensus.cc:1275] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:18:59.922473 15847 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.923943 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:18:59.924662 12780 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:18:59.925072 15847 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:18:59.857167 15458 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:18:59.931972 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:59.949960 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 89143.9us
I20260709 13:18:59.950758 12629 raft_consensus.cc:1275] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:59.932411 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:18:59.914822 12747 connection.cc:256] RPC call timeout handler was delayed by -1.9395352120000098s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
W20260709 13:18:59.952720 12747 connection.cc:256] RPC call timeout handler was delayed by -1.9089352119999603s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1785s for a response
W20260709 13:18:59.953194 12747 connection.cc:256] RPC call timeout handler was delayed by -1.8486352120000902s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18520000000000003s for a response
W20260709 13:18:59.953507 12747 connection.cc:256] RPC call timeout handler was delayed by -1.812635212000032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1892s for a response
W20260709 13:18:59.953918 12747 connection.cc:256] RPC call timeout handler was delayed by -1.812635212000032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1892s for a response
W20260709 13:18:59.954279 12747 connection.cc:256] RPC call timeout handler was delayed by -1.8009352120000131s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1905s for a response
W20260709 13:18:59.954608 12747 connection.cc:256] RPC call timeout handler was delayed by -1.8009352120000131s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1905s for a response
W20260709 13:18:59.955003 12747 connection.cc:256] RPC call timeout handler was delayed by -1.6551352120000047s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20670000000000002s for a response
W20260709 13:18:59.955351 12747 connection.cc:256] RPC call timeout handler was delayed by -1.1394352120000804s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.264s for a response
W20260709 13:18:59.955695 12747 connection.cc:256] RPC call timeout handler was delayed by -1.1394352120000804s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.264s for a response
I20260709 13:18:59.956490 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:59.958290 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:59.958793 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.935925 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:18:59.944258 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.968386 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:18:59.969164 12780 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:59.970413 15949 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.970518 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:59.971061 15949 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:18:59.971401 12632 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:18:59.974637 15900 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:59.976683 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:59.984490 12747 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:18:59.992695 12747 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
W20260709 13:18:59.994148 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 251467us
I20260709 13:18:59.996953 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:18:59.989153 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:00.001878 13384 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
W20260709 13:18:59.986027 16036 log.cc:927] Time spent T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.478s	user 0.000s	sys 0.003s
I20260709 13:18:59.860517 16019 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.103s
I20260709 13:19:00.005368 12672 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:00.008914 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.019762 12780 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:00.022171 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:00.024679 12629 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:18:59.993906 15892 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:19:00.027990 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 282090us
I20260709 13:19:00.034873 13348 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:00.036300 12597 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:00.009114 12672 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:00.025904 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:00.037578 12634 raft_consensus.cc:2393] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:18:59.966094 15847 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:00.018471 15921 consensus_meta.cc:287] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.865s	user 0.006s	sys 0.007s
I20260709 13:19:00.043826 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:00.045077 12597 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:00.046509 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:00.046830 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:00.047000 12783 raft_consensus.cc:1275] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:19:00.047971 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:00.037480 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:00.000996 15949 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:00.052346 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:00.031064 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:00.048142 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:00.053375 12780 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:00.054889 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:00.055263 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:00.055691 12632 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:00.056071 12780 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:00.024267 15886 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:59.988070 15458 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:00.066582 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 57556.7us
I20260709 13:19:00.033931 15962 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:00.053786 12783 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:18:59.999588 15919 consensus_queue.cc:1059] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:19:00.058045 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:00.067613 12672 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:00.071312 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:00.072081 12780 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:19:00.007190 16030 consensus_queue.cc:1059] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:19:00.042202 15814 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:00.046248 15812 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:00.078898 12672 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:00.051146 15921 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:00.064949 15886 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.075678 16006 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:00.051306 15847 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:00.045521 15998 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:19:00.067376 15892 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:00.110327 12672 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:00.114820 16063 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:19:00.115022 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 93310.6us
I20260709 13:19:00.082504 15962 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.125032 12747 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:00.083308 16055 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:00.085929 12776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2180) took 1609 ms. Trace:
I20260709 13:19:00.068974 15898 consensus_queue.cc:1059] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:19:00.087782 12625 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:19:00.067194 15456 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:00.126353 12747 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:00.066004 15945 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:00.082451 15812 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.133844 12776 rpcz_store.cc:276] 0709 13:18:58.476485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.477264 (+   779us) service_pool.cc:224] Handling call
0709 13:18:58.477553 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.477829 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:59.390563 (+912734us) log.cc:844] Serialized 14 byte log entry
0709 13:18:59.474069 (+ 83506us) log.cc:844] Serialized 14 byte log entry
0709 13:18:59.484829 (+ 10760us) spinlock_profiling.cc:243] Waited 5.85 ms on lock 0x7b18002f5780. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c53f3 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:18:59.911337 (+426508us) spinlock_profiling.cc:243] Waited 5.52 ms on lock 0x7b18002f5780. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a23fc0a 00007f4c7397f219 00007f4c6a2c7a40 00007f4c6a2c24e4 00007f4c6a2c2455 00007f4c6d3aa060 00007f4c6d3bc716 00007f4c6d3cd287 00007f4c6d3cd208
0709 13:19:00.084161 (+172824us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:00.084232 (+    71us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:00.084284 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:00.084378 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:00.084533 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":431782,"prepare.queue_time_us":498371,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":90,"raft.queue_time_us":442146,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":175,"spinlock_wait_cycles":26158464,"thread_start_us":1147967,"threads_started":3,"wal-append.queue_time_us":181525}
I20260709 13:19:00.068516 15953 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:00.081786 16018 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:00.144305 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:00.145850 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:00.128799 15962 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:00.084800 15720 consensus_queue.cc:1048] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:19:00.136266 15554 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.123914 15886 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:00.133388 12747 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:00.132891 16006 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:00.106204 16054 consensus_queue.cc:1048] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 13:19:00.164016 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:00.116608 16057 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:00.127985 16055 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:00.136618 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 101865us
I20260709 13:19:00.170794 13348 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:00.148435 12628 raft_consensus.cc:2393] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 7: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:00.138235 15841 raft_consensus.cc:3055] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:00.084800 15900 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:00.139873 15812 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:00.156088 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 106894us
I20260709 13:19:00.141532 16056 raft_consensus.cc:3055] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:00.157469 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:00.158318 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:00.187645 12776 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:00.143821 15980 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:00.167586 12747 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:00.172071 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.166213 16006 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.172124 12672 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:00.182083 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:00.185765 13349 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:00.183146 13348 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:00.158566 15892 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:00.151746 16018 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:00.157088 15847 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:00.155897 15953 raft_consensus.cc:697] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:00.147612 15911 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:00.180243 15961 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:00.169665 16057 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.196836 12672 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:00.209046 12672 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:00.195042 12597 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:00.160900 15554 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:00.211143 16063 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:00.212700 15554 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:00.181710 15841 raft_consensus.cc:740] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:00.213949 15841 consensus_queue.cc:260] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:00.215368 15841 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:00.217785 15458 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:00.222785 15919 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:00.223253 15919 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:00.206231 12776 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:00.223606 15919 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:00.224805 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:00.206661 16023 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:00.225543 12776 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:19:00.166599 16064 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:00.182384 16055 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:00.227262 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:00.236312 12776 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:00.229717 12702 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:00.187425 16056 raft_consensus.cc:740] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:00.190204 16030 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:00.197741 16006 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:00.236011 12597 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:00.241150 12709 raft_consensus.cc:1275] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:19:00.241856 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:00.263361 12597 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:00.243011 16063 raft_consensus.cc:697] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:00.272688 15998 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:00.273114 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:00.273478 16019 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:00.272971 16063 consensus_queue.cc:237] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:00.273880 16019 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.274318 16019 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:00.245818 15961 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:00.197249 15980 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:00.248452 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:00.287225 12783 raft_consensus.cc:2393] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 8: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:00.288240 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:00.289050 12783 raft_consensus.cc:2410] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:19:00.253952 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:00.241916 12708 raft_consensus.cc:1275] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:19:00.290812 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:00.254791 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:00.292335 12776 raft_consensus.cc:2393] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:00.234095 16010 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:00.293535 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:00.273883 12705 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
W20260709 13:19:00.302548 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 135031us
I20260709 13:19:00.302953 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:00.304543 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:00.305285 12705 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:19:00.306654 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:00.282804 12672 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:00.187695 16067 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:19:00.272701 16062 log.cc:927] Time spent T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:19:00.273192 15998 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:00.286386 15847 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:00.234407 15892 raft_consensus.cc:697] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:00.233528 16057 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:00.253304 15942 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:19:00.300417 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 46580.1us
I20260709 13:19:00.235180 16064 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:00.286561 15961 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.336606 15961 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:00.312356 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 106645us
I20260709 13:19:00.317318 16054 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:00.311447 15910 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:00.330889 15942 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.335386 16064 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:00.335978 16025 log.cc:927] Time spent T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.951s	user 0.002s	sys 0.000s
I20260709 13:19:00.315850 12672 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:00.340126 12776 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:00.323419 15998 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:00.343953 12782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 391) took 1065 ms. Trace:
I20260709 13:19:00.346204 15942 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:00.356616 12782 rpcz_store.cc:276] 0709 13:18:59.279026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.714839 (+435813us) service_pool.cc:224] Handling call
0709 13:18:59.729592 (+ 14753us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:59.917789 (+188197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:59.917831 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:59.918365 (+   534us) log.cc:844] Serialized 26 byte log entry
0709 13:19:00.082986 (+164621us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:00.083025 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:00.083087 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:00.343712 (+260625us) raft_consensus.cc:1685] finished
0709 13:19:00.343733 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:00.343878 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":187790,"raft.queue_time_us":191805,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"thread_start_us":164220,"threads_started":1,"wal-append.queue_time_us":228825}
W20260709 13:19:00.358423 16060 log.cc:927] Time spent T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:19:00.360229 12782 pending_rounds.cc:85] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 3
I20260709 13:19:00.360522 12782 pending_rounds.cc:107] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 5.4
W20260709 13:19:00.352981 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 402065us
I20260709 13:19:00.328992 16056 consensus_queue.cc:260] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:00.361606 16023 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:19:00.332288 15592 consensus_meta.cc:287] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.527s	user 0.012s	sys 0.004s
I20260709 13:19:00.363987 12782 raft_consensus.cc:2889] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:00.372004 12705 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
W20260709 13:19:00.364296 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.377480 16075 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:00.378240 15931 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:00.380512 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:00.381358 12630 raft_consensus.cc:2393] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 8: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:00.380447 12625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56832 (request call id 2349) took 1205 ms. Trace:
I20260709 13:19:00.381955 12625 rpcz_store.cc:276] 0709 13:18:59.175289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.175589 (+   300us) service_pool.cc:224] Handling call
0709 13:19:00.380259 (+1204670us) spinlock_profiling.cc:243] Waited 898 ms on lock 0x7b540007a3d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:00.380364 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2064396160}
I20260709 13:19:00.382230 15892 consensus_queue.cc:237] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:00.385941 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:00.386653 12778 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:19:00.390548 15980 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:00.391525 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.392031 12673 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:00.401036 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:00.402024 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:00.402340 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:00.402593 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:00.403118 13385 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:00.403599 13382 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 4.
I20260709 13:19:00.405401 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:00.401804 13384 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:19:00.409909 12783 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: []
W20260709 13:19:00.367964 15906 consensus_meta.cc:287] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.567s	user 0.012s	sys 0.003s
W20260709 13:19:00.410545 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.413131 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:00.346355 16054 raft_consensus.cc:697] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:00.406245 13379 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:19:00.415786 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:00.418434 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:00.419647 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:00.422902 12598 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:00.362689 15847 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:00.318156 15953 consensus_queue.cc:237] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:00.424356 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:00.425408 12673 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:00.428187 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:00.432637 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:00.369393 16023 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.430286 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:00.433211 12673 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:00.431280 12672 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:00.435477 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:00.439896 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:00.443254 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:00.445349 12598 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:00.410679 15980 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:00.447930 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:00.454660 12598 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:00.448385 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:00.455163 12673 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:00.394902 15962 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:00.456071 12672 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:00.411444 16074 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:00.424132 16010 consensus_queue.cc:1048] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 13:19:00.452786 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:00.441658 12632 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:19:00.435884 16069 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:00.454823 12634 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
W20260709 13:19:00.450928 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 112671us
I20260709 13:19:00.449863 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:00.450618 15910 raft_consensus.cc:697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:00.453917 15906 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:00.442317 16023 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:00.459415 15554 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:19:00.488775 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.489080 12747 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:00.440028 15720 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:00.470433 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:00.492044 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:00.492589 13348 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e
W20260709 13:19:00.466245 15999 consensus_meta.cc:287] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.684s	user 0.014s	sys 0.000s
I20260709 13:19:00.462154 15437 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:00.456030 12629 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:19:00.440696 15592 consensus_queue.cc:1048] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:19:00.509722 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:00.500962 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:00.507476 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:00.501628 16073 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:00.461906 15952 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:00.466217 15965 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:00.470842 15921 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:00.453591 15496 consensus_queue.cc:1048] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:19:00.435729 15847 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:00.505422 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:00.505664 12630 raft_consensus.cc:2393] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 10: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:00.508894 15910 consensus_queue.cc:237] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:00.548244 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 55747us
I20260709 13:19:00.549105 13348 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:00.502920 15720 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:00.526014 13385 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:19:00.527027 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:00.526033 12747 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:00.499614 15554 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:00.498796 16032 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:00.544034 12708 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:00.547951 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:00.518538 16030 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:00.555372 12629 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 9. Current term is 10. Ops: []
W20260709 13:19:00.555858 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 99860.8us
I20260709 13:19:00.470433 16074 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:00.570240 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:00.571604 16055 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:00.567307 16063 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:19:00.555613 15841 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:00.555864 15720 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:00.577340 15847 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:00.580698 12672 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:00.552232 15921 raft_consensus.cc:697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:00.594223 12672 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:00.546918 15981 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:00.581013 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:00.598933 12708 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:00.565448 15554 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:00.555771 16066 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:00.593174 12627 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:19:00.513700 15999 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:00.568734 16032 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.600693 12747 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:00.602505 12632 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:00.604358 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:00.490365 16054 consensus_queue.cc:237] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:00.581849 15458 consensus_queue.cc:1059] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:19:00.619501 16087 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:00.632509 15998 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:00.633869 15720 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:00.638113 15921 consensus_queue.cc:237] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:00.640377 15822 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:00.607906 15965 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:00.610621 16010 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:00.651103 16010 raft_consensus.cc:697] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:00.652077 16010 consensus_queue.cc:237] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:00.609465 16032 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:00.656826 15834 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:00.659498 12775 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:19:00.660115 12775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54346 (request call id 406) took 1114 ms. Trace:
I20260709 13:19:00.659969 15812 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:00.660506 12775 rpcz_store.cc:276] 0709 13:18:59.545418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.620681 (+ 75263us) service_pool.cc:224] Handling call
0709 13:19:00.659157 (+1038476us) spinlock_profiling.cc:243] Waited 890 us on lock 0x7b54001280a8. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7318f924 00007f4c72378227 00007f4c72377ab8 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:00.660054 (+   897us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2046848}
I20260709 13:19:00.641039 15822 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:00.611203 15847 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:00.663735 12775 raft_consensus.cc:1240] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:00.611891 16066 raft_consensus.cc:697] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:19:00.668332 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 211980us
I20260709 13:19:00.637105 15919 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:00.673748 16082 raft_consensus.cc:3055] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:19:00.680330 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 122574us
I20260709 13:19:00.680992 13385 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:00.682228 12746 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:00.687405 12746 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:00.688391 12747 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:00.692720 12598 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:00.689373 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:00.703222 12747 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:00.712649 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:00.715238 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:00.715479 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:00.715866 13384 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:00.672334 16019 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:00.654654 15965 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:00.723901 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:00.720074 12598 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:00.701793 16080 log.cc:927] Time spent T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 13:19:00.665707 15847 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:00.729112 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:00.732062 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:00.680546 16082 raft_consensus.cc:740] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:00.735049 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:19:00.732721 16079 log.cc:927] Time spent T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:19:00.702131 16087 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:00.730226 13383 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:19:00.729712 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:00.749138 12627 raft_consensus.cc:2410] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:19:00.712455 16073 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:00.713483 15931 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:00.743221 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.744087 12625 raft_consensus.cc:2368] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 8. Current term is 9.
I20260709 13:19:00.717458 16067 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:00.746179 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:00.750519 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:00.722007 15953 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:00.746912 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:00.751914 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:00.755616 12632 raft_consensus.cc:2368] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8.
I20260709 13:19:00.728911 15965 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:00.760712 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:00.761329 13378 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:19:00.733448 15592 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:00.761653 12597 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:00.735600 15847 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:00.771322 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 101007us
I20260709 13:19:00.725932 15496 consensus_queue.cc:1059] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:19:00.758261 16092 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.753146 16073 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:00.783310 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:00.795174 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 92063us
I20260709 13:19:00.758523 16067 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:00.796239 12747 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:00.775559 12673 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:19:00.810567 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 47603.6us
W20260709 13:19:00.814909 12597 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.554s (SENT) [suppressed 26 similar messages]
W20260709 13:19:00.816850 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.820228 15592 raft_consensus.cc:697] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:00.821615 15592 consensus_queue.cc:237] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:00.772907 15921 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:00.738505 16057 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:00.759213 15931 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:00.839625 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 152292us
I20260709 13:19:00.864907 12746 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:00.844998 16035 log.cc:927] Time spent T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 1.351s	user 0.005s	sys 0.000s
I20260709 13:19:00.877583 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:00.802479 16067 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:00.849469 16001 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:00.851790 15592 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.853329 15557 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:00.862581 12777 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:00.878531 12703 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:19:00.892364 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:00.893535 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:00.869655 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.804423 16087 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:00.802774 16073 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:00.896518 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:00.898458 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.899534 12707 raft_consensus.cc:2393] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 11: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:00.833521 15720 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:00.905076 12629 raft_consensus.cc:2393] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:19:00.911751 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:00.899035 16081 log.cc:927] Time spent T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:19:00.882001 15919 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:00.889812 15921 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.889285 16055 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:00.838536 15906 consensus_queue.cc:1059] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 13:19:00.998126 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:01.011508 12704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1475) took 1408 ms. Trace:
I20260709 13:19:00.884377 15812 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:00.885846 15834 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:00.921612 15998 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:00.924968 16094 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:00.917423 16074 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:00.828055 16092 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:00.921388 15822 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:01.043951 12704 rpcz_store.cc:276] 0709 13:18:59.603040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.605443 (+  2403us) service_pool.cc:224] Handling call
0709 13:18:59.605850 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.804548 (+198698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:01.000507 (+1195959us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b540021e8d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:01.011437 (+ 10930us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":455653248}
I20260709 13:19:00.921064 15999 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:01.000568 16088 log.cc:927] Time spent T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.386s	user 0.000s	sys 0.001s
W20260709 13:19:00.995239 15886 consensus_meta.cc:287] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.829s	user 0.008s	sys 0.004s
I20260709 13:19:00.796056 16066 consensus_queue.cc:237] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:01.053005 13384 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:19:01.056079 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1490) took 1399 ms. Trace:
I20260709 13:19:00.897495 15592 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:00.898869 16091 consensus_queue.cc:1059] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.402s
W20260709 13:19:01.060282 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.009733 16001 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:01.066417 15812 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.059679 15844 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:01.050798 16094 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.054740 15834 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:01.072134 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:01.116068 12781 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:01.073014 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:01.074298 16055 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.074703 16106 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.013742 15921 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:01.075515 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:01.132366 12776 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:01.084075 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:01.085179 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:01.133221 12704 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:19:01.133489 13378 raft_consensus.cc:2393] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 7: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:01.134336 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:01.135541 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:01.056636 15961 consensus_meta.cc:287] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.720s	user 0.008s	sys 0.003s
I20260709 13:19:01.141991 15961 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:01.144400 15961 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:01.088091 16093 log.cc:927] Time spent T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.322s	user 0.001s	sys 0.000s
I20260709 13:19:01.089187 12702 rpcz_store.cc:276] 0709 13:18:59.656167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.755693 (+ 99526us) service_pool.cc:224] Handling call
0709 13:19:01.055901 (+1300208us) spinlock_profiling.cc:243] Waited 406 ms on lock 0x7b54003eb7d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:01.056020 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":934916352}
I20260709 13:19:01.146651 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:01.104508 12706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1485) took 1451 ms. Trace:
I20260709 13:19:00.839833 16082 consensus_queue.cc:260] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:01.149494 12706 rpcz_store.cc:276] 0709 13:18:59.652971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.653103 (+   132us) service_pool.cc:224] Handling call
0709 13:19:01.104281 (+1451178us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b54003b33d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:01.104438 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":231674752}
I20260709 13:19:01.157064 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:01.157970 12706 raft_consensus.cc:2376] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10. Re-sending same reply.
I20260709 13:19:01.109869 12597 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:01.078130 16019 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:01.117764 12779 raft_consensus.cc:2393] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 9: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:01.166563 12779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36488 (request call id 1886) took 1245 ms. Trace:
I20260709 13:19:01.167096 12779 rpcz_store.cc:276] 0709 13:18:59.920872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.927556 (+  6684us) service_pool.cc:224] Handling call
0709 13:19:01.072827 (+1145271us) spinlock_profiling.cc:243] Waited 7.75 ms on lock 0x7b5800193e98. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c723780a0 00007f4c72377984 00007f4c6b8f26d4 00007f4c6b8f2662
0709 13:19:01.166479 (+ 93652us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17834880}
I20260709 13:19:01.086026 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:01.106599 15980 consensus_meta.cc:287] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.644s	user 0.009s	sys 0.003s
I20260709 13:19:01.182062 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:01.180883 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.111965 15844 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:01.111413 15812 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:01.199067 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 89306us
I20260709 13:19:01.115516 16094 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:01.165930 16090 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.415s	user 0.001s	sys 0.000s
I20260709 13:19:01.199112 13379 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:19:01.106487 15906 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:01.183739 12704 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:19:01.186187 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:01.209039 12597 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:19:01.215727 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 419536us
I20260709 13:19:01.216560 12747 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:01.222507 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.104094 15886 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:01.201197 15980 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:01.223392 12774 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1883) took 1303 ms. Trace:
I20260709 13:19:01.222961 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:01.222862 16069 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:01.208604 15945 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:01.203848 16082 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:01.244381 12747 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:01.210744 16108 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:01.246783 12704 raft_consensus.cc:2376] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8. Re-sending same reply.
I20260709 13:19:01.268534 12704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1795) took 1012 ms. Trace:
I20260709 13:19:01.166246 16055 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:01.208838 16019 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:01.256381 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:01.269965 13379 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:01.256532 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:01.246491 12774 rpcz_store.cc:276] 0709 13:18:59.919892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.930277 (+ 10385us) service_pool.cc:224] Handling call
0709 13:19:00.381379 (+451102us) spinlock_profiling.cc:243] Waited 5.01 ms on lock 0x7b5800193e98. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c7238bfc0 00007f4c72376929 00007f4c723763d2 00007f4c6b8edbd4
0709 13:19:00.381598 (+   219us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:00.424569 (+ 42971us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:00.424609 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:00.432235 (+  7626us) log.cc:844] Serialized 26 byte log entry
0709 13:19:00.504584 (+ 72349us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:00.504638 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:00.504726 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:01.216267 (+711541us) raft_consensus.cc:1685] finished
0709 13:19:01.216297 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:01.223322 (+  7025us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29865,"raft.queue_time_us":27173,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":35958,"spinlock_wait_cycles":11532160,"thread_start_us":30367,"threads_started":1,"wal-append.queue_time_us":105348}
I20260709 13:19:01.273587 12747 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:01.225893 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 450362us
I20260709 13:19:01.268962 12704 rpcz_store.cc:276] 0709 13:19:00.256367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.222668 (+966301us) service_pool.cc:224] Handling call
0709 13:19:01.268461 (+ 45793us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:01.179467 16106 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:01.274896 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:01.274290 12702 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:01.277516 12747 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:01.293246 12747 leader_election.cc:400] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8.
I20260709 13:19:01.293658 12747 leader_election.cc:403] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:19:01.278563 12776 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
W20260709 13:19:01.294062 12775 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.042s	sys 0.043s
W20260709 13:19:01.294505 12775 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.042s	sys 0.043s
I20260709 13:19:01.295593 12775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1895) took 1265 ms. Trace:
I20260709 13:19:01.218614 15906 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:01.312529 15906 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:01.313514 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.291867 16087 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:01.316718 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:01.315335 15844 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:01.319933 15844 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:01.296473 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1805) took 1037 ms. Trace:
W20260709 13:19:01.297755 15554 consensus_meta.cc:287] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.687s	user 0.007s	sys 0.007s
I20260709 13:19:01.296375 12775 rpcz_store.cc:276] 0709 13:19:00.029948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.756267 (+726319us) service_pool.cc:224] Handling call
0709 13:19:00.756577 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:00.756770 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:19:00.775362 (+ 18592us) log.cc:844] Serialized 14 byte log entry
0709 13:19:01.294893 (+519531us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:01.294956 (+    63us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:01.295002 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:01.295212 (+   210us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:01.295501 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18821,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":2083,"thread_start_us":519154,"threads_started":1}
I20260709 13:19:01.321869 15554 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:01.302331 16064 consensus_meta.cc:287] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.935s	user 0.011s	sys 0.002s
I20260709 13:19:01.322412 15921 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:01.322856 16064 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:01.323963 15554 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:01.324826 12706 raft_consensus.cc:2393] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 5: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:01.325275 16064 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:01.303819 16023 consensus_meta.cc:287] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.797s	user 0.008s	sys 0.001s
I20260709 13:19:01.326159 16023 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:01.328832 16023 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:01.330000 15898 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:01.330567 13348 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:01.320712 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:01.331920 16089 log.cc:927] Time spent T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.440s	user 0.002s	sys 0.000s
I20260709 13:19:01.331983 16103 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:01.333853 12782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2204) took 1373 ms. Trace:
I20260709 13:19:01.335312 15931 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:01.335996 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:01.306191 16092 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:01.308025 15942 consensus_meta.cc:287] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.950s	user 0.010s	sys 0.003s
I20260709 13:19:01.269438 16057 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:01.295699 15892 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:19:01.342602 16095 log.cc:927] Time spent T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 13:19:01.261044 15946 consensus_queue.cc:1059] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:19:01.343788 12778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1890) took 1356 ms. Trace:
I20260709 13:19:01.306181 15886 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:01.343916 12779 raft_consensus.cc:1240] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:01.310264 15834 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:01.276262 16108 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:01.343998 15942 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:01.344179 12778 rpcz_store.cc:276] 0709 13:18:59.987716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.627753 (+640037us) service_pool.cc:224] Handling call
0709 13:19:00.628101 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:00.628492 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:00.628524 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:00.629098 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 13:19:00.779252 (+150154us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:00.779308 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:00.798235 (+ 18927us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:01.343564 (+545329us) raft_consensus.cc:1685] finished
0709 13:19:01.343583 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:01.343729 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1397,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"thread_start_us":149713,"threads_started":1,"wal-append.queue_time_us":413621}
I20260709 13:19:01.346290 16108 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:01.334235 12782 rpcz_store.cc:276] 0709 13:18:59.960201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.359238 (+399037us) service_pool.cc:224] Handling call
0709 13:19:00.360023 (+   785us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:00.371871 (+ 11848us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:00.371903 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:00.407800 (+ 35897us) log.cc:844] Serialized 26 byte log entry
0709 13:19:00.595418 (+187618us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:00.704676 (+109258us) log.cc:844] Serialized 14 byte log entry
0709 13:19:00.798995 (+ 94319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:00.799095 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:01.333644 (+534549us) raft_consensus.cc:1685] finished
0709 13:19:01.333666 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:01.333786 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":89285,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":124589,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":6485,"thread_start_us":218738,"threads_started":2,"wal-append.queue_time_us":270904}
I20260709 13:19:01.334604 12704 raft_consensus.cc:2393] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:01.346746 15942 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:01.330586 12702 rpcz_store.cc:276] 0709 13:19:00.258967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.146332 (+887365us) service_pool.cc:224] Handling call
0709 13:19:01.296416 (+150084us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:01.272221 16069 raft_consensus.cc:697] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:19:01.339015 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:01.329275 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 72827us
I20260709 13:19:01.348663 12704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1501) took 1039 ms. Trace:
I20260709 13:19:01.347898 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:01.304849 16106 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:01.376675 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:01.379112 13379 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:01.355041 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:01.258232 16100 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:01.279899 15945 raft_consensus.cc:697] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:01.341677 16101 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:01.362391 12704 rpcz_store.cc:276] 0709 13:19:00.309713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.319378 (+1009665us) service_pool.cc:224] Handling call
0709 13:19:01.348497 (+ 29119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:01.382479 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:01.380817 12672 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:01.387666 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:01.334797 16099 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:01.289793 16001 consensus_queue.cc:1059] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 13:19:01.276495 16019 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:01.394271 16061 raft_consensus.cc:3055] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:01.381816 16087 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:19:01.391880 16032 consensus_meta.cc:287] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.702s	user 0.007s	sys 0.004s
I20260709 13:19:01.396065 16055 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:01.413830 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:01.426430 13379 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:01.426936 15952 consensus_meta.cc:287] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.896s	user 0.008s	sys 0.004s
I20260709 13:19:01.427356 15952 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:01.428211 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:01.414359 12598 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:01.429450 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:01.430256 12704 raft_consensus.cc:2393] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 8: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:01.430322 15814 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:01.432039 12700 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:01.391321 15998 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:01.416234 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:01.390789 15892 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:01.403578 16067 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:01.407816 16061 raft_consensus.cc:740] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:01.404233 16100 raft_consensus.cc:697] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:01.424521 16057 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:19:01.402448 16073 consensus_meta.cc:287] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.506s	user 0.005s	sys 0.013s
I20260709 13:19:01.436561 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:01.416116 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:01.432049 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:01.450976 12704 raft_consensus.cc:2376] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10. Re-sending same reply.
I20260709 13:19:01.436724 13379 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:01.434165 13384 raft_consensus.cc:1275] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:19:01.410061 15847 consensus_meta.cc:287] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.625s	user 0.009s	sys 0.001s
I20260709 13:19:01.446527 12672 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:01.459061 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:01.459923 12704 raft_consensus.cc:2393] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 10: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:01.460705 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:01.461210 12704 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:01.412082 16087 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:01.466914 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:19:01.469035 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 47541.8us
I20260709 13:19:01.470698 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:01.474108 12780 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:01.474139 12777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1898) took 1276 ms. Trace:
I20260709 13:19:01.444783 16054 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:19:01.505268 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.454308 16055 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.480590 12704 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:01.449522 15554 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:01.440086 16032 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:01.476574 12777 rpcz_store.cc:276] 0709 13:19:00.197708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.892929 (+695221us) service_pool.cc:224] Handling call
0709 13:19:00.893352 (+   423us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:19:00.893850 (+   498us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:00.893888 (+    38us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:19:00.894789 (+   901us) log.cc:844] Serialized 104 byte log entry
0709 13:19:00.920881 (+ 26092us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:01.009850 (+ 88969us) log.cc:844] Serialized 14 byte log entry
0709 13:19:01.396417 (+386567us) spinlock_profiling.cc:243] Waited 3.95 ms on lock 0x7b180036bc20. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a23fc0a 00007f4c7397f219 00007f4c6a2c7a40 00007f4c6a2c24e4 00007f4c6a2c2455 00007f4c70daa3c7 00007f4c6d3bc9a7 00007f4c6d3cc0bf 00007f4c6d3cc038
0709 13:19:01.396608 (+   191us) log.cc:844] Serialized 14 byte log entry
0709 13:19:01.459572 (+ 62964us) log.cc:844] Serialized 14 byte log entry
0709 13:19:01.468887 (+  9315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:01.473771 (+  4884us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:01.473852 (+    81us) raft_consensus.cc:1685] finished
0709 13:19:01.473872 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:01.474070 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":379342,"prepare.queue_time_us":499284,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":98,"raft.queue_time_us":140303,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":5822,"spinlock_wait_cycles":9079552,"thread_start_us":71723,"threads_started":2,"wal-append.queue_time_us":443260}
W20260709 13:19:01.523705 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 77298us
I20260709 13:19:01.471107 15892 raft_consensus.cc:697] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:01.473021 16101 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:01.473050 16087 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:01.516981 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.478015 16001 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:01.533978 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:01.445593 15998 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:01.522289 15437 consensus_meta.cc:287] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 1.008s	user 0.010s	sys 0.002s
I20260709 13:19:01.552285 12703 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:01.533761 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:01.553874 12704 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:01.534621 12777 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:01.539955 12781 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:01.556057 12705 raft_consensus.cc:2368] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9.
I20260709 13:19:01.478389 15834 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.542479 16061 consensus_queue.cc:260] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:01.524749 16006 consensus_meta.cc:287] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 1.237s	user 0.003s	sys 0.011s
I20260709 13:19:01.520424 16057 raft_consensus.cc:697] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:01.469219 16073 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:01.562456 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:01.514161 16128 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.584493 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:01.534583 15886 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:01.472039 15847 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:01.481986 15945 consensus_queue.cc:237] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:01.582329 16006 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:01.478102 16069 consensus_queue.cc:237] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:01.555974 12781 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36488 (request call id 1881) took 1636 ms (client timeout 1554 ms). Trace:
I20260709 13:19:01.578496 12782 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:01.602486 12781 rpcz_store.cc:269] 0709 13:18:59.919100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.925981 (+  6881us) service_pool.cc:224] Handling call
0709 13:19:01.071931 (+1145950us) spinlock_profiling.cc:243] Waited 3.11 ms on lock 0x7b5800193e98. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c723780a0 00007f4c72377984 00007f4c6b8f26d4 00007f4c6b8f2662
0709 13:19:01.555872 (+483941us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7164288}
W20260709 13:19:01.588692 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 77904.7us
I20260709 13:19:01.551424 16109 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:01.563325 15437 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:01.564404 15812 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:01.521014 16054 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:01.524477 16055 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:01.607923 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:01.604135 12747 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:01.590019 12777 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
W20260709 13:19:01.601979 15965 consensus_meta.cc:287] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.828s	user 0.007s	sys 0.004s
I20260709 13:19:01.610275 12672 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:01.609017 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:01.611157 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:01.612118 13348 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:01.613807 16122 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.606981 16115 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.597429 15886 raft_consensus.cc:697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:01.619710 12702 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:01.621852 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.606520 16061 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:01.627281 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:01.627105 16065 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:01.640323 12702 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:19:01.579205 15834 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:01.583588 16128 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:01.620281 16106 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:01.645774 16134 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:01.655656 16134 raft_consensus.cc:697] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:01.657055 16018 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:01.656661 16134 consensus_queue.cc:237] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:01.662665 15898 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8.
I20260709 13:19:01.646351 15496 raft_consensus.cc:3055] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:01.646669 15847 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:01.598497 16100 consensus_queue.cc:237] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:01.625839 16019 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:01.621137 15965 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:01.684626 12777 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:01.647710 16067 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:01.648320 12774 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2221) took 1369 ms. Trace:
W20260709 13:19:01.648427 16132 log.cc:927] Time spent T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:19:01.651546 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:01.698601 16055 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:01.679308 16131 log.cc:927] Time spent T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 13:19:01.652766 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:01.653776 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:01.670171 12704 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:01.616668 16109 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:01.674600 12782 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:01.717180 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:01.718322 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:01.718473 16109 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:01.719012 12782 raft_consensus.cc:2393] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 10: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:01.721230 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:01.722272 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:01.722389 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:01.723129 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:01.724336 12631 raft_consensus.cc:1240] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:19:01.666131 16001 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:01.724586 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:01.725253 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:01.724611 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:01.726003 13378 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 9
W20260709 13:19:01.674871 16111 log.cc:927] Time spent T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.338s	user 0.002s	sys 0.000s
I20260709 13:19:01.726361 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:01.676317 16106 raft_consensus.cc:697] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:01.671898 15496 raft_consensus.cc:740] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:01.678579 12700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 387) took 2007 ms. Trace:
I20260709 13:19:01.654529 16056 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:01.588519 16118 consensus_queue.cc:1048] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 13:19:01.693602 12774 rpcz_store.cc:276] 0709 13:19:00.278568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.279694 (+1001126us) service_pool.cc:224] Handling call
0709 13:19:01.289751 (+ 10057us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:01.290349 (+   598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:01.290415 (+    66us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:01.291847 (+  1432us) log.cc:844] Serialized 26 byte log entry
0709 13:19:01.445813 (+153966us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:01.480158 (+ 34345us) log.cc:844] Serialized 14 byte log entry
0709 13:19:01.480385 (+   227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:01.480458 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:01.648089 (+167631us) raft_consensus.cc:1685] finished
0709 13:19:01.648111 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:01.648260 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35047,"prepare.queue_time_us":34414,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":2830,"raft.queue_time_us":1395,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":43309,"thread_start_us":147700,"threads_started":1,"wal-append.queue_time_us":324270}
I20260709 13:19:01.743501 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:01.706848 12781 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:19:01.711007 12746 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:01.744863 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:01.655686 15998 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:01.745443 12626 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:01.746001 12597 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:01.747227 12597 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:01.714107 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:01.747815 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:01.675782 16054 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:01.726617 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:01.727708 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:01.728576 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:01.728755 12708 raft_consensus.cc:2393] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 6: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:01.676626 12780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2211) took 1550 ms. Trace:
I20260709 13:19:01.640164 16122 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:01.732724 12701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 382) took 2063 ms. Trace:
I20260709 13:19:01.619117 15892 consensus_queue.cc:237] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:01.732419 12700 rpcz_store.cc:276] 0709 13:18:59.671188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.916640 (+245452us) service_pool.cc:224] Handling call
0709 13:19:01.105507 (+1188867us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:01.432576 (+327069us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:01.677594 (+245018us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b54003b33d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:01.678386 (+   792us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54003b33d0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:01.678521 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3483912448}
W20260709 13:19:01.736047 12628 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56832 (request call id 2371) took 1689 ms (client timeout 1880 ms). Trace:
I20260709 13:19:01.715994 12776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2222) took 1360 ms. Trace:
I20260709 13:19:01.628089 16115 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:01.766644 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:01.767120 12782 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:01.766928 12598 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:01.769433 12705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38496 (request call id 389) took 1390 ms (client timeout 1542 ms). Trace:
W20260709 13:19:01.769984 12705 rpcz_store.cc:269] 0709 13:19:00.379378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.503422 (+124044us) service_pool.cc:224] Handling call
0709 13:19:01.769199 (+1265777us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400249ad4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:01.769362 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2324159744}
I20260709 13:19:01.771486 12783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2207) took 1810 ms. Trace:
I20260709 13:19:01.758643 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:01.771961 12783 rpcz_store.cc:276] 0709 13:18:59.961474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.414557 (+453083us) service_pool.cc:224] Handling call
0709 13:19:00.414941 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:00.568022 (+153081us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:00.568050 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:00.568089 (+    39us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:00.568115 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:01.771198 (+1203083us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b54001fe854. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:01.771255 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:01.771419 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":351601536}
I20260709 13:19:01.773664 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:01.670694 16057 consensus_queue.cc:237] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:01.753069 12704 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:01.753757 12780 rpcz_store.cc:276] 0709 13:19:00.126405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.860179 (+733774us) service_pool.cc:224] Handling call
0709 13:19:00.860474 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:01.474525 (+614051us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:01.676375 (+201850us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b5400089654. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:01.676561 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1410817920}
I20260709 13:19:01.779610 12704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1800) took 1522 ms. Trace:
I20260709 13:19:01.780042 12704 rpcz_store.cc:276] 0709 13:19:00.257485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.727492 (+1470007us) service_pool.cc:224] Handling call
0709 13:19:01.779533 (+ 52041us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:01.762977 12628 rpcz_store.cc:269] 0709 13:19:00.046427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.046550 (+   123us) service_pool.cc:224] Handling call
0709 13:19:01.735792 (+1689242us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b54002fe9d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:01.735978 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":231629056}
I20260709 13:19:01.748486 12702 raft_consensus.cc:2393] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:01.782109 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1497) took 1476 ms. Trace:
I20260709 13:19:01.763046 12776 rpcz_store.cc:276] 0709 13:19:00.355803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.294648 (+938845us) service_pool.cc:224] Handling call
0709 13:19:01.296077 (+  1429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:01.456114 (+160037us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:01.456153 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:01.456729 (+   576us) log.cc:844] Serialized 26 byte log entry
0709 13:19:01.628271 (+171542us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:01.628311 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:01.628376 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:01.715618 (+ 87242us) raft_consensus.cc:1685] finished
0709 13:19:01.715639 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:01.715929 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40349,"raft.queue_time_us":45940,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"thread_start_us":225581,"threads_started":2,"wal-append.queue_time_us":244133}
I20260709 13:19:01.782398 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:01.782410 12702 rpcz_store.cc:276] 0709 13:19:00.305285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.713815 (+1408530us) service_pool.cc:224] Handling call
0709 13:19:01.782053 (+ 68238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:01.783525 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:01.783655 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:19:01.783847 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:01.765874 12703 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1806) took 1506 ms (client timeout 1903 ms). Trace:
I20260709 13:19:01.784638 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:01.774736 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:01.774883 12783 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:19:01.775315 12673 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:01.759351 13378 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:01.785989 12627 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
W20260709 13:19:01.785004 12703 rpcz_store.cc:269] 0709 13:19:00.259223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.895839 (+636616us) service_pool.cc:224] Handling call
0709 13:19:01.765716 (+869877us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b54002486d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:01.765821 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":863188736}
I20260709 13:19:01.786670 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:01.778100 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:01.760355 12701 rpcz_store.cc:276] 0709 13:18:59.669469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.899917 (+230448us) service_pool.cc:224] Handling call
0709 13:19:01.011466 (+1111549us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:01.136579 (+125113us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:01.248579 (+112000us) log.cc:844] Serialized 14 byte log entry
0709 13:19:01.445055 (+196476us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:01.445578 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 13:19:01.445707 (+   129us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:01.445736 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:01.445810 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:01.719024 (+273214us) raft_consensus.cc:1685] finished
0709 13:19:01.719044 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:01.732553 (+ 13509us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540021e8d0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:01.732667 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11888,"prepare.queue_time_us":276909,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":76525,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":6410,"spinlock_wait_cycles":2562281600,"thread_start_us":374786,"threads_started":3,"wal-append.queue_time_us":258033}
I20260709 13:19:01.787555 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:01.788882 12783 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:01.788894 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:01.790982 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:01.791517 12701 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:19:01.776897 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:01.762544 15998 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:01.753088 16067 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:01.752460 16018 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:01.785346 12628 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:01.739486 16001 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:01.797992 12673 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:01.795890 12631 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:01.797884 12598 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:01.802657 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:01.777238 15931 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:01.807394 12598 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:01.809176 12747 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:01.811594 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:01.813746 12673 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:01.815547 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.794504 15980 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:01.794170 15437 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:01.819937 12673 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:01.820840 12631 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:19:01.826627 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:01.828625 12627 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:01.829572 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:01.830332 12780 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:01.827469 12700 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:01.832140 12700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1797) took 1575 ms. Trace:
I20260709 13:19:01.832517 12700 rpcz_store.cc:276] 0709 13:19:00.256805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.807101 (+1550296us) service_pool.cc:224] Handling call
0709 13:19:01.827306 (+ 20205us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:01.831878 (+  4572us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:01.832086 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:01.833881 12700 raft_consensus.cc:1240] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:01.832947 12673 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:01.836002 12700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1799) took 1578 ms. Trace:
I20260709 13:19:01.803153 15847 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:01.837543 12782 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:01.838644 12673 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:01.836359 12700 rpcz_store.cc:276] 0709 13:19:00.257255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.833354 (+1576099us) service_pool.cc:224] Handling call
0709 13:19:01.833704 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:01.834298 (+   594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:01.835937 (+  1639us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:01.819120 15949 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:01.819268 15906 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:01.752020 15886 consensus_queue.cc:237] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:01.841722 16067 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.841849 12709 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 10. Current term is 11. Ops: [8.9-10.10]
I20260709 13:19:01.853492 12709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1802) took 1595 ms. Trace:
I20260709 13:19:01.843603 16115 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:01.826293 16006 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:01.826586 15898 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:01.858038 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:01.822613 16054 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:01.844637 16073 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:01.818397 15844 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:01.846992 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:01.864612 13378 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:01.847607 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:01.802193 16106 consensus_queue.cc:237] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:01.795818 16123 consensus_queue.cc:1059] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:19:01.853952 12709 rpcz_store.cc:276] 0709 13:19:00.257989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.838087 (+1580098us) service_pool.cc:224] Handling call
0709 13:19:01.841588 (+  3501us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:01.853106 (+ 11518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:01.853353 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:01.837157 16010 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:01.861995 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:01.863144 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:01.836642 15437 raft_consensus.cc:697] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:01.874315 12747 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:01.853031 16121 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:01.853389 15949 raft_consensus.cc:697] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:01.845732 15456 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:01.859007 15906 raft_consensus.cc:697] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:01.871031 16122 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:01.787919 15496 consensus_queue.cc:260] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:01.858615 16109 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:19:01.853911 16094 consensus_meta.cc:287] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.637s	user 0.011s	sys 0.000s
I20260709 13:19:01.893746 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:01.896831 12774 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:19:01.897109 12627 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:01.860239 16067 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:01.879185 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:01.869663 15909 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:01.912566 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:01.849298 15847 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:01.886003 15965 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:01.928544 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:01.928143 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:01.965342 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.959517 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:01.999454 13378 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:19:02.000993 12627 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:01.984102 13379 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:01.877604 12626 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:01.974061 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:02.003819 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:02.004289 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:02.004686 12626 raft_consensus.cc:2410] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:19:02.002149 13380 raft_consensus.cc:2393] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 10: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:02.001210 12782 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 9: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:01.930111 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:02.005800 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:02.006755 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:02.007347 12777 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:19:02.008026 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:01.878566 15898 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:01.967796 12598 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:02.011363 12700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 391) took 1612 ms. Trace:
I20260709 13:19:01.876655 16121 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.885810 15834 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:02.013266 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:02.012333 12700 rpcz_store.cc:276] 0709 13:19:00.398817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.839898 (+1441081us) service_pool.cc:224] Handling call
0709 13:19:01.840191 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:01.840391 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:01.859579 (+ 19188us) log.cc:844] Serialized 14 byte log entry
0709 13:19:02.010908 (+151329us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:02.011005 (+    97us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:02.011067 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.011148 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:02.011284 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8078,"prepare.queue_time_us":14113,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"thread_start_us":142739,"threads_started":1}
I20260709 13:19:02.017207 12777 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:19:02.023353 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:02.018783 13380 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:19:01.911983 15950 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.975502 15953 raft_consensus.cc:3055] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:02.027418 12672 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:01.910956 16073 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:01.898483 16109 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.921882 15592 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:01.974059 16001 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:01.913323 15496 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:01.921231 16128 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:02.030911 12777 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:19:01.911180 16094 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:01.976496 15906 consensus_queue.cc:237] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:02.015445 16082 consensus_meta.cc:287] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.748s	user 0.008s	sys 0.003s
I20260709 13:19:02.035130 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:02.061901 12700 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 5. Current term is 7. Ops: []
I20260709 13:19:02.062922 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:02.039215 15953 raft_consensus.cc:740] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:02.049077 16109 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:02.012060 16122 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.045756 12776 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:02.016839 16121 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:02.008597 15437 consensus_queue.cc:237] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:02.087996 12629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56838 (request call id 1417) took 1355 ms. Trace:
I20260709 13:19:02.088477 12629 rpcz_store.cc:276] 0709 13:19:00.734641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.734816 (+   175us) service_pool.cc:224] Handling call
0709 13:19:02.087775 (+1352959us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b54000dc854. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:02.087931 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":389138688}
I20260709 13:19:02.043246 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:01.967676 16087 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:02.063728 12783 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
W20260709 13:19:02.097963 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 63221.4us
I20260709 13:19:02.098839 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:02.099123 12597 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:02.099460 12783 raft_consensus.cc:2376] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10. Re-sending same reply.
I20260709 13:19:02.043066 15950 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:02.066836 12777 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:02.047003 16145 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:19:02.069533 16141 log.cc:927] Time spent T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:19:02.070040 16094 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:02.058401 16019 consensus_queue.cc:1059] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:19:02.070981 15847 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:02.042302 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:02.048035 16101 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:02.029762 16010 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.921892 15949 consensus_queue.cc:237] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:02.081110 12776 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:19:02.081934 15999 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.038941 16054 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:19:02.015255 15945 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:02.094483 16122 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:02.139933 12783 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:02.049196 16073 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:02.111354 16145 raft_consensus.cc:697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:19:02.114053 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 128976us
I20260709 13:19:02.124862 15965 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:02.149334 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:02.152398 15998 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:02.151918 16145 consensus_queue.cc:237] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:02.134617 15999 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:02.143463 15841 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.102890 16069 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:02.122466 13385 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:02.126626 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:02.166448 12776 raft_consensus.cc:2393] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 6: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:02.168862 12776 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:02.105803 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:02.103093 16082 consensus_queue.cc:1059] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
W20260709 13:19:02.118978 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 77466.8us
I20260709 13:19:02.172613 12672 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:02.147408 12783 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:02.158998 13384 raft_consensus.cc:2393] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 6: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:02.174113 12783 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:02.119331 12746 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:19:02.174708 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 55514.1us
I20260709 13:19:02.156322 15965 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.128938 16010 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:02.117637 15953 consensus_queue.cc:260] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:02.186848 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:02.187651 12778 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:02.190137 12776 raft_consensus.cc:1275] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:02.170804 15841 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:02.194761 12634 raft_consensus.cc:1275] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:02.195710 15841 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:02.196856 12634 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:19:02.199879 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:02.200739 12705 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:02.201839 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:02.202491 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.202389 12776 raft_consensus.cc:1240] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:19:02.204461 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:02.205603 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:02.206221 12705 raft_consensus.cc:2376] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9. Re-sending same reply.
I20260709 13:19:02.207243 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:02.208745 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:02.209409 12705 raft_consensus.cc:2376] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8. Re-sending same reply.
I20260709 13:19:02.210435 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:02.213212 12709 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:02.186534 15847 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:02.186163 15965 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:02.188899 16082 consensus_queue.cc:1243] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer 3d539f8d95d343d8b1696803f18b30d2 log is divergent from this leader: its last log entry 10.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:19:02.224694 12598 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:02.188838 16067 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:02.189374 16123 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:02.212260 15953 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:02.235198 12747 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:02.239778 16006 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:19:02.240113 16150 log.cc:927] Time spent T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:19:02.240440 16006 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:02.218142 16106 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:02.244947 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.245361 16122 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:02.230957 16127 consensus_queue.cc:1059] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:19:02.246121 15950 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:02.220494 15847 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.246738 15847 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:02.233721 16082 raft_consensus.cc:3055] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:02.248194 16082 raft_consensus.cc:740] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:02.249197 16082 consensus_queue.cc:260] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:02.235198 16123 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.250537 16123 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:02.261950 16087 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:02.263510 15878 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.264057 15878 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:02.274957 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:02.234019 16069 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:19:02.277269 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 174976us
W20260709 13:19:02.279037 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 128868us
I20260709 13:19:02.282874 13384 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:02.275703 16082 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:02.274152 16067 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:02.272054 16087 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:02.299932 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.287639 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:02.278286 16101 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:02.279282 16069 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.278893 16010 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:02.304232 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 83018.7us
I20260709 13:19:02.272023 16152 consensus_queue.cc:1059] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:19:02.309067 12598 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:02.290789 15999 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:02.314239 15999 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:02.309195 16069 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:02.293783 16073 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:02.287358 16001 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:19:02.312516 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:02.304522 16087 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:02.321635 16152 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:02.334985 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:02.336370 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:02.339151 12707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1502) took 1966 ms (client timeout 1620 ms). Trace:
W20260709 13:19:02.339567 12707 rpcz_store.cc:269] 0709 13:19:00.372967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.702019 (+329052us) service_pool.cc:224] Handling call
0709 13:19:02.338966 (+1636947us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b5400404554. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:02.339089 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":427610880}
W20260709 13:19:02.345230 16157 log.cc:927] Time spent T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:19:02.322400 15965 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:02.348094 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:02.349437 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:02.350005 12707 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:19:02.350791 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:02.351271 12707 raft_consensus.cc:2376] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8. Re-sending same reply.
I20260709 13:19:02.322359 16159 consensus_queue.cc:1059] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:19:02.329900 16152 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:02.339962 16056 consensus_meta.cc:287] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.596s	user 0.010s	sys 0.002s
I20260709 13:19:02.358790 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:02.361058 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:02.343912 15949 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:19:02.382959 15965 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:02.359562 16121 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:02.384227 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:02.385432 16121 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:02.382534 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:02.386739 12704 raft_consensus.cc:1275] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:19:02.387295 12704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1816) took 1778 ms. Trace:
I20260709 13:19:02.387593 12704 rpcz_store.cc:276] 0709 13:19:00.609355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.386158 (+1776803us) service_pool.cc:224] Handling call
0709 13:19:02.386442 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:02.387131 (+   689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.387256 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:02.388479 12705 raft_consensus.cc:2368] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10.
I20260709 13:19:02.373636 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:02.390916 12704 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:02.370949 16152 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:02.387140 15911 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.392045 15911 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:02.393625 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:02.394193 12783 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:19:02.394989 12781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54346 (request call id 414) took 1013 ms. Trace:
I20260709 13:19:02.395386 12781 rpcz_store.cc:276] 0709 13:19:01.381200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.608811 (+227611us) service_pool.cc:224] Handling call
0709 13:19:02.394836 (+786025us) spinlock_profiling.cc:243] Waited 67.8 ms on lock 0x7b54001ebc54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:02.394934 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155904896}
I20260709 13:19:02.396508 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:02.396626 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:02.397112 12781 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:19:02.397207 12783 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:02.402839 16087 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:02.383448 16123 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:02.405473 16123 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:02.405956 12709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 392) took 2005 ms. Trace:
I20260709 13:19:02.406318 12709 rpcz_store.cc:276] 0709 13:19:00.400279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.869715 (+1469436us) service_pool.cc:224] Handling call
0709 13:19:01.872010 (+  2295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:02.213987 (+341977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.397429 (+183442us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b5400248bd4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:02.405896 (+  8467us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":784184064}
I20260709 13:19:02.407619 12709 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:02.408206 12709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 393) took 1273 ms. Trace:
I20260709 13:19:02.408494 12709 rpcz_store.cc:276] 0709 13:19:01.135308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.407185 (+1271877us) service_pool.cc:224] Handling call
0709 13:19:02.407464 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:02.407978 (+   514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.408155 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:02.431545 16159 raft_consensus.cc:3055] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:19:02.410909 12708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1807) took 2151 ms (client timeout 1707 ms). Trace:
I20260709 13:19:02.432080 12700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1504) took 1846 ms. Trace:
W20260709 13:19:02.432046 12708 rpcz_store.cc:269] 0709 13:19:00.259455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.604089 (+344634us) service_pool.cc:224] Handling call
0709 13:19:02.410738 (+1806649us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54001086d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:02.410850 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2576292224}
I20260709 13:19:02.432627 12709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 395) took 1294 ms. Trace:
I20260709 13:19:02.433121 12709 rpcz_store.cc:276] 0709 13:19:01.138729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.431820 (+1293091us) service_pool.cc:224] Handling call
0709 13:19:02.432200 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:02.432362 (+   162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:02.432385 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:02.432421 (+    36us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:02.432449 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.432496 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:02.432581 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:02.437543 15878 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:02.451433 16006 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:02.451721 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 47747.5us
I20260709 13:19:02.452415 13348 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:02.453511 13348 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:02.453801 16066 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:02.454232 16167 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:02.454449 16066 raft_consensus.cc:697] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:02.454640 16167 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.455080 16167 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:02.455628 16066 consensus_queue.cc:237] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:02.458303 12707 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:02.458918 12707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1819) took 1315 ms. Trace:
I20260709 13:19:02.459215 12707 rpcz_store.cc:276] 0709 13:19:01.144096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.457671 (+1313575us) service_pool.cc:224] Handling call
0709 13:19:02.457961 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:02.458679 (+   718us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.458864 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:02.462486 16071 log.cc:927] Time spent T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 2.138s	user 0.008s	sys 0.004s
I20260709 13:19:02.466634 16032 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:02.484683 12779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2219) took 2223 ms. Trace:
W20260709 13:19:02.489106 12625 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56838 (request call id 1394) took 2076 ms (client timeout 1516 ms). Trace:
W20260709 13:19:02.489553 12625 rpcz_store.cc:269] 0709 13:19:00.412625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.412772 (+   147us) service_pool.cc:224] Handling call
0709 13:19:02.488895 (+2076123us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b54002ce054. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:02.489035 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":758822784}
I20260709 13:19:02.414291 12783 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:19:02.416697 16123 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:02.419276 12706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38496 (request call id 390) took 2021 ms (client timeout 1911 ms). Trace:
W20260709 13:19:02.496763 12706 rpcz_store.cc:269] 0709 13:19:00.398025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.181701 (+783676us) service_pool.cc:224] Handling call
0709 13:19:02.419000 (+1237299us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b540014bed4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:02.419171 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326181248}
I20260709 13:19:02.420059 12701 raft_consensus.cc:1240] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:02.422112 13382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 608) took 1689 ms. Trace:
W20260709 13:19:02.422461 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.498607 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:02.499956 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:02.429809 16156 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:02.429816 16109 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:02.431164 12704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1817) took 1821 ms. Trace:
I20260709 13:19:02.403687 16018 raft_consensus.cc:3055] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:02.433876 12708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 396) took 1294 ms. Trace:
I20260709 13:19:02.505743 13384 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:19:02.413012 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:02.507615 12781 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:19:02.508989 13348 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:02.510308 12781 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:02.505971 12708 rpcz_store.cc:276] 0709 13:19:01.139953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.433062 (+1293109us) service_pool.cc:224] Handling call
0709 13:19:02.433306 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:02.433460 (+   154us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:02.433482 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:02.433518 (+    36us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:02.433548 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.433732 (+   184us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:02.433828 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:02.516873 12708 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:19:02.517433 12708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1507) took 1131 ms. Trace:
I20260709 13:19:02.517925 12708 rpcz_store.cc:276] 0709 13:19:01.386515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.516283 (+1129768us) service_pool.cc:224] Handling call
0709 13:19:02.516550 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:02.517227 (+   677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.517383 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:02.428562 16069 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:02.520524 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:02.521123 12630 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:02.523296 12706 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:02.523874 12706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1505) took 1190 ms. Trace:
I20260709 13:19:02.444329 16152 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:02.444875 12782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2227) took 1189 ms. Trace:
I20260709 13:19:02.410979 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:02.432811 12700 rpcz_store.cc:276] 0709 13:19:00.585236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.017387 (+1432151us) service_pool.cc:224] Handling call
0709 13:19:02.017694 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:02.062441 (+ 44747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.431782 (+369341us) spinlock_profiling.cc:243] Waited 44 ms on lock 0x7b5400172ad4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:02.432002 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":101223296}
I20260709 13:19:02.526830 12782 rpcz_store.cc:276] 0709 13:19:01.255161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.024816 (+769655us) service_pool.cc:224] Handling call
0709 13:19:02.049497 (+ 24681us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:02.170486 (+120989us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:02.170525 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:02.170575 (+    50us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:02.170619 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.444619 (+274000us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b540044cad4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:02.444668 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:02.444807 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":277811456}
I20260709 13:19:02.528201 12700 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:19:02.529733 12700 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:02.531826 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:02.533169 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:02.533814 12633 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:02.456991 12709 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:02.536160 12709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 397) took 1395 ms. Trace:
I20260709 13:19:02.536594 12709 rpcz_store.cc:276] 0709 13:19:01.141411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.449890 (+1308479us) service_pool.cc:224] Handling call
0709 13:19:02.450231 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:02.535939 (+ 85708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.536108 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:02.457320 15554 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:02.538086 15554 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:02.541167 12776 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:02.415549 15847 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:02.545701 12700 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:19:02.501401 12707 raft_consensus.cc:1240] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:02.502487 13379 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: [8.10-8.10]
W20260709 13:19:02.503090 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 139013us
I20260709 13:19:02.503239 12634 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:02.497859 13382 rpcz_store.cc:276] 0709 13:19:00.733159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.735721 (+  2562us) service_pool.cc:224] Handling call
0709 13:19:00.735998 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:00.736228 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:00.751209 (+ 14981us) log.cc:844] Serialized 14 byte log entry
0709 13:19:02.421481 (+1670272us) spinlock_profiling.cc:243] Waited 42.9 ms on lock 0x7b180023b4e0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6cf5a4bc 00007f4c6a2a848b 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c2455 00007f4c6d3aa060 00007f4c6d3bc716 00007f4c6d3cd287
0709 13:19:02.421630 (+   149us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:02.421724 (+    94us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:02.421783 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.421886 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:02.422041 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4724,"prepare.queue_time_us":14817,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":4222,"raft.queue_time_us":79383,"raft.run_cpu_time_us":13059,"raft.run_wall_time_us":73793,"spinlock_wait_cycles":98699008,"thread_start_us":248890,"threads_started":2,"wal-append.queue_time_us":467274,"wal-append.run_cpu_time_us":1414,"wal-append.run_wall_time_us":1059851}
I20260709 13:19:02.505781 12704 rpcz_store.cc:276] 0709 13:19:00.609644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.388146 (+1778502us) service_pool.cc:224] Handling call
0709 13:19:02.390716 (+  2570us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:02.430912 (+ 40196us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.431108 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:02.549716 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:02.524142 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:02.521173 16069 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:02.550689 12625 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:02.554802 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:02.555572 12625 raft_consensus.cc:2393] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 9: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:02.556665 12625 raft_consensus.cc:1240] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:19:02.556764 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:02.522949 16018 raft_consensus.cc:740] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:02.557756 16170 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:02.558255 16018 consensus_queue.cc:260] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:02.559423 16018 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:02.562513 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:02.524164 12706 rpcz_store.cc:276] 0709 13:19:01.333761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.504983 (+1171222us) service_pool.cc:224] Handling call
0709 13:19:02.505351 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:02.523673 (+ 18322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.523830 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:02.485250 12779 rpcz_store.cc:276] 0709 13:19:00.260856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.217980 (+957124us) service_pool.cc:224] Handling call
0709 13:19:01.343723 (+125743us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:01.344482 (+   759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.484421 (+1139939us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b540008a050. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:02.484592 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":288384768}
I20260709 13:19:02.563869 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:02.564860 12627 raft_consensus.cc:2393] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 8: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:02.496505 15370 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.567960 15370 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:02.529975 15931 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:19:02.571115 15370 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:02.572649 12708 raft_consensus.cc:1275] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:02.550587 12776 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:02.566717 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:02.577625 12625 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:02.507030 16123 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:02.525429 16156 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:02.586568 12673 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:02.587700 16109 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:02.588311 16109 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:02.588457 15911 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:02.588606 12673 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:02.590250 16032 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:02.590782 16032 raft_consensus.cc:697] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:02.592096 13385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 601) took 1878 ms. Trace:
I20260709 13:19:02.591754 16032 consensus_queue.cc:237] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:02.592514 13385 rpcz_store.cc:276] 0709 13:19:00.713994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.714119 (+   125us) service_pool.cc:224] Handling call
0709 13:19:00.714408 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:02.165551 (+1451143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.591842 (+426291us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54004bfd54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:02.592031 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3237782272}
I20260709 13:19:02.596596 12629 raft_consensus.cc:1275] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:02.598030 12629 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:19:02.598366 12783 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:19:02.601281 16123 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:02.605010 12630 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:02.605953 12597 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:02.607681 12597 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:02.608279 12629 raft_consensus.cc:3055] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:02.608659 12629 raft_consensus.cc:740] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:02.608954 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:02.609517 12629 consensus_queue.cc:260] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:02.610571 12629 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:02.612183 12778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36478 (request call id 2233) took 1156 ms. Trace:
I20260709 13:19:02.612586 12778 rpcz_store.cc:276] 0709 13:19:01.455401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.627005 (+171604us) service_pool.cc:224] Handling call
0709 13:19:02.611831 (+984826us) spinlock_profiling.cc:243] Waited 497 ms on lock 0x7b54001fd454. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:02.612123 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1142746496}
I20260709 13:19:02.437646 16159 raft_consensus.cc:740] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:02.613531 16152 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:02.613770 12777 raft_consensus.cc:1240] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:19:02.614202 16152 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:02.614391 16159 consensus_queue.cc:260] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:02.615412 16159 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:02.615535 15554 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:02.617883 12709 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:02.618747 15885 raft_consensus.cc:3055] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:02.619300 15885 raft_consensus.cc:740] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:02.620357 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:02.620563 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:02.549607 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:02.621027 12631 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:19:02.621657 12634 raft_consensus.cc:2393] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 12: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:02.609030 12708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1829) took 1079 ms. Trace:
I20260709 13:19:02.578678 15966 raft_consensus.cc:3055] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:02.629326 15966 raft_consensus.cc:740] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:02.629948 15885 consensus_queue.cc:260] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:02.631503 15885 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:02.631768 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 122917us
I20260709 13:19:02.607640 15953 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:02.634049 15847 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:02.630299 15966 consensus_queue.cc:260] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:02.635078 15966 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:02.635560 15911 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:02.635996 15911 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.638185 15911 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:02.622051 12708 rpcz_store.cc:276] 0709 13:19:01.529357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.518609 (+989252us) service_pool.cc:224] Handling call
0709 13:19:02.518876 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:02.573043 (+ 54167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.608783 (+ 35740us) spinlock_profiling.cc:243] Waited 53.4 ms on lock 0x7b540021e3d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:02.608965 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":122845568}
W20260709 13:19:02.623592 12517 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 981fb2f435e24d06a81fc02cf4591cc5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 19cd864c053744bebd96093a7982537e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:02.627146 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:02.645740 15953 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:02.647415 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:02.647630 16082 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:02.648114 16082 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.648490 16082 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:02.648769 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:02.620607 12627 raft_consensus.cc:1275] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:19:02.640908 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:02.652204 12625 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:02.642091 12778 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:19:02.642712 15998 consensus_queue.cc:1048] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:02.622373 12633 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:19:02.655169 13382 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:19:02.667009 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.667164 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:02.652127 12777 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:02.660046 12776 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:02.668308 12598 leader_election.cc:400] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10.
I20260709 13:19:02.668678 12598 leader_election.cc:403] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:19:02.642676 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:02.677002 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:02.683854 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:02.692703 13382 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:19:02.697218 12629 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:19:02.697754 12629 pending_rounds.cc:85] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 5
I20260709 13:19:02.698020 12629 pending_rounds.cc:107] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:19:02.698267 12629 raft_consensus.cc:2889] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:02.698388 12673 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:19:02.697068 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 45861.8us
W20260709 13:19:02.700826 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 91962.9us
I20260709 13:19:02.704264 12597 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:02.707481 12705 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:19:02.708875 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:02.547683 12707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1820) took 1402 ms. Trace:
I20260709 13:19:02.709651 12705 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:19:02.710011 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:02.709899 12707 rpcz_store.cc:276] 0709 13:19:01.145664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.459890 (+1314226us) service_pool.cc:224] Handling call
0709 13:19:02.501170 (+ 41280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:02.547422 (+ 46252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.547620 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:02.710822 12782 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:02.713150 12782 raft_consensus.cc:1275] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:19:02.706969 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:02.719362 12709 raft_consensus.cc:2393] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:02.715822 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:02.714871 12672 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:02.720580 12673 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:02.721606 12782 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:19:02.722710 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:02.723914 12706 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:02.735818 12708 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:19:02.740321 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:02.743461 12705 raft_consensus.cc:2393] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 5: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
W20260709 13:19:02.744901 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:02.745807 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 47771.6us
I20260709 13:19:02.741129 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:02.748191 12782 raft_consensus.cc:2393] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 9: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
W20260709 13:19:02.733043 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.699564 16018 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10.
I20260709 13:19:02.753240 12704 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:19:02.732278 12597 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:02.717888 16152 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:19:02.770144 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 79224.4us
W20260709 13:19:02.770465 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 50009.3us
I20260709 13:19:02.771571 12673 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:02.763902 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:19:02.772805 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 57984.9us
I20260709 13:19:02.696888 16109 consensus_queue.cc:1048] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.092s
W20260709 13:19:02.754127 16108 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.081s	user 0.005s	sys 0.065s
I20260709 13:19:02.696775 15953 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:19:02.752580 15554 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:02.701663 15592 consensus_queue.cc:1048] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:19:02.705961 15847 consensus_queue.cc:1048] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:19:02.734730 15557 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:19:02.779917 16108 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.119s	user 0.005s	sys 0.066s
I20260709 13:19:02.772755 15456 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:02.786321 15946 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:02.769788 16152 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.754743 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:02.761119 16127 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:02.761129 12782 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:02.821238 12706 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:02.762362 12597 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:02.843995 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 81721.5us
I20260709 13:19:02.844816 12597 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:02.822628 12777 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:02.779569 15952 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:02.763307 16103 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:19:02.792979 16087 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:02.837848 12780 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:02.836248 12672 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:02.752089 16100 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:02.837137 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 44602.8us
I20260709 13:19:02.765259 16116 consensus_queue.cc:1059] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:19:02.772595 16056 consensus_queue.cc:1059] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:19:02.857519 12673 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:02.865078 12597 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:02.842374 16152 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:02.842770 15554 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:19:02.844434 15946 raft_consensus.cc:697] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:02.849970 16107 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:02.869957 16018 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:19:02.881762 16177 log.cc:927] Time spent T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
W20260709 13:19:02.893904 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 122794us
I20260709 13:19:02.863368 15844 consensus_queue.cc:1059] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:19:02.895454 12746 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:02.885192 15931 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:02.867635 16065 consensus_queue.cc:1059] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:19:02.863070 16167 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:02.899976 16061 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:02.900180 16010 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:02.903024 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:02.866605 16087 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:02.870361 15557 consensus_queue.cc:1059] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:19:02.907341 16010 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.908864 16010 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:02.901336 15909 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:02.938452 12703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1822) took 1735 ms. Trace:
W20260709 13:19:02.957875 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 61654.1us
I20260709 13:19:03.024972 12703 rpcz_store.cc:276] 0709 13:19:01.203080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.474521 (+1271441us) service_pool.cc:224] Handling call
0709 13:19:02.474900 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:02.475285 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:02.475323 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:02.475854 (+   531us) log.cc:844] Serialized 26 byte log entry
0709 13:19:02.663666 (+187812us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:02.821875 (+158209us) log.cc:844] Serialized 14 byte log entry
0709 13:19:02.899717 (+ 77842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.899796 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:02.938227 (+ 38431us) raft_consensus.cc:1685] finished
0709 13:19:02.938248 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:02.938381 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77563,"raft.queue_time_us":91299,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":147,"thread_start_us":269989,"threads_started":2,"wal-append.queue_time_us":409628}
I20260709 13:19:03.026605 15911 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:03.027180 12597 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:03.027462 16082 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:03.028688 13385 raft_consensus.cc:2368] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 8. Current term is 9.
I20260709 13:19:03.029690 16115 raft_consensus.cc:2764] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:19:03.029901 15878 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:03.029628 16069 consensus_queue.cc:1048] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:03.030165 15844 raft_consensus.cc:3055] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:03.030638 15878 raft_consensus.cc:697] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:19:02.961537 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.030640 15844 raft_consensus.cc:740] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:03.033093 15592 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.032809 15844 consensus_queue.cc:260] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:03.033691 15592 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:03.034305 15844 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:03.035920 15592 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:02.902426 15847 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:02.898145 16159 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:19:03.036338 15878 consensus_queue.cc:237] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:03.006716 16094 raft_consensus.cc:3055] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:03.038007 16094 raft_consensus.cc:740] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:02.895078 16018 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:03.041884 16018 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:03.044903 16092 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.045686 16092 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:03.047386 16083 raft_consensus.cc:3055] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:03.047842 16083 raft_consensus.cc:740] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:02.997135 12701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1796) took 2740 ms. Trace:
I20260709 13:19:03.048588 16092 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:03.048810 12701 rpcz_store.cc:276] 0709 13:19:00.256585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.792087 (+1535502us) service_pool.cc:224] Handling call
0709 13:19:02.419861 (+627774us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:02.502609 (+ 82748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.996872 (+494263us) spinlock_profiling.cc:243] Waited 627 ms on lock 0x7b540014bed0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:02.997060 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1443115264}
I20260709 13:19:03.005468 16010 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:03.051563 16010 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:03.052910 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:03.053819 16156 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:03.055647 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.055547 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:03.056139 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.056344 12703 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
W20260709 13:19:03.020802 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 184674us
W20260709 13:19:03.058311 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 156874us
I20260709 13:19:03.058553 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:03.058732 12672 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:03.060256 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:03.061496 12708 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:19:03.059144 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:03.064226 12703 raft_consensus.cc:2393] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 10: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:02.906529 15953 consensus_queue.cc:1059] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.101s
W20260709 13:19:03.019806 16183 log.cc:927] Time spent T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:19:03.071244 12673 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:02.892902 16100 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:03.066625 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:03.070432 12672 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:03.085533 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:03.022362 16175 log.cc:927] Time spent T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.189s	user 0.000s	sys 0.007s
I20260709 13:19:02.984055 16001 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:03.088954 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:03.092710 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:03.095117 12707 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:19:03.105814 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:03.111565 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1821) took 1913 ms. Trace:
I20260709 13:19:03.112071 12702 rpcz_store.cc:276] 0709 13:19:01.198519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.471890 (+1273371us) service_pool.cc:224] Handling call
0709 13:19:02.472449 (+   559us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:02.472902 (+   453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:02.472942 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:02.473486 (+   544us) log.cc:844] Serialized 26 byte log entry
0709 13:19:02.700553 (+227067us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:02.876610 (+176057us) log.cc:844] Serialized 14 byte log entry
0709 13:19:02.901139 (+ 24529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.901210 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:03.111368 (+210158us) raft_consensus.cc:1685] finished
0709 13:19:03.111390 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:03.111506 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":186778,"raft.queue_time_us":43455,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":1645,"thread_start_us":276966,"threads_started":2,"wal-append.queue_time_us":343998}
I20260709 13:19:03.105504 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:03.117414 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 46230.6us
I20260709 13:19:03.118583 12673 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:03.117482 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 47164.1us
I20260709 13:19:03.127673 12672 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:03.093878 16165 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:03.065814 15946 consensus_queue.cc:237] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:03.093438 16082 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:03.147056 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:03.191821 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:03.204350 12705 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:03.194566 12632 pending_rounds.cc:85] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 5
I20260709 13:19:03.205237 12632 pending_rounds.cc:107] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:19:03.205649 12632 raft_consensus.cc:2889] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:03.132444 16057 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:19:03.148864 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:03.211715 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.135653 16108 raft_consensus.cc:3055] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:03.126125 15950 raft_consensus.cc:3055] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:03.133002 15554 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.131678 16010 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:03.208920 12704 raft_consensus.cc:1240] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:03.196192 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:03.125131 15966 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.093s
W20260709 13:19:03.202175 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 74648.1us
I20260709 13:19:03.048645 16083 consensus_queue.cc:260] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:03.185974 16023 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:03.229151 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:03.230233 12705 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:19:03.134503 15911 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:03.132278 16001 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:03.114704 16123 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.108s
I20260709 13:19:03.216506 16057 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:03.211078 15965 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:03.200163 15370 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:19:03.202844 15980 consensus_queue.cc:1048] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:19:03.217131 16108 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:03.221360 15950 raft_consensus.cc:740] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:03.241719 16057 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:03.226742 15554 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:03.250392 16121 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:03.253293 15952 consensus_queue.cc:1048] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:19:03.267130 16156 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.267866 16156 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:03.260733 16167 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:03.271611 16180 log.cc:927] Time spent T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.274s	user 0.001s	sys 0.000s
I20260709 13:19:03.272619 15911 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.282168 16087 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:03.300611 16190 log.cc:927] Time spent T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.101s	user 0.001s	sys 0.001s
W20260709 13:19:03.304488 16193 log.cc:927] Time spent T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:19:03.308535 16057 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:03.310075 16018 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:03.312007 16057 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:03.313012 16018 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:03.296032 16100 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:03.308470 16152 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:03.315593 16156 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:03.316632 16010 raft_consensus.cc:697] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:03.318177 16152 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:03.370776 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 342142us
I20260709 13:19:03.375661 12597 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:03.389166 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:03.216997 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:03.395049 12627 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:03.399530 12633 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:03.402248 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:03.420511 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:03.421180 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:03.424149 12627 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:19:03.424564 12705 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 8. Current term is 9. Ops: []
W20260709 13:19:03.427114 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.427518 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:03.427680 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:03.436112 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.440658 12774 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:19:03.443269 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:03.445945 12781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 425) took 1042 ms. Trace:
I20260709 13:19:03.446452 12780 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:03.447212 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:03.446590 12781 rpcz_store.cc:276] 0709 13:19:02.403652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.510981 (+107329us) service_pool.cc:224] Handling call
0709 13:19:02.511258 (+   277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:02.511494 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:02.511510 (+    16us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:02.511990 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 13:19:02.795649 (+283659us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:02.795688 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.795771 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:03.445317 (+649546us) raft_consensus.cc:1685] finished
0709 13:19:03.445355 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:03.445713 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":97,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":129,"thread_start_us":283349,"threads_started":1,"wal-append.queue_time_us":382805}
I20260709 13:19:03.448252 12780 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:03.450138 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:19:03.281047 15885 consensus_meta.cc:287] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.649s	user 0.009s	sys 0.009s
I20260709 13:19:03.469426 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:03.475061 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:03.474249 15812 consensus_queue.cc:1059] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
W20260709 13:19:03.477478 12528 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40528 (request call id 136) took 20031 ms (client timeout 15000 ms). Trace:
I20260709 13:19:03.478080 15953 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:03.479107 15953 raft_consensus.cc:697] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:03.479992 15592 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:03.487258 16001 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:03.490051 16057 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.490914 16057 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:03.491664 15841 raft_consensus.cc:3055] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:03.492148 15841 raft_consensus.cc:740] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:03.480706 15911 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:03.473492 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:03.281544 15950 consensus_queue.cc:260] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:03.503084 16087 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:03.505491 12781 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:03.507768 16087 consensus_queue.cc:1059] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:03.509799 16001 raft_consensus.cc:3055] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:03.510193 16001 raft_consensus.cc:740] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:03.511865 16001 consensus_queue.cc:260] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:03.514397 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 57794.3us
I20260709 13:19:03.517537 12627 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 10. (term mismatch)
I20260709 13:19:03.518008 12627 pending_rounds.cc:85] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 9
I20260709 13:19:03.518158 12631 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:19:03.518580 12627 pending_rounds.cc:107] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 8.10
I20260709 13:19:03.518967 12627 raft_consensus.cc:2889] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:03.503499 15950 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:03.525663 16107 raft_consensus.cc:3055] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:03.526126 16107 raft_consensus.cc:740] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:03.526518 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:03.527434 16107 consensus_queue.cc:260] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:03.532232 15911 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:03.495529 16010 consensus_queue.cc:237] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:03.534463 16087 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:03.536593 16087 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:03.538232 15554 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:03.517913 12747 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:03.554479 15841 consensus_queue.cc:260] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:03.557451 12774 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:03.561182 15841 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:03.481122 15953 consensus_queue.cc:237] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:03.569658 16057 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:03.571070 16184 log.cc:927] Time spent T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.687s	user 0.002s	sys 0.000s
W20260709 13:19:03.573268 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.582875 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:03.583244 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:03.584086 13378 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:03.584913 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:03.585655 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:03.595357 12781 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
W20260709 13:19:03.569012 16182 log.cc:927] Time spent T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.339s	user 0.000s	sys 0.001s
I20260709 13:19:03.580731 16108 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:03.613170 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.618090 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:03.618268 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 100784us
I20260709 13:19:03.618908 12626 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:19:03.620366 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:03.621503 12625 raft_consensus.cc:2410] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:19:03.622897 16100 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:03.631400 16107 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:03.633327 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:03.635552 16170 consensus_queue.cc:1059] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:19:03.478046 12528 rpcz_store.cc:269] 0709 13:18:43.445814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.446236 (+   422us) service_pool.cc:224] Handling call
0709 13:19:03.477188 (+20030952us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:00.184887 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:00.226032 (+ 41145us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:00.226072 (+    40us) write_op.cc:435] Acquired schema lock
0709 13:19:00.226089 (+    17us) tablet.cc:662] Decoding operations
0709 13:19:00.461819 (+235730us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:00.467126 (+  5307us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:00.471190 (+  4064us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:19:00.780713 (+309523us) tablet.cc:701] Row locks acquired
0709 13:19:00.780754 (+    41us) write_op.cc:260] PREPARE: finished
0709 13:19:00.829149 (+ 48395us) write_op.cc:270] Start()
0709 13:19:00.838581 (+  9432us) write_op.cc:276] Timestamp: P: 1783603140825637 usec, L: 0
0709 13:19:00.838616 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 13:19:01.056318 (+217702us) log.cc:844] Serialized 6672 byte log entry
0709 13:19:01.893193 (+836875us) op_driver.cc:464] REPLICATION: finished
0709 13:19:02.128162 (+234969us) write_op.cc:301] APPLY: starting
0709 13:19:02.136543 (+  8381us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:02.219498 (+ 82955us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:02.236282 (+ 16784us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:02.238162 (+  1880us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:02.238452 (+   290us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 13:19:02.238493 (+    41us) write_op.cc:312] APPLY: finished
0709 13:19:02.238832 (+   339us) log.cc:844] Serialized 119 byte log entry
0709 13:19:02.239002 (+   170us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:02.239159 (+   157us) write_op.cc:454] Released schema lock
0709 13:19:02.239222 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:02.411987 (+172765us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b540011eed4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3ade28 00007f4c70de8581 00007f4c70de7d58 00007f4c70dec9f1 00007f4c70dec9a9 00007f4c70dec939 00007f4c70dec901 00007f4c70debb2d 000000000038f354 000000000038f199
0709 13:19:02.412297 (+   310us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:02.412333 (+    36us) write_op.cc:454] Released schema lock
0709 13:19:02.483616 (+ 71283us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b540011eed4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3af32a 00007f4c6d334f14 00007f4c6d334ec3 00007f4c6d334e38 00007f4c6d334df3 00007f4c6d3340dc 00007f4c6d35de16 00007f4c6d345cbb 00007f4c6d33039d 00007f4c6d333d75
Metrics: {"mutex_wait_us":83199,"reported_tablets":25,"child_traces":[["op",{"apply.queue_time_us":219233,"apply.run_cpu_time_us":4587,"apply.run_wall_time_us":117586,"num_ops":17,"prepare.queue_time_us":311015,"prepare.run_cpu_time_us":50339,"prepare.run_wall_time_us":2249173,"raft.queue_time_us":178273,"raft.run_cpu_time_us":63857,"raft.run_wall_time_us":631009,"replication_time_us":1022033,"spinlock_wait_cycles":74172928,"thread_start_us":737266,"threads_started":4,"wal-append.queue_time_us":554935,"wal-append.run_cpu_time_us":18046,"wal-append.run_wall_time_us":1621633}]]}
I20260709 13:19:03.636651 16170 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:03.637195 16170 raft_consensus.cc:697] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 9 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:03.637844 16116 raft_consensus.cc:3055] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:03.611392 12597 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:03.621236 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:03.644737 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:03.655088 12625 raft_consensus.cc:1240] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: []
W20260709 13:19:03.660553 16206 log.cc:927] Time spent T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 13:19:03.667361 16087 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:03.666819 16170 consensus_queue.cc:237] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:03.676407 16094 consensus_queue.cc:260] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:03.678921 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:03.674249 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 444981us
I20260709 13:19:03.681665 15911 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:03.620378 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:03.690079 15966 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:03.643684 16116 raft_consensus.cc:740] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:19:03.695403 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 84228.8us
W20260709 13:19:03.737000 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 116788us
I20260709 13:19:03.736886 16116 consensus_queue.cc:260] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:03.738232 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:03.738776 16116 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:03.740701 16196 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:03.741606 16196 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:03.723516 16100 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:03.743860 16087 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:03.688555 16010 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
W20260709 13:19:03.784137 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.784495 16115 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:03.793888 16115 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:03.794379 16115 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:03.799152 16094 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:03.800163 12780 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:03.743701 15966 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
W20260709 13:19:03.784904 15878 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.017s	sys 0.006s
W20260709 13:19:03.804669 15878 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.017s	sys 0.006s
I20260709 13:19:03.805668 12672 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:03.808938 15946 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:03.810672 16100 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:03.790060 16127 consensus_queue.cc:1059] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:19:03.743115 12708 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:19:03.815759 16195 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:03.819052 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.782886 16108 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:03.837441 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.829593 16195 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:03.832131 12780 raft_consensus.cc:1275] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:19:03.834128 16087 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:03.825284 12708 raft_consensus.cc:606] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:03.753147 15911 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:03.839265 13385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51762 (request call id 610) took 1188 ms. Trace:
I20260709 13:19:03.842084 13385 rpcz_store.cc:276] 0709 13:19:02.646702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.666765 (+ 20063us) service_pool.cc:224] Handling call
0709 13:19:03.834888 (+1168123us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b5400512f54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:03.835139 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":828885376}
I20260709 13:19:03.843165 12632 raft_consensus.cc:3055] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:03.845142 12632 raft_consensus.cc:740] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:19:03.849869 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 109900us
I20260709 13:19:03.823827 16100 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:03.858253 12774 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:19:03.864846 16001 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:03.868160 15557 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 13:19:03.868803 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:03.868673 16087 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:03.870100 15557 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:03.871939 16087 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:03.872726 15919 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:03.874061 16179 log.cc:927] Time spent T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.993s	user 0.000s	sys 0.003s
I20260709 13:19:03.875674 16069 consensus_queue.cc:1048] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:03.876466 12780 raft_consensus.cc:1240] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:19:03.877164 12775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 409) took 3190 ms. Trace:
I20260709 13:19:03.877540 12775 rpcz_store.cc:276] 0709 13:19:00.686160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.322052 (+635892us) service_pool.cc:224] Handling call
0709 13:19:01.322365 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:01.348047 (+ 25682us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:01.348075 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:01.348469 (+   394us) log.cc:844] Serialized 26 byte log entry
0709 13:19:02.492834 (+1144365us) spinlock_profiling.cc:243] Waited 6.12 ms on lock 0x7b18001c0c00. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf520d1 00007f4c6d36da0f
0709 13:19:02.653975 (+161141us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:02.754781 (+100806us) log.cc:844] Serialized 14 byte log entry
0709 13:19:02.754989 (+   208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.755054 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:03.822428 (+1067374us) spinlock_profiling.cc:243] Waited 3.73 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:03.876778 (+ 54350us) raft_consensus.cc:1685] finished
0709 13:19:03.876797 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:03.876992 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27589,"prepare.queue_time_us":87748,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":6989,"raft.queue_time_us":34038,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":14293,"spinlock_wait_cycles":22651520,"thread_start_us":160865,"threads_started":1,"wal-append.queue_time_us":1451310}
I20260709 13:19:03.881289 16198 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:03.883309 15998 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:03.884359 12701 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:03.887959 16168 raft_consensus.cc:3055] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:03.888378 16168 raft_consensus.cc:740] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:03.889739 15946 raft_consensus.cc:3055] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:03.890255 15946 raft_consensus.cc:740] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:03.891388 15946 consensus_queue.cc:260] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:03.894210 16092 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:03.898401 12632 consensus_queue.cc:260] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:03.899628 15931 consensus_queue.cc:1059] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:03.900081 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:03.901938 16103 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:03.905216 16083 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:03.910655 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 49999.1us
W20260709 13:19:03.911428 16216 log.cc:927] Time spent T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.065s	user 0.002s	sys 0.000s
I20260709 13:19:03.913333 16006 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.916795 15911 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:03.865242 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:03.917877 16195 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:03.938639 16181 log.cc:927] Time spent T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.934s	user 0.004s	sys 0.000s
I20260709 13:19:03.931171 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:03.940382 13380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 623) took 1455 ms. Trace:
I20260709 13:19:03.940974 13380 rpcz_store.cc:276] 0709 13:19:02.485154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:02.485282 (+   128us) service_pool.cc:224] Handling call
0709 13:19:02.485668 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:02.487414 (+  1746us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:02.487463 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:02.507993 (+ 20530us) log.cc:844] Serialized 26 byte log entry
0709 13:19:02.770392 (+262399us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:02.770433 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:02.777717 (+  7284us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:03.896478 (+1118761us) spinlock_profiling.cc:243] Waited 7.85 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:03.938140 (+ 41662us) spinlock_profiling.cc:243] Waited 25.3 ms on lock 0x7b2400317a68. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a14b9c4 00007f4c6a149acb 00007f4c6a304dc9 00007f4c6a304a55 00007f4c6a30409d 00007f4c6cf3610e 00007f4c6cf4d88c 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221
0709 13:19:03.939922 (+  1782us) raft_consensus.cc:1685] finished
0709 13:19:03.939953 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:03.940182 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":988,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":216,"spinlock_wait_cycles":76214784,"thread_start_us":262037,"threads_started":1,"wal-append.queue_time_us":396325}
I20260709 13:19:03.942510 12632 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:03.931425 12701 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:19:03.943513 12701 raft_consensus.cc:606] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:19:03.931952 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:03.948244 12625 raft_consensus.cc:1275] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:19:03.949527 16006 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:03.956382 12598 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:03.958191 15950 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:03.958636 15557 consensus_queue.cc:1059] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:03.958806 15950 raft_consensus.cc:697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:03.960055 16198 raft_consensus.cc:3055] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:03.960713 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:03.961755 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:03.962838 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:03.960726 15950 consensus_queue.cc:237] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:03.873317 15919 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:03.960745 16198 raft_consensus.cc:740] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:03.966691 15919 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:03.966641 16198 consensus_queue.cc:260] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:04.005344 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.952607 16006 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:03.975598 16100 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:04.026634 13380 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:19:04.029786 16092 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:04.030809 16092 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:04.032367 15952 consensus_queue.cc:1048] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:04.034210 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:04.035231 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 72578.7us
I20260709 13:19:04.035387 16092 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:04.037721 16219 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.047765 16196 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:04.049002 16001 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:04.051422 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:04.056416 16103 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:04.057114 12774 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:19:04.058977 16115 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:04.061810 16205 log.cc:927] Time spent T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:19:04.073239 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:04.094059 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:03.968355 16168 consensus_queue.cc:260] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:04.099220 12701 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:19:04.080457 16100 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:19:04.105624 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.099959 12701 raft_consensus.cc:606] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:04.106186 12701 raft_consensus.cc:614] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:19:04.111559 15456 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:19:04.114109 13384 raft_consensus.cc:2410] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:19:04.114239 16115 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:04.117978 15946 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:04.134150 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:04.154501 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 62509us
I20260709 13:19:04.072683 16219 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:04.166018 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.167011 16103 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:04.167691 12629 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:19:04.167649 16103 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:04.169284 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:04.169945 13380 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:04.172662 16103 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:04.178692 12701 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:19:04.179412 12701 raft_consensus.cc:606] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:04.179859 12701 raft_consensus.cc:614] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:04.182219 16092 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.182911 16092 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:04.105229 16087 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:04.185895 16087 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:04.138177 16221 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:04.192840 16221 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:04.195284 16221 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:04.154779 16100 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:04.107795 16196 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:04.095399 16001 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:04.202744 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:04.203541 16219 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:04.204572 13383 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:19:04.204874 16170 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:04.214026 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:19:04.214229 16219 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:04.215823 16221 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:04.216506 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:04.219594 15886 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:04.220332 15886 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:04.222846 15886 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:04.220346 16032 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:04.224009 16032 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:04.157742 16168 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:04.230310 16198 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:04.231065 16196 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:04.185406 16092 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:04.237340 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:04.294764 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.233939 15953 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:04.322585 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 78965us
I20260709 13:19:04.257895 16115 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:04.216511 16221 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:04.335778 13380 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:04.337843 12701 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:19:04.343367 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:04.344010 12705 raft_consensus.cc:1240] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:19:04.344406 13379 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:19:04.345887 16226 log.cc:927] Time spent T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:19:04.350682 12629 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:19:04.354734 12700 raft_consensus.cc:1275] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:19:04.361313 16127 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.361954 16127 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:04.364302 16127 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:04.381100 16215 log.cc:927] Time spent T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.493s	user 0.003s	sys 0.005s
I20260709 13:19:04.336792 15878 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:04.380900 16217 log.cc:927] Time spent T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.488s	user 0.002s	sys 0.002s
I20260709 13:19:04.311416 16092 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:04.409196 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:04.413460 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:04.416213 12634 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
W20260709 13:19:04.416880 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:04.413666 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.382054 15878 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:04.403263 15931 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.435014 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:04.440810 12701 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:19:04.408070 16069 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.430435 12597 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:19:04.426539 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 348522us
I20260709 13:19:04.390971 15953 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:04.456842 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:04.411442 16115 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:04.456789 12707 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: []
W20260709 13:19:04.460704 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.466118 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:04.467024 12701 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:04.470734 16195 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:19:04.470916 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 286673us
W20260709 13:19:04.478603 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 48276.1us
I20260709 13:19:04.503412 12529 catalog_manager.cc:5697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 5 to 7, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 7 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:04.504798 12707 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:04.506915 15878 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:04.509017 16115 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:04.509904 16100 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:04.455219 15931 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:04.511305 16221 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:04.513209 15931 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:04.515134 16118 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.520535 16118 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:04.521782 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:04.522562 12709 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:04.523217 16118 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:04.523313 12630 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:19:04.515839 16032 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:04.444631 16092 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:04.526199 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:04.526713 16209 log.cc:927] Time spent T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:19:04.527485 16092 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:04.521829 12706 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:19:04.527871 16165 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.528527 16165 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:04.530580 16087 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:04.530686 16165 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:04.457681 16069 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:04.533229 16069 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:04.518332 16001 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:04.488803 16170 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.534318 16170 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:04.535411 12709 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:19:04.536692 16170 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:04.542106 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.544152 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:04.518855 12529 catalog_manager.cc:5697] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 4 to 6, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 6 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
W20260709 13:19:04.546702 16229 log.cc:927] Time spent T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 13:19:04.547780 16224 log.cc:927] Time spent T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:19:04.547932 12529 catalog_manager.cc:5697] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:04.550252 16236 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:04.550995 16236 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:04.551576 16236 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:04.553372 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 96686.8us
I20260709 13:19:04.553946 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:04.554224 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:04.559840 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:04.562292 12783 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:19:04.563863 16234 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:04.564504 16234 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:04.566954 16234 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:04.567615 16234 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:04.568295 15557 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:04.568111 16234 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:04.568892 15557 raft_consensus.cc:697] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:04.578166 13384 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:19:04.578595 13384 pending_rounds.cc:85] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: Aborting all ops after (but not including) 10
I20260709 13:19:04.578809 13384 pending_rounds.cc:107] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:19:04.579031 13384 raft_consensus.cc:2889] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:04.581746 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.596424 12709 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 11. Current term is 12. Ops: [8.9-11.11]
I20260709 13:19:04.599180 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:04.600186 13380 raft_consensus.cc:2368] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 11. Current term is 12.
I20260709 13:19:04.598397 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:04.605314 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:04.602082 12673 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:04.619024 12707 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:19:04.620561 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:04.620772 12780 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:19:04.614351 12628 pending_rounds.cc:85] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 5
I20260709 13:19:04.621970 12628 pending_rounds.cc:107] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:19:04.592232 16221 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:04.622318 12628 raft_consensus.cc:2889] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:04.592907 16010 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.625349 16234 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:04.629523 12673 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:04.646112 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:04.646696 15952 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:04.647022 12777 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:19:04.647185 15952 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.647641 15952 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:04.650012 16010 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:04.646291 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:04.550788 12529 catalog_manager.cc:5697] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 2 to 8, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 8 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:04.662456 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:04.668510 12701 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
W20260709 13:19:04.653723 13382 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.018s	sys 0.049s
W20260709 13:19:04.670014 13382 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.018s	sys 0.050s
I20260709 13:19:04.670456 16236 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:04.629900 16092 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:04.663230 12633 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:04.646337 16235 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.689585 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:04.690411 12627 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:19:04.695024 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:04.701550 16195 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:04.670444 16241 consensus_queue.cc:1059] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 13:19:04.702172 16195 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:04.680932 16092 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:04.769747 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:04.694175 15946 consensus_meta.cc:287] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.576s	user 0.012s	sys 0.000s
I20260709 13:19:04.773020 16168 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.702728 16092 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:04.701973 16010 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:04.809240 12627 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:04.789749 15946 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:04.844879 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 289330us
W20260709 13:19:04.850862 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 356601us
I20260709 13:19:04.788915 16108 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:04.806902 16219 raft_consensus.cc:3055] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:04.877084 12529 catalog_manager.cc:5697] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 4 to 6, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 6 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
W20260709 13:19:04.884397 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.908524 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:04.909968 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:04.911077 12774 raft_consensus.cc:2410] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:19:04.813318 15998 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:04.701627 16235 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:04.801293 16168 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:04.909250 16181 log.cc:927] Time spent T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:19:04.922597 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:04.774379 15557 consensus_queue.cc:237] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:04.884490 16219 raft_consensus.cc:740] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:04.897847 15950 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:04.926311 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.928150 12709 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:19:04.885298 16107 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.872515 15946 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:04.932045 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:04.935966 12747 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:04.944500 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 325735us
I20260709 13:19:04.926594 15931 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:04.946441 16196 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:04.928411 16198 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:04.945663 12529 catalog_manager.cc:5697] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 3 to 4, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 4 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:04.922400 16108 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:04.954946 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:04.946587 15966 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:19:04.964761 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.965163 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:04.966069 13380 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:04.933018 15950 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:04.948845 15931 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:04.937088 15998 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:04.957736 16237 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:04.959894 16236 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:04.975287 16156 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.954445 16196 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:04.980742 16109 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:19:04.981297 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:04.986312 16243 log.cc:927] Time spent T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:19:04.987115 15931 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:04.989897 12672 leader_election.cc:400] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 8. Current term is 9.
I20260709 13:19:05.044926 12672 leader_election.cc:403] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:19:04.990273 16238 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:19:04.990744 12746 leader_election.cc:400] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 11. Current term is 12.
I20260709 13:19:04.991144 13385 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
W20260709 13:19:04.997890 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 73459.5us
I20260709 13:19:05.046448 12746 leader_election.cc:403] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:04.999101 15950 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:05.003822 16235 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:04.979764 16168 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:05.007205 15998 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:04.955987 16107 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:04.971119 16252 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:04.985332 12777 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:04.973716 16198 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:05.020607 12707 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:19:05.009310 16236 raft_consensus.cc:697] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:05.022442 12776 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
W20260709 13:19:05.049903 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 86245.5us
I20260709 13:19:05.054479 15946 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:05.017535 12529 catalog_manager.cc:5697] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:05.064893 12701 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:19:05.066125 16087 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 11. Current term is 12.
I20260709 13:19:05.069886 16236 consensus_queue.cc:237] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:05.074215 12529 catalog_manager.cc:5697] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:05.075985 12529 catalog_manager.cc:5697] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 2 to 5, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 5 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
W20260709 13:19:05.078043 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 105209us
I20260709 13:19:05.077762 16252 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:05.063372 16235 consensus_queue.cc:1059] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:19:05.078392 12529 catalog_manager.cc:5697] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:05.079871 16234 raft_consensus.cc:3055] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:05.079973 16252 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:05.080314 16234 raft_consensus.cc:740] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:05.080410 12529 catalog_manager.cc:5697] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 1 to 7. New cstate: current_term: 7 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:05.082315 12781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1937) took 1319 ms. Trace:
I20260709 13:19:05.081471 16234 consensus_queue.cc:260] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:05.084229 16234 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:05.086024 12672 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:05.087388 16165 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:05.089025 16165 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:05.068779 12746 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:05.082692 12781 rpcz_store.cc:276] 0709 13:19:03.762782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.762933 (+   151us) service_pool.cc:224] Handling call
0709 13:19:03.784087 (+ 21154us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:04.088792 (+304705us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:19:04.088850 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:04.104740 (+ 15890us) log.cc:844] Serialized 26 byte log entry
0709 13:19:04.467871 (+363131us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:04.467911 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.468027 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:05.082086 (+614059us) raft_consensus.cc:1685] finished
0709 13:19:05.082105 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:05.082235 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18474,"raft.queue_time_us":77063,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":51869,"thread_start_us":631697,"threads_started":2,"wal-append.queue_time_us":687936}
I20260709 13:19:05.089964 12529 catalog_manager.cc:5697] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 4 to 6, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 6 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:05.093456 12702 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
W20260709 13:19:05.098668 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 171948us
I20260709 13:19:05.097889 12529 catalog_manager.cc:5697] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:05.062896 16168 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.021725 16237 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:05.102373 12776 raft_consensus.cc:1275] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:19:05.103228 16196 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:05.104045 16237 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:05.092444 15966 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:05.021693 16156 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:05.105777 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.098901 15878 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:19:05.109279 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:05.111626 16156 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:05.092195 15370 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 8. Current term is 9.
I20260709 13:19:05.113992 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:05.099274 16251 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:05.114924 12597 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:05.118685 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:05.119376 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.043784 16219 consensus_queue.cc:260] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:05.120656 12776 raft_consensus.cc:2393] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 12: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:05.122716 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:05.126901 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:05.127568 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:05.127722 16152 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 13:19:05.129120 16108 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:05.129658 16100 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:05.130362 16100 raft_consensus.cc:697] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:05.131305 16100 consensus_queue.cc:237] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:05.131800 16107 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:05.132707 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:05.133430 12774 raft_consensus.cc:2393] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 10: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:05.133608 15931 consensus_queue.cc:1059] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:05.134408 15931 raft_consensus.cc:3055] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:05.134761 15931 raft_consensus.cc:740] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:05.134929 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:05.135491 12775 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:19:05.135632 15931 consensus_queue.cc:260] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:05.136472 16219 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:05.136808 15931 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:05.127592 12777 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:05.134204 16018 consensus_queue.cc:1048] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:05.118039 16251 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:05.175922 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:05.177678 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:05.178450 12702 raft_consensus.cc:1813] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Handling vote request from an unknown peer 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:05.178941 12702 raft_consensus.cc:2393] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 5: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:05.180186 13348 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:05.121388 12529 catalog_manager.cc:5697] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 1 to 4, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 4 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
W20260709 13:19:05.142480 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.146934 16092 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:05.158219 16156 consensus_queue.cc:1059] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:05.163769 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:05.164230 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:05.165547 15998 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:05.166046 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:05.167793 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:05.107946 16168 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:05.138475 15557 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:05.204319 12627 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
W20260709 13:19:05.203331 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.174573 16251 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:05.215552 15919 raft_consensus.cc:3055] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:05.216046 15919 raft_consensus.cc:740] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:05.216876 15919 consensus_queue.cc:260] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:05.218212 15919 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:05.219259 15953 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:05.220935 16092 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:05.223148 16254 log.cc:927] Time spent T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.170s	user 0.002s	sys 0.000s
I20260709 13:19:05.225091 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:05.225781 16156 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:05.226310 16156 raft_consensus.cc:697] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:05.227245 16156 consensus_queue.cc:237] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:05.234886 13384 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:19:05.235565 12782 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:05.204994 16249 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:05.245541 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:05.234797 16261 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:19:05.246537 12706 raft_consensus.cc:2393] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:05.200790 16107 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:19:05.206990 15557 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.245630 16249 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:05.254478 16195 consensus_meta.cc:287] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.552s	user 0.011s	sys 0.000s
I20260709 13:19:05.255131 16195 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:05.256462 16249 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:05.257121 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:05.253377 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:05.265152 12672 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:05.265488 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:05.250912 15557 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:05.268237 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:05.266188 12782 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
W20260709 13:19:05.263114 15952 consensus_meta.cc:287] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.613s	user 0.010s	sys 0.003s
I20260709 13:19:05.268069 13348 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:05.257311 16195 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:05.302646 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:05.303519 12776 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:19:05.304580 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:05.307382 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:05.308431 12707 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:05.300848 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:19:05.309976 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 44911.1us
I20260709 13:19:05.312759 12529 catalog_manager.cc:5697] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:05.318253 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:05.273542 16198 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:05.302189 15952 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:05.309679 15878 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:05.319329 12776 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:05.326741 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:05.347708 12782 raft_consensus.cc:2393] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 11: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:05.348264 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:05.348579 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:05.329001 15952 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:05.337184 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:05.337922 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.336781 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:05.338068 16087 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.338176 16234 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:05.349396 12776 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
W20260709 13:19:05.406024 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 50027.2us
I20260709 13:19:05.409178 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:05.346655 16263 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:05.338575 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:05.339216 12529 catalog_manager.cc:5697] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 5 to 7, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 7 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:05.351342 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:05.349511 12782 raft_consensus.cc:2410] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:19:05.342691 12707 raft_consensus.cc:2368] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8.
W20260709 13:19:05.391253 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.418864 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:05.421399 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 68899.1us
I20260709 13:19:05.339902 12672 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:05.435365 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:05.436280 12776 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
W20260709 13:19:05.449976 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 110209us
I20260709 13:19:05.422887 12703 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:05.340087 16010 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.111s
I20260709 13:19:05.451467 12672 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:05.451781 15952 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:05.450073 12782 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:05.327029 16168 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:05.453754 12672 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:19:05.459800 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 244963us
I20260709 13:19:05.463784 12747 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:05.474471 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:05.473613 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 260422us
I20260709 13:19:05.483796 12627 raft_consensus.cc:2410] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:19:05.353961 16001 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:05.485024 12597 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:05.483191 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:05.490037 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 228925us
I20260709 13:19:05.493288 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:05.417204 16195 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:05.565989 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 112296us
I20260709 13:19:05.586817 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:05.589295 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 104373us
I20260709 13:19:05.590147 12597 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:05.602046 16127 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:05.658432 16165 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.659118 16270 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:05.659036 16165 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:05.659650 16270 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:05.660072 16270 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:05.660722 16257 log.cc:927] Time spent T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.608s	user 0.001s	sys 0.013s
I20260709 13:19:05.662237 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:05.662798 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.663555 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:05.665902 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:05.666536 13380 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:05.666610 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:05.667181 16198 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:05.668002 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:05.667815 16198 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:05.668874 12625 raft_consensus.cc:2393] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 10: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:05.494053 16196 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:05.669951 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:05.671487 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:05.673004 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:05.673964 12626 raft_consensus.cc:2393] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 7: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:05.664770 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:05.680126 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:05.680528 16265 log.cc:927] Time spent T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.493s	user 0.002s	sys 0.014s
I20260709 13:19:05.681648 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:05.682823 16231 log.cc:927] Time spent T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 1.033s	user 0.000s	sys 0.025s
I20260709 13:19:05.683243 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:05.684155 12626 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 9: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:19:05.686028 16255 log.cc:927] Time spent T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.697s	user 0.003s	sys 0.015s
W20260709 13:19:05.687573 16258 log.cc:927] Time spent T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.606s	user 0.012s	sys 0.003s
I20260709 13:19:05.688973 12628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2464) took 1092 ms. Trace:
I20260709 13:19:05.689337 12628 rpcz_store.cc:276] 0709 13:19:04.597025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.597312 (+   287us) service_pool.cc:224] Handling call
0709 13:19:04.614133 (+ 16821us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:04.664066 (+ 49933us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:04.664099 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:04.672648 (+  8549us) log.cc:844] Serialized 26 byte log entry
0709 13:19:04.951608 (+278960us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:04.951836 (+   228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.951914 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:05.686821 (+734907us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:05.688633 (+  1812us) raft_consensus.cc:1685] finished
0709 13:19:05.688656 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:05.688892 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28312960,"thread_start_us":251233,"threads_started":1,"wal-append.queue_time_us":392529}
I20260709 13:19:05.690253 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:05.691305 15919 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:05.692015 15919 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:05.694228 16250 log.cc:927] Time spent T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.706s	user 0.017s	sys 0.009s
I20260709 13:19:05.695323 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:05.696230 12700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1858) took 1337 ms. Trace:
I20260709 13:19:05.686077 15946 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.696257 12634 raft_consensus.cc:2410] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:19:05.697585 15946 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:05.697880 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:05.699765 15946 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:05.700343 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:05.700649 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:05.701097 12775 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:19:05.696655 12700 rpcz_store.cc:276] 0709 13:19:04.358602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.387586 (+ 28984us) service_pool.cc:224] Handling call
0709 13:19:04.407445 (+ 19859us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:04.482885 (+ 75440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:04.482919 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:04.498494 (+ 15575us) log.cc:844] Serialized 26 byte log entry
0709 13:19:04.934372 (+435878us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:04.948786 (+ 14414us) log.cc:844] Serialized 14 byte log entry
0709 13:19:04.949016 (+   230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.949095 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:05.231706 (+282611us) spinlock_profiling.cc:243] Waited 2.11 ms on lock 0x7b18002c04a0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c53f3 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:19:05.693171 (+461465us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:05.695999 (+  2828us) raft_consensus.cc:1685] finished
0709 13:19:05.696020 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:05.696142 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12990,"prepare.queue_time_us":14697,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":53,"raft.queue_time_us":17246,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":15182,"spinlock_wait_cycles":237787392,"thread_start_us":423821,"threads_started":1,"wal-append.queue_time_us":477865}
I20260709 13:19:05.610898 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:05.654789 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 161585us
I20260709 13:19:05.704861 12627 raft_consensus.cc:1813] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Handling vote request from an unknown peer 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:05.705549 12627 raft_consensus.cc:2410] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:19:05.705710 12747 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:05.706297 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.715951 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:05.716609 12781 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
W20260709 13:19:05.716725 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 126659us
I20260709 13:19:05.716979 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:19:05.717612 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 220328us
I20260709 13:19:05.720304 15950 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:05.694725 15919 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:05.720753 15950 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:05.721276 15950 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:05.722914 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:05.723657 16165 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:05.726250 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:05.726833 12634 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:05.728524 16219 consensus_meta.cc:287] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.592s	user 0.007s	sys 0.000s
I20260709 13:19:05.729480 16092 raft_consensus.cc:993] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530: : Instructing follower 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:19:05.729945 16092 raft_consensus.cc:1081] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Signalling peer 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:19:05.732028 12597 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:05.422626 16234 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:05.735059 12634 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:05.736560 12627 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:05.420359 16087 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:05.735627 13349 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:05.610311 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:05.748287 12630 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:19:05.463480 16109 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:05.754132 12631 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:19:05.760531 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 193486us
I20260709 13:19:05.465268 16263 raft_consensus.cc:697] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:05.756927 12776 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:05.725281 15557 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:05.738121 15946 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:05.740720 16219 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.588557 16235 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:05.752861 16251 consensus_meta.cc:287] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.541s	user 0.011s	sys 0.000s
I20260709 13:19:05.750635 15919 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:05.754277 16252 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:05.777419 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:05.773313 12672 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:05.789083 13380 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:05.794275 12633 raft_consensus.cc:2393] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 11: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:05.797873 15370 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:05.779347 16261 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:05.798287 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:05.798946 16261 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.799499 12778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1944) took 1773 ms. Trace:
I20260709 13:19:05.798364 15370 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.799389 16261 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:05.604377 16195 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:05.603359 16001 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:05.765413 16092 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:19:05.779707 16069 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.770545 15952 consensus_queue.cc:1048] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:19:05.831893 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:05.836757 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:05.779837 15946 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:05.799953 12778 rpcz_store.cc:276] 0709 13:19:04.026271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.026929 (+   658us) service_pool.cc:224] Handling call
0709 13:19:04.073291 (+ 46362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:04.410588 (+337297us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:19:04.818550 (+407962us) log.cc:844] Serialized 14 byte log entry
0709 13:19:05.013182 (+194632us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:05.054962 (+ 41780us) log.cc:844] Serialized 26 byte log entry
0709 13:19:05.055132 (+   170us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:05.055174 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:05.799111 (+743937us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54001fd954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:05.799160 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:05.799206 (+    46us) raft_consensus.cc:1685] finished
0709 13:19:05.799300 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:05.799441 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":56884,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":26458,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":5212,"spinlock_wait_cycles":302853376,"thread_start_us":721333,"threads_started":3,"wal-append.queue_time_us":275422}
I20260709 13:19:05.805958 15370 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:05.846802 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:05.784476 16108 consensus_queue.cc:1059] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:19:05.783186 16219 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:05.842206 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:05.784348 16251 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:05.850873 12702 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.027s	sys 0.038s
I20260709 13:19:05.855746 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:05.855870 13379 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:19:05.862511 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:05.845288 16198 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:05.872151 12702 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.027s	sys 0.038s
W20260709 13:19:05.864833 13349 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 2.019s (SENT) [suppressed 14 similar messages]
I20260709 13:19:05.860445 15998 raft_consensus.cc:3055] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:05.841722 16069 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:05.856725 16010 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:05.893808 15998 raft_consensus.cc:740] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:05.885433 16103 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:19:05.923007 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 212261us
I20260709 13:19:05.926146 12631 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
W20260709 13:19:05.930356 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 136616us
W20260709 13:19:05.933821 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 127440us
I20260709 13:19:05.932057 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:05.935096 12672 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:05.939155 12701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 420) took 1122 ms. Trace:
I20260709 13:19:05.942576 12597 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:05.939888 12701 rpcz_store.cc:276] 0709 13:19:04.816989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.817162 (+   173us) service_pool.cc:224] Handling call
0709 13:19:05.062455 (+245293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:05.065357 (+  2902us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:05.938945 (+873588us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b54001d8150. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:05.939096 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":563152768}
I20260709 13:19:05.947603 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:05.947953 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:05.948589 12707 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:05.949738 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:05.950708 12707 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:05.948768 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:05.954524 12706 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:05.910339 16010 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:05.960069 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:05.960847 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:05.961174 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:05.922284 16234 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:05.959470 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:05.958134 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:05.962661 12701 raft_consensus.cc:2393] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 11: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:05.964363 12700 raft_consensus.cc:2393] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 8: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:05.943545 15952 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:05.954437 16198 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:05.969954 16198 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:05.971189 16103 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:05.971722 16103 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:05.975240 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:05.975843 13384 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
W20260709 13:19:05.976034 13385 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51762 (request call id 615) took 2077 ms (client timeout 2000 ms). Trace:
W20260709 13:19:05.976464 13385 rpcz_store.cc:269] 0709 13:19:03.898421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.926716 (+ 28295us) service_pool.cc:224] Handling call
0709 13:19:05.975807 (+2049091us) spinlock_profiling.cc:243] Waited 699 ms on lock 0x7b54000596d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:05.975900 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1608560000}
I20260709 13:19:05.965023 16107 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:19:05.945986 15592 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:05.981424 16248 log.cc:927] Time spent T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.988s	user 0.010s	sys 0.010s
I20260709 13:19:05.981909 15592 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:05.983381 12632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2456) took 1591 ms. Trace:
I20260709 13:19:05.969074 15950 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:05.856524 16263 consensus_queue.cc:237] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:05.988559 16259 log.cc:927] Time spent T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.988s	user 0.009s	sys 0.026s
W20260709 13:19:05.994530 16274 log.cc:927] Time spent T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 13:19:05.996565 12704 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:19:05.945700 16165 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:05.983794 12632 rpcz_store.cc:276] 0709 13:19:04.392183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.392353 (+   170us) service_pool.cc:224] Handling call
0709 13:19:04.392775 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:04.423063 (+ 30288us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:04.423095 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:04.444437 (+ 21342us) log.cc:844] Serialized 26 byte log entry
0709 13:19:04.848556 (+404119us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:04.848611 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.848700 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:05.980754 (+1132054us) spinlock_profiling.cc:243] Waited 90.2 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:05.983064 (+  2310us) raft_consensus.cc:1685] finished
0709 13:19:05.983090 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:05.983317 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14739,"raft.queue_time_us":26031,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":5226,"spinlock_wait_cycles":207503232,"thread_start_us":372815,"threads_started":1,"wal-append.queue_time_us":512999}
I20260709 13:19:05.957366 16249 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:05.995761 15952 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:06.003284 16165 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:06.002126 15557 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:06.009402 16219 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:06.030925 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:06.015049 16249 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:06.012581 16001 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:06.041801 12783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2280) took 1478 ms. Trace:
I20260709 13:19:06.030249 16165 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:06.029793 15952 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:06.050185 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 95269.5us
I20260709 13:19:06.028273 15878 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:06.050271 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 344617us
I20260709 13:19:06.030485 15946 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:06.041366 16249 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:06.046017 12783 rpcz_store.cc:276] 0709 13:19:04.563328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.584648 (+ 21320us) service_pool.cc:224] Handling call
0709 13:19:04.597446 (+ 12798us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:04.672838 (+ 75392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:04.672889 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:04.684200 (+ 11311us) log.cc:844] Serialized 26 byte log entry
0709 13:19:04.955621 (+271421us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:04.970479 (+ 14858us) log.cc:844] Serialized 14 byte log entry
0709 13:19:04.970963 (+   484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.971143 (+   180us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:05.059115 (+ 87972us) spinlock_profiling.cc:243] Waited 4.19 ms on lock 0x7b18003c7720. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c53f3 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:19:05.987239 (+928124us) spinlock_profiling.cc:243] Waited 6.96 ms on lock 0x7b18003c7720. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2932a6 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db
0709 13:19:05.987774 (+   535us) spinlock_profiling.cc:243] Waited 83.4 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:06.041473 (+ 53699us) raft_consensus.cc:1685] finished
0709 13:19:06.041509 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:06.041723 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16453,"prepare.queue_time_us":16236,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":38,"raft.queue_time_us":35069,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":27703,"spinlock_wait_cycles":217390592,"thread_start_us":179737,"threads_started":1,"wal-append.queue_time_us":301465}
I20260709 13:19:06.071257 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:06.074357 16244 log.cc:927] Time spent T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 1.143s	user 0.008s	sys 0.008s
I20260709 13:19:06.078166 12705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1857) took 1732 ms. Trace:
I20260709 13:19:06.078702 12705 rpcz_store.cc:276] 0709 13:19:04.345430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.358852 (+ 13422us) service_pool.cc:224] Handling call
0709 13:19:04.359189 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:04.359547 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:04.359585 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:04.360095 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 13:19:04.622044 (+261949us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:04.622084 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.622151 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:06.073549 (+1451398us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:06.077747 (+  4198us) raft_consensus.cc:1685] finished
0709 13:19:06.077786 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:06.078075 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1363,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":79,"spinlock_wait_cycles":344824704,"thread_start_us":261397,"threads_started":1,"wal-append.queue_time_us":482803}
I20260709 13:19:06.081278 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:06.086794 16270 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:06.088289 12625 raft_consensus.cc:2393] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 9: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:06.090500 16010 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:06.055675 16251 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:06.096241 12673 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:06.098892 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:06.100544 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:06.117486 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:06.118232 12709 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:06.101593 12776 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:06.118590 12598 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:06.102137 15878 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:06.104537 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:06.119562 12598 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:06.119832 12782 raft_consensus.cc:2393] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 10: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:06.105521 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:06.120435 12783 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 8
W20260709 13:19:06.106124 16253 log.cc:927] Time spent T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 1.116s	user 0.012s	sys 0.008s
I20260709 13:19:06.054188 16290 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:06.108564 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:06.128245 12705 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:06.110010 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:06.114240 16152 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:06.113706 16103 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:06.115537 16087 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:06.069162 16195 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:06.115969 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:06.141682 12702 raft_consensus.cc:2376] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11. Re-sending same reply.
I20260709 13:19:06.104285 15592 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:06.089473 16270 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:06.125537 12673 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:06.129462 12703 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:06.136458 12598 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:06.152451 12598 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:06.138456 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 76414.6us
I20260709 13:19:06.157393 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:06.158190 12747 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:06.161403 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.160790 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:06.162492 12672 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:06.163276 16284 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:19:06.140413 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 84858.4us
I20260709 13:19:06.143399 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:06.139462 15557 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:06.171212 12747 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:06.172927 12747 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:06.180600 12747 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:06.181828 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:06.171703 13349 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:06.184034 16263 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:06.178333 16195 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:06.184453 16263 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:06.184899 16263 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:06.140168 16109 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:06.188794 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:06.192787 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:06.193856 12705 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:06.149798 16271 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:06.198397 12774 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:19:06.166812 15592 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:06.155679 16270 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:06.167721 15946 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:06.178736 15557 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:06.211485 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 480104us
I20260709 13:19:06.180075 16251 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:06.186324 16001 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:06.186546 16219 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:06.217151 12746 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:06.187157 16069 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:06.194361 16109 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:06.214371 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:06.192539 16087 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:06.185348 12782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:36478
I20260709 13:19:06.237015 12597 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:06.150352 16152 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:06.150100 16290 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:06.274127 12782 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:06.278185 12782 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:06.311184 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:06.311614 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:06.279088 12779 raft_consensus.cc:2393] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 12: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:06.312441 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:06.314828 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:06.318883 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:06.322621 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:06.313295 16069 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:06.342379 13383 raft_consensus.cc:2393] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:06.323761 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:06.324882 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:06.216534 15946 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:06.242385 16219 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:06.282243 16236 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.209s	user 0.014s	sys 0.037s
W20260709 13:19:06.377971 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 194715us
I20260709 13:19:06.369877 13348 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:06.273391 16100 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:06.185407 16280 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:19:06.190004 15998 consensus_queue.cc:260] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:06.337882 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:06.391098 12706 raft_consensus.cc:2393] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:06.311585 16109 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:06.278127 16235 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:06.339608 12705 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:06.366525 15946 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:06.342280 12630 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:06.341434 13377 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:06.347697 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:06.347995 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:06.407989 12702 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:06.349974 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:06.366477 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:06.370446 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:06.367264 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:06.376777 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:06.376832 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:06.243983 15557 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:06.336313 16241 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.381106 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:06.381364 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:06.331881 16271 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:06.382375 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:06.339166 16270 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:06.332453 16251 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:06.384100 12673 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:06.315676 15370 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:06.399931 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:06.344148 12597 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:19:06.459188 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 115114us
W20260709 13:19:06.377967 16236 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.305s	user 0.014s	sys 0.037s
I20260709 13:19:06.345594 16069 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:06.352065 15456 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:06.377596 16219 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:06.408517 12709 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:06.339344 16165 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:06.378116 15592 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:06.400586 16261 consensus_meta.cc:287] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.588s	user 0.008s	sys 0.003s
I20260709 13:19:06.384502 16100 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:06.442694 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:06.485100 13378 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:19:06.388427 16087 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:06.440353 16235 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:06.405186 16115 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:06.448436 16297 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:19:06.448204 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:06.496569 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 126863us
W20260709 13:19:06.478243 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 308908us
I20260709 13:19:06.444888 15998 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:06.490260 12704 raft_consensus.cc:2368] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10.
I20260709 13:19:06.445533 16252 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:06.493201 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:06.495008 12747 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:06.470934 16103 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:06.472659 15812 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:06.489205 16100 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:06.463429 16069 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:06.458343 16241 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:06.551872 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:06.685261 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 187177us
I20260709 13:19:06.498842 16195 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:06.471338 16270 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:06.477638 16251 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:06.478446 16249 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:06.476768 15456 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:06.486014 16261 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:06.975008 12782 consensus_meta.cc:287] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.658s	user 0.009s	sys 0.000s
W20260709 13:19:06.971685 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 474281us
I20260709 13:19:06.824079 13385 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
W20260709 13:19:06.981819 12783 consensus_meta.cc:287] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.848s	user 0.008s	sys 0.002s
W20260709 13:19:07.000998 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 506023us
I20260709 13:19:06.464681 16271 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.011875 12783 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:06.510493 16310 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:19:07.012928 13349 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.003s	sys 0.008s
W20260709 13:19:07.026321 13349 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.003s	sys 0.009s
W20260709 13:19:07.026676 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 583872us
I20260709 13:19:07.012135 12747 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:07.008798 12782 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:06.552630 15812 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:06.685909 16236 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:06.552937 15878 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:07.049419 12782 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:06.785207 16269 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:07.059310 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:06.959949 16152 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:06.805168 16290 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:06.807783 16313 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:07.065037 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 52978.7us
I20260709 13:19:07.005434 15966 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:06.506197 16115 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:06.505307 16087 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:07.026213 16310 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:06.520103 15952 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:06.515596 16252 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.076663 12747 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:07.078645 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:06.686012 16286 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.521s
I20260709 13:19:06.520177 15592 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.018199 16316 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.100s
I20260709 13:19:07.131454 12700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1875) took 1103 ms. Trace:
I20260709 13:19:07.064010 15812 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:07.125860 12783 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:19:07.125874 12777 raft_consensus.cc:2410] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:19:07.025866 16195 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:07.071322 16127 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.017s	sys 0.039s
W20260709 13:19:07.123399 12673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.015s	sys 0.044s
I20260709 13:19:07.088215 16281 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:07.118914 16290 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:19:07.137439 12673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.015s	sys 0.045s
I20260709 13:19:07.118299 16156 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:07.138516 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 754560us
I20260709 13:19:07.134074 12700 rpcz_store.cc:276] 0709 13:19:06.028206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.028353 (+   147us) service_pool.cc:224] Handling call
0709 13:19:06.039823 (+ 11470us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:06.059649 (+ 19826us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.059696 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:06.101538 (+ 41842us) log.cc:844] Serialized 52 byte log entry
0709 13:19:06.206354 (+104816us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:06.250344 (+ 43990us) log.cc:844] Serialized 14 byte log entry
0709 13:19:06.393318 (+142974us) log.cc:844] Serialized 14 byte log entry
0709 13:19:06.393551 (+   233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.393685 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.131225 (+737540us) raft_consensus.cc:1685] finished
0709 13:19:07.131247 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.131396 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":399057,"prepare.queue_time_us":73195,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":12444,"raft.queue_time_us":530185,"raft.run_cpu_time_us":569,"raft.run_wall_time_us":154391,"thread_start_us":165499,"threads_started":2,"wal-append.queue_time_us":225775}
I20260709 13:19:07.139499 12673 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:07.124696 15370 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:07.145722 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 69569.5us
I20260709 13:19:07.147874 12747 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:07.139613 16127 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.019s	sys 0.041s
I20260709 13:19:07.150288 16236 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:07.154013 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:07.136914 16165 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:07.149386 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:07.197877 12673 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:07.200822 12673 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:07.141018 16270 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:07.123752 16310 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:07.123637 15966 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:07.085806 16269 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:07.196687 16236 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:07.238533 16308 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:19:07.239432 16304 log.cc:927] Time spent T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.637s	user 0.001s	sys 0.001s
I20260709 13:19:07.239977 16324 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:07.226451 16285 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:07.241974 12707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 1873) took 1239 ms. Trace:
I20260709 13:19:07.242456 12707 rpcz_store.cc:276] 0709 13:19:06.002614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.002744 (+   130us) service_pool.cc:224] Handling call
0709 13:19:06.003070 (+   326us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:06.023396 (+ 20326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.023427 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:06.027658 (+  4231us) log.cc:844] Serialized 52 byte log entry
0709 13:19:06.361705 (+334047us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:06.361765 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.361874 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.241649 (+879775us) raft_consensus.cc:1685] finished
0709 13:19:07.241680 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.241900 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16853,"raft.queue_time_us":23753,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":2511,"thread_start_us":306790,"threads_started":1,"wal-append.queue_time_us":1179459}
W20260709 13:19:07.246057 16312 log.cc:927] Time spent T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.327s	user 0.000s	sys 0.001s
I20260709 13:19:07.247485 12780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2307) took 1055 ms. Trace:
W20260709 13:19:07.247689 16301 log.cc:927] Time spent T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 13:19:07.247856 12780 rpcz_store.cc:276] 0709 13:19:06.191612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.191756 (+   144us) service_pool.cc:224] Handling call
0709 13:19:06.209701 (+ 17945us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:06.274324 (+ 64623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.274352 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:06.334102 (+ 59750us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.424126 (+ 90024us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:06.424187 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.424289 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.247191 (+822902us) raft_consensus.cc:1685] finished
0709 13:19:07.247215 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.247417 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7914,"spinlock_wait_cycles":14094720,"thread_start_us":41571,"threads_started":1,"wal-append.queue_time_us":339708}
W20260709 13:19:07.249140 16299 log.cc:927] Time spent T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:19:07.251564 15952 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:07.262108 16241 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:07.262970 16241 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:07.263130 16195 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:07.207311 16235 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:07.263913 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 49772.6us
I20260709 13:19:07.264138 16165 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:07.264791 12673 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:07.265626 16165 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:07.140059 16281 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.272861 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:07.274923 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:07.273734 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:07.275760 12633 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:07.276022 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:07.283255 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:07.214977 15370 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:07.147720 16156 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:07.298821 12778 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.889s	user 0.014s	sys 0.034s
W20260709 13:19:07.299222 12778 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.014s	sys 0.034s
W20260709 13:19:07.339048 12597 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.871s	user 0.026s	sys 0.038s
W20260709 13:19:07.348599 12597 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.880s	user 0.026s	sys 0.038s
W20260709 13:19:07.340014 16108 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.009s	sys 0.009s
W20260709 13:19:07.348941 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 880815us
I20260709 13:19:07.276034 16236 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.349598 12597 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:07.364102 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:07.365970 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:07.367910 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:07.319736 16285 raft_consensus.cc:697] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:07.334656 16317 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:07.389709 12707 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20db0f202ea546699789c2325d241b35"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:19:07.338444 16249 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:07.393122 12707 raft_consensus.cc:606] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:07.341110 15370 raft_consensus.cc:697] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:07.397936 12634 raft_consensus.cc:1240] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:19:07.348775 16261 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:07.358628 16270 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:07.411006 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:07.387854 16152 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:07.387069 15950 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:07.425141 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 275822us
I20260709 13:19:07.362088 16269 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:07.386654 16108 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.835s	user 0.009s	sys 0.010s
W20260709 13:19:07.426982 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 52834.7us
I20260709 13:19:07.438836 12747 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:07.439078 13348 leader_election.cc:336] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.556s (SENT)
I20260709 13:19:07.439535 13348 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:07.409777 15456 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:07.415354 16168 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:07.418057 16087 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:07.457674 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 46738us
I20260709 13:19:07.443388 16010 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.466328 12707 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:19:07.467160 12707 raft_consensus.cc:606] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:07.463421 13349 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:07.465210 15919 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:07.425819 16152 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:07.461683 16315 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:07.426739 16316 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:07.442379 15878 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.443678 15952 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.477231 16315 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:07.481281 15952 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.483335 16315 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:07.489722 15370 consensus_queue.cc:237] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.436761 16270 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.494011 16168 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:07.494349 16298 log.cc:927] Time spent T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:19:07.495251 16270 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:07.474670 12629 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:07.470645 16103 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:19:07.478137 16219 consensus_meta.cc:287] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.986s	user 0.008s	sys 0.004s
W20260709 13:19:07.496301 15998 consensus_meta.cc:287] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.991s	user 0.014s	sys 0.000s
I20260709 13:19:07.470711 16152 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:07.475775 12781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2303) took 1366 ms. Trace:
I20260709 13:19:07.497519 12781 rpcz_store.cc:276] 0709 13:19:06.109539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.111216 (+  1677us) service_pool.cc:224] Handling call
0709 13:19:06.111519 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:06.333229 (+221710us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.333273 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:06.333330 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:06.333374 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.475488 (+1142114us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b540044cad4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:07.475551 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.475700 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":509298560}
I20260709 13:19:07.499440 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:07.500002 12781 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:07.480633 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:07.502024 13385 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:07.501727 16285 consensus_queue.cc:237] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.504133 12626 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:07.506399 15878 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.462796 16311 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:19:07.509049 16001 consensus_meta.cc:287] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 1.259s	user 0.011s	sys 0.002s
I20260709 13:19:07.509505 16001 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:07.511600 12778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2301) took 1404 ms. Trace:
I20260709 13:19:07.512032 12778 rpcz_store.cc:276] 0709 13:19:06.107282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.108056 (+   774us) service_pool.cc:224] Handling call
0709 13:19:06.108427 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:06.108778 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.108810 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:06.109304 (+   494us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.362122 (+252818us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:07.316324 (+954202us) log.cc:844] Serialized 14 byte log entry
0709 13:19:07.316711 (+   387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.316807 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.511384 (+194577us) raft_consensus.cc:1685] finished
0709 13:19:07.511405 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.511542 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1061132,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":2436,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":13884,"thread_start_us":1147468,"threads_started":2,"wal-append.queue_time_us":1236228}
I20260709 13:19:07.468952 15592 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:07.514197 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:19:07.516139 16109 consensus_meta.cc:287] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 1.098s	user 0.012s	sys 0.000s
I20260709 13:19:07.516682 16109 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.408316 16317 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:07.519131 13385 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:07.520038 16251 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:07.520226 16317 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:07.520961 16317 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:07.455874 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:07.481189 16281 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:07.523113 12625 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56838 (request call id 1455) took 1818 ms (client timeout 1941 ms). Trace:
I20260709 13:19:07.520274 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:07.459007 16321 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.525288 12781 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:07.542968 12633 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:07.543665 12781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36488 (request call id 1975) took 1094 ms. Trace:
I20260709 13:19:07.494817 16168 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:07.522465 12777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54346 (request call id 444) took 1308 ms. Trace:
W20260709 13:19:07.522912 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 67175.4us
W20260709 13:19:07.528645 12625 rpcz_store.cc:269] 0709 13:19:05.656444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.689929 (+ 33485us) service_pool.cc:224] Handling call
0709 13:19:07.474202 (+1784273us) spinlock_profiling.cc:243] Waited 397 ms on lock 0x7b54003d8954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:07.474355 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":913317120}
W20260709 13:19:07.548074 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 149916us
I20260709 13:19:07.548276 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:07.544387 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 122100us
I20260709 13:19:07.461001 16306 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.547768 12777 rpcz_store.cc:276] 0709 13:19:06.213803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.551459 (+337656us) service_pool.cc:224] Handling call
0709 13:19:07.522288 (+970829us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b540011ce54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:07.522412 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":332328832}
I20260709 13:19:07.544121 12781 rpcz_store.cc:276] 0709 13:19:06.449239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.499121 (+1049882us) service_pool.cc:224] Handling call
0709 13:19:07.543600 (+ 44479us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:19:07.529629 15946 consensus_meta.cc:287] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 1.078s	user 0.015s	sys 0.000s
I20260709 13:19:07.550222 12673 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:07.478276 16010 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:07.552840 12707 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:19:07.523478 16280 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.555431 12707 raft_consensus.cc:606] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:07.556197 12672 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:07.521813 16252 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:07.554586 12706 raft_consensus.cc:1240] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:07.554402 12747 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:07.556160 13377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51762 (request call id 628) took 1115 ms. Trace:
I20260709 13:19:07.559725 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:07.559842 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:07.496356 16219 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.560420 12705 raft_consensus.cc:2376] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9. Re-sending same reply.
I20260709 13:19:07.521539 16317 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:07.523624 16316 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.557363 16315 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:07.562989 12747 leader_election.cc:400] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8.
I20260709 13:19:07.564761 12777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 448) took 1331 ms. Trace:
I20260709 13:19:07.578238 12747 leader_election.cc:403] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:07.578300 12777 rpcz_store.cc:276] 0709 13:19:06.233182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.556854 (+1323672us) service_pool.cc:224] Handling call
0709 13:19:07.564257 (+  7403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:07.564445 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:07.564470 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:07.564513 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:07.564549 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.564604 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.564706 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:07.560999 12672 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:07.565356 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:07.568466 12778 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:07.579878 12706 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:07.563541 16285 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:07.575829 12781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 443) took 1369 ms. Trace:
I20260709 13:19:07.564185 12705 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:07.559237 13377 rpcz_store.cc:276] 0709 13:19:06.441286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.450215 (+  8929us) service_pool.cc:224] Handling call
0709 13:19:06.450564 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:06.800408 (+349844us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.800443 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:07.040761 (+240318us) log.cc:844] Serialized 26 byte log entry
0709 13:19:07.337739 (+296978us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:07.465887 (+128148us) log.cc:844] Serialized 14 byte log entry
0709 13:19:07.555854 (+ 89967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.555927 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.555969 (+    42us) raft_consensus.cc:1685] finished
0709 13:19:07.555981 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.556102 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2967,"prepare.queue_time_us":217448,"raft.queue_time_us":80679,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":151432,"thread_start_us":694495,"threads_started":4,"wal-append.queue_time_us":347735}
I20260709 13:19:07.583588 16330 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:07.558542 16280 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:07.564734 16263 consensus_meta.cc:287] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 1.371s	user 0.010s	sys 0.003s
I20260709 13:19:07.597934 16263 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:07.599646 13384 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 10: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:07.600980 16263 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:07.564778 16252 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:07.589164 12706 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
W20260709 13:19:07.565143 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 101778us
W20260709 13:19:07.590251 16100 consensus_meta.cc:287] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.804s	user 0.012s	sys 0.000s
I20260709 13:19:07.590628 16317 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:07.591302 16109 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:07.556160 15946 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:07.592543 16219 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:07.556491 16306 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.615538 16281 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.592711 12747 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:07.594657 16010 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:07.595839 16284 consensus_queue.cc:1059] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:07.595721 16152 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.597157 12777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 449) took 1363 ms. Trace:
I20260709 13:19:07.625097 12777 rpcz_store.cc:276] 0709 13:19:06.233418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.579379 (+1345961us) service_pool.cc:224] Handling call
0709 13:19:07.596657 (+ 17278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:07.596840 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:07.596868 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:07.596911 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:07.596942 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.596999 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.597099 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:07.580051 12700 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:07.613799 16156 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:07.602011 13348 leader_election.cc:336] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.513s (SENT)
W20260709 13:19:07.577116 15557 consensus_meta.cc:287] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 1.133s	user 0.011s	sys 0.003s
I20260709 13:19:07.602025 16330 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8.
I20260709 13:19:07.547681 16321 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:07.605968 16168 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:07.606107 16325 log.cc:927] Time spent T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 13:19:07.607774 16316 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:07.608170 12776 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:07.610235 16310 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:07.611706 15812 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.612511 16234 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:07.631597 12673 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:07.589679 12781 rpcz_store.cc:276] 0709 13:19:06.206378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.556747 (+1350369us) service_pool.cc:224] Handling call
0709 13:19:07.562400 (+  5653us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:07.575317 (+ 12917us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:07.575358 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:07.575428 (+    70us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:07.575483 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.575579 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.575751 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:07.569410 15966 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:07.620541 12747 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:07.591324 16318 log.cc:927] Time spent T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:19:07.639489 16316 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:07.639573 16234 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.640250 16316 raft_consensus.cc:697] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:07.641211 16316 consensus_queue.cc:237] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.642679 13380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 653) took 1366 ms. Trace:
I20260709 13:19:07.643141 13380 rpcz_store.cc:276] 0709 13:19:06.276265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.276390 (+   125us) service_pool.cc:224] Handling call
0709 13:19:06.326174 (+ 49784us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:06.391789 (+ 65615us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.391818 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:06.419693 (+ 27875us) log.cc:844] Serialized 52 byte log entry
0709 13:19:06.496228 (+ 76535us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:06.496267 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.496354 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.642426 (+1146072us) raft_consensus.cc:1685] finished
0709 13:19:07.642448 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.642611 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4284,"raft.queue_time_us":99819,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":10905,"thread_start_us":74362,"threads_started":2,"wal-append.queue_time_us":948502}
I20260709 13:19:07.646039 16234 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:07.646235 13376 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:19:07.652323 16330 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:07.652820 16330 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:07.630410 15592 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:07.580178 12778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36488 (request call id 1973) took 1160 ms. Trace:
I20260709 13:19:07.675266 12778 rpcz_store.cc:276] 0709 13:19:06.419350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.513893 (+1094543us) service_pool.cc:224] Handling call
0709 13:19:07.580116 (+ 66223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:07.589166 16335 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:07.628156 12704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1874) took 1602 ms (client timeout 1915 ms). Trace:
I20260709 13:19:07.624646 15931 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:19:07.677085 12704 rpcz_store.cc:269] 0709 13:19:06.023070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.023538 (+   468us) service_pool.cc:224] Handling call
0709 13:19:07.625579 (+1602041us) spinlock_profiling.cc:243] Waited 456 ms on lock 0x7b54002481d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:07.625715 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1049362688}
I20260709 13:19:07.627682 13348 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:07.618263 16285 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:19:07.628036 16069 consensus_meta.cc:287] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.843s	user 0.013s	sys 0.000s
I20260709 13:19:07.685083 16290 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:07.685446 16069 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:07.685663 16290 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.685830 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:07.687023 12780 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 5.
I20260709 13:19:07.622562 16237 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:07.633131 16198 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:07.627132 16109 raft_consensus.cc:3055] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:19:07.634526 16275 log.cc:927] Time spent T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 1.625s	user 0.007s	sys 0.031s
I20260709 13:19:07.657650 15946 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:07.635578 12776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 450) took 1393 ms. Trace:
I20260709 13:19:07.618260 16100 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:07.613153 16236 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:07.653214 12779 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:07.667790 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:07.662712 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:07.670608 12598 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:19:07.621896 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 60969.2us
I20260709 13:19:07.668702 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:07.653177 16330 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:07.656492 12700 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:07.633220 16107 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:07.680073 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:07.731984 12633 raft_consensus.cc:2410] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:19:07.615197 16252 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:07.625377 16311 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:19:07.624179 16281 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.683303 12627 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:07.690791 16290 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:07.677917 16310 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:07.682225 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:07.634903 16321 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:07.754998 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:07.755745 12625 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:19:07.756623 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:07.757120 12747 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:07.703032 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 101045us
I20260709 13:19:07.659265 16152 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:07.676342 16234 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.691462 16069 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:07.704536 12776 rpcz_store.cc:276] 0709 13:19:06.241972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.607467 (+1365495us) service_pool.cc:224] Handling call
0709 13:19:07.607826 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:07.632280 (+ 24454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.635514 (+  3234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:07.772504 16234 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.773046 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:07.774554 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:07.775003 16234 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:07.776060 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:07.776976 12776 raft_consensus.cc:2393] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 12: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:07.778674 12776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2311) took 1427 ms. Trace:
I20260709 13:19:07.778947 15812 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:07.779120 12776 rpcz_store.cc:276] 0709 13:19:06.351023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.777683 (+1426660us) service_pool.cc:224] Handling call
0709 13:19:07.778024 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:07.778261 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:07.778312 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:07.778371 (+    59us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:07.778416 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.778485 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.778611 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:07.780539 13376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51762 (request call id 630) took 1338 ms. Trace:
I20260709 13:19:07.729692 12783 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:19:07.780969 13376 rpcz_store.cc:276] 0709 13:19:06.442080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.803960 (+361880us) service_pool.cc:224] Handling call
0709 13:19:06.804683 (+   723us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:07.646779 (+842096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.780270 (+133491us) spinlock_profiling.cc:243] Waited 841 ms on lock 0x7b5400486054. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:07.780480 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1934684928}
I20260709 13:19:07.707610 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:07.735512 12672 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:07.739513 12780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1969) took 1409 ms. Trace:
I20260709 13:19:07.740170 16281 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:07.740614 16236 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:07.736693 16219 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:07.735831 16107 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:07.728055 16280 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:07.743276 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:07.735563 16252 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:07.680794 15592 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:07.748574 12779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1967) took 1419 ms. Trace:
I20260709 13:19:07.749511 13382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 643) took 2086 ms. Trace:
I20260709 13:19:07.749553 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:07.736469 12705 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 5.
I20260709 13:19:07.750423 16198 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:19:07.751425 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 80919.6us
I20260709 13:19:07.752528 12782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1970) took 1422 ms. Trace:
I20260709 13:19:07.706033 16286 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:07.755043 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:07.656196 16306 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:07.630673 15557 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.757818 16311 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:07.759400 12777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2309) took 1517 ms. Trace:
I20260709 13:19:07.757721 16087 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.738042 12701 raft_consensus.cc:3055] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:07.688750 15931 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:07.702718 16109 raft_consensus.cc:740] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:07.760123 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:07.757341 15952 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:07.783353 12704 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:19:07.786216 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:07.837970 12707 raft_consensus.cc:2393] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:07.800294 13348 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:07.859009 12778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1968) took 1529 ms. Trace:
W20260709 13:19:07.794543 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 59117us
I20260709 13:19:07.841861 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:07.842350 12598 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:07.780120 16292 log.cc:927] Time spent T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 1.448s	user 0.005s	sys 0.013s
I20260709 13:19:07.782752 12780 rpcz_store.cc:276] 0709 13:19:06.330283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.711766 (+1381483us) service_pool.cc:224] Handling call
0709 13:19:07.738707 (+ 26941us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:07.738883 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:07.738914 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:07.738957 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:07.738992 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.739048 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.739308 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:07.838364 12779 rpcz_store.cc:276] 0709 13:19:06.329696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.652284 (+1322588us) service_pool.cc:224] Handling call
0709 13:19:07.652760 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:07.734418 (+ 81658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.748511 (+ 14093us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:07.843266 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:07.843343 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:07.853435 12701 raft_consensus.cc:740] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:07.768004 12627 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:07.855057 12673 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:07.852275 12777 rpcz_store.cc:276] 0709 13:19:06.242174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.626185 (+1384011us) service_pool.cc:224] Handling call
0709 13:19:07.632123 (+  5938us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:07.632372 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:07.632413 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:07.632844 (+   431us) log.cc:844] Serialized 26 byte log entry
0709 13:19:07.753814 (+120970us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:07.753858 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.753927 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.753975 (+    48us) raft_consensus.cc:1685] finished
0709 13:19:07.753987 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.759319 (+  5332us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":61138,"threads_started":1,"wal-append.queue_time_us":40025}
I20260709 13:19:07.729321 16100 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:07.837442 12782 rpcz_store.cc:276] 0709 13:19:06.330858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.751743 (+1420885us) service_pool.cc:224] Handling call
0709 13:19:07.752045 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:07.752218 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:07.752246 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:07.752292 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:07.752324 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.752378 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.752474 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:07.856766 12634 raft_consensus.cc:2410] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 13:19:07.858930 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 58732.2us
I20260709 13:19:07.837347 13382 rpcz_store.cc:276] 0709 13:19:05.663509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.665203 (+  1694us) service_pool.cc:224] Handling call
0709 13:19:05.665516 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:05.670995 (+  5479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:05.671034 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:05.671555 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 13:19:05.870501 (+198946us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:05.870538 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:05.870632 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.633922 (+1763290us) spinlock_profiling.cc:243] Waited 4.17 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:07.749213 (+115291us) raft_consensus.cc:1685] finished
0709 13:19:07.749234 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.749451 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1351,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"spinlock_wait_cycles":9590144,"thread_start_us":198400,"threads_started":1,"wal-append.queue_time_us":266391}
I20260709 13:19:07.879540 12598 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:07.781528 12776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 1971) took 1429 ms. Trace:
W20260709 13:19:07.868456 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 82311.7us
I20260709 13:19:07.870288 12705 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:19:07.880250 12776 rpcz_store.cc:276] 0709 13:19:06.352679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.780556 (+1427877us) service_pool.cc:224] Handling call
0709 13:19:07.780905 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:07.781113 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:07.781150 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:07.781210 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:07.781253 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.781337 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.781465 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:07.840047 16234 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:07.882472 12782 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:07.883711 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:07.884276 12634 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:07.886535 12598 leader_election.cc:400] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10.
I20260709 13:19:07.885453 12747 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:07.888265 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:07.861114 16198 raft_consensus.cc:697] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:07.891618 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:07.892195 12780 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:07.892256 16198 consensus_queue.cc:237] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.881611 16001 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:07.854740 16321 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:07.947189 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 61628.4us
I20260709 13:19:07.948187 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:07.948597 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 93573.8us
I20260709 13:19:07.949998 12672 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:07.883589 12701 consensus_queue.cc:260] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.963220 12701 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:07.967036 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 85482.1us
I20260709 13:19:07.968055 12747 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:07.879666 16310 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:07.790872 16219 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.790642 16236 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:07.997042 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:08.005064 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:07.996126 15950 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.014469 15950 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:08.015080 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:08.016674 15950 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:08.017148 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 67259.9us
I20260709 13:19:08.017812 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:08.018118 16263 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:08.018935 15946 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:08.019140 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:08.019356 15946 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:08.019737 15946 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:08.019893 12775 raft_consensus.cc:2393] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 11: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:08.020758 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:08.017267 16290 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:08.018592 13379 raft_consensus.cc:2410] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:19:08.006515 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:08.015460 16322 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:08.014736 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:07.886158 15812 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:07.895356 16069 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:08.018623 12672 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:08.022149 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:07.972952 16324 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:08.010170 16001 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.857954 16087 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:08.053896 12700 raft_consensus.cc:2393] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 11: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:07.946733 16342 consensus_queue.cc:1059] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.158s
I20260709 13:19:07.973109 16100 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:08.059731 12775 raft_consensus.cc:2410] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:19:08.043599 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:08.080830 13380 raft_consensus.cc:2376] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9. Re-sending same reply.
I20260709 13:19:08.067041 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:08.009007 16316 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.030499 16306 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:07.996992 16321 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:07.883347 16315 consensus_queue.cc:1059] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.142s
I20260709 13:19:08.043076 16290 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:08.101215 16290 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:08.101877 15931 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:08.104614 16322 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:08.105114 16322 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:08.105520 16322 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:08.106604 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:07.859596 12778 rpcz_store.cc:276] 0709 13:19:06.330036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.675936 (+1345900us) service_pool.cc:224] Handling call
0709 13:19:07.676245 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:07.676473 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:19:07.679698 (+  3225us) log.cc:844] Serialized 14 byte log entry
0709 13:19:07.858548 (+178850us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:07.858622 (+    74us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:07.858695 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.858785 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.858933 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3620,"prepare.queue_time_us":3509,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":64,"raft.queue_time_us":3751,"thread_start_us":174760,"threads_started":1}
I20260709 13:19:08.107515 12775 raft_consensus.cc:2410] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:19:08.030773 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:08.092221 16306 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:08.109637 16306 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:08.074008 16069 raft_consensus.cc:697] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:08.054526 15812 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:08.023744 16311 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:19:08.099325 16236 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:08.096740 16324 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:08.099800 15557 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:08.100373 16315 raft_consensus.cc:3055] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:08.100718 15592 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:08.072336 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:08.157917 13382 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:08.083997 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:08.000638 13348 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:19:08.160321 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 159780us
I20260709 13:19:08.113142 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:08.162367 12777 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:08.113219 12781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2310) took 1796 ms. Trace:
I20260709 13:19:08.120733 12778 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:08.100396 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:08.129380 12625 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:08.092805 16168 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:08.131625 15812 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:08.072937 16317 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:19:08.217790 16087 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:08.219218 16280 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:08.219969 16280 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:08.142673 16315 raft_consensus.cc:740] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:08.222867 16281 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:08.222790 16315 consensus_queue.cc:260] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:08.223654 16281 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:08.224598 16315 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:08.217123 15557 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:08.024639 16109 consensus_queue.cc:260] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:08.078199 16001 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:08.140774 12597 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:08.246363 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 105691us
I20260709 13:19:08.243106 12780 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:08.141644 12701 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
W20260709 13:19:08.163865 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.247367 12597 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:08.163314 12781 rpcz_store.cc:276] 0709 13:19:06.316998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.639181 (+1322183us) service_pool.cc:224] Handling call
0709 13:19:07.680930 (+ 41749us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:07.681224 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:19:07.882266 (+201042us) log.cc:844] Serialized 14 byte log entry
0709 13:19:08.112688 (+230422us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:08.112755 (+    67us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:08.112816 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:08.112979 (+   163us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:08.113122 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2453,"prepare.queue_time_us":53742,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":108,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":6717,"thread_start_us":406572,"threads_started":2}
I20260709 13:19:08.099521 16107 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:08.172027 12628 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 9: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:08.176426 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:08.096975 16310 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:19:08.139146 15950 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:08.178527 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:08.188426 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:08.190085 16289 raft_consensus.cc:3055] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:08.176087 16103 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:08.136153 16324 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:08.298017 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:08.298633 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:08.298837 12633 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:08.299190 12630 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:08.195279 12626 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56838 (request call id 1461) took 1855 ms (client timeout 1711 ms). Trace:
I20260709 13:19:08.196724 15878 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:08.197919 16321 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:08.093503 16316 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:08.199666 15946 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:08.200915 16322 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:08.205925 16308 raft_consensus.cc:993] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530: : Instructing follower 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:19:08.205900 16168 raft_consensus.cc:697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:08.208142 16115 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:08.214531 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:08.214161 16069 consensus_queue.cc:237] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:08.039953 12634 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:19:08.266803 12625 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
W20260709 13:19:08.276522 12703 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1541) took 1940 ms (client timeout 1711 ms). Trace:
W20260709 13:19:08.276619 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 261614us
W20260709 13:19:08.280233 12783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54346 (request call id 447) took 2034 ms (client timeout 2221 ms). Trace:
W20260709 13:19:08.323942 12783 rpcz_store.cc:269] 0709 13:19:06.232915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.138235 (+905320us) service_pool.cc:224] Handling call
0709 13:19:08.267262 (+1129027us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b54000cc954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:08.267418 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1321980160}
I20260709 13:19:08.265290 15998 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:19:08.299656 12626 rpcz_store.cc:269] 0709 13:19:06.340329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.341523 (+  1194us) service_pool.cc:224] Handling call
0709 13:19:08.195107 (+1853584us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54000dcd54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:08.195213 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2491278208}
I20260709 13:19:08.326994 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:08.327584 12626 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:19:08.245424 15557 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:08.309332 12629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2479) took 2589 ms. Trace:
I20260709 13:19:08.267694 16109 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:08.311738 12781 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:08.254232 16270 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:08.318224 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:08.324522 12747 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:08.323712 12703 rpcz_store.cc:269] 0709 13:19:06.336272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.342394 (+  6122us) service_pool.cc:224] Handling call
0709 13:19:08.269757 (+1927363us) spinlock_profiling.cc:243] Waited 4.94 ms on lock 0x7b5400248528. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7318f924 00007f4c72378227 00007f4c72377ab8 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:08.276443 (+  6686us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11372928}
I20260709 13:19:08.294225 16281 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:08.336479 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:08.345261 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:08.345865 12634 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:19:08.283740 16289 raft_consensus.cc:740] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:08.308315 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:08.338330 12629 rpcz_store.cc:276] 0709 13:19:05.713646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.713780 (+   134us) service_pool.cc:224] Handling call
0709 13:19:07.474446 (+1760666us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:07.496036 (+ 21590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:08.303095 (+807059us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b54003d8950. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:08.303229 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3991622912}
W20260709 13:19:08.317888 16237 consensus_meta.cc:287] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.615s	user 0.007s	sys 0.004s
I20260709 13:19:08.350087 12628 raft_consensus.cc:2368] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 9. Current term is 10.
I20260709 13:19:08.305047 16324 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:08.328577 16358 log.cc:927] Time spent T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:19:08.352855 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:08.352995 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:08.354082 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:08.354667 16322 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:08.294274 15950 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:08.357182 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:08.360863 12628 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:08.357734 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:08.343678 16343 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.014s	sys 0.023s
W20260709 13:19:08.361747 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 343186us
I20260709 13:19:08.356101 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:08.374051 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:08.374562 12634 raft_consensus.cc:2393] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 8: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:08.365279 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:08.342123 16308 raft_consensus.cc:1081] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Signalling peer 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:19:08.383607 16316 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:08.387461 16330 consensus_meta.cc:287] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.654s	user 0.009s	sys 0.000s
I20260709 13:19:08.388319 12630 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
W20260709 13:19:08.388974 13384 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51754 (request call id 655) took 2070 ms (client timeout 1782 ms). Trace:
W20260709 13:19:08.374615 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.363960 12633 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:19:08.375681 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:08.389339 13384 rpcz_store.cc:269] 0709 13:19:06.318413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.318572 (+   159us) service_pool.cc:224] Handling call
0709 13:19:08.388783 (+2070211us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54004bfd54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:08.388906 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2892028672}
I20260709 13:19:08.390082 12706 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:19:08.381721 12703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:19:08.391013 12703 raft_consensus.cc:606] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:08.360751 16237 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:08.392592 12630 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:08.394687 16280 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:08.348928 16219 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:08.344873 15998 raft_consensus.cc:697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:08.329260 16290 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:08.377712 16168 consensus_queue.cc:237] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:08.378834 16289 consensus_queue.cc:260] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:08.410678 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:08.411582 12708 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
W20260709 13:19:08.381719 16343 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.015s	sys 0.023s
I20260709 13:19:08.382966 15950 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:08.376852 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:08.415179 12701 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 11: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:08.377529 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:08.356417 16087 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:08.378110 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:08.419685 12632 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:19:08.420758 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:08.401952 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:08.349787 16270 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:08.387876 16330 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:08.402935 13382 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:19:08.401461 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:08.432559 12780 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:19:08.404067 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
W20260709 13:19:08.400528 16252 consensus_meta.cc:287] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.538s	user 0.010s	sys 0.000s
I20260709 13:19:08.356115 15946 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:08.393332 15557 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:08.401748 12633 raft_consensus.cc:1240] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:08.436751 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:08.428217 13349 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:08.434916 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:08.434926 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:08.440876 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:08.440658 15998 consensus_queue.cc:237] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:08.441419 12780 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:19:08.441460 16156 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:08.430599 16270 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:08.457114 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:08.459646 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:08.464233 12746 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:08.460526 13384 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
W20260709 13:19:08.465227 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 193646us
W20260709 13:19:08.465689 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 66339.8us
I20260709 13:19:08.466073 12780 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:19:08.419278 16087 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:08.470453 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:08.482577 12597 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:08.483840 12597 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:08.500182 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:08.403668 16103 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.062s
W20260709 13:19:08.529073 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 52777.5us
I20260709 13:19:08.529060 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:08.538875 12673 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:08.540181 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:08.532424 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:08.532675 16198 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:08.538901 12672 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:08.467608 16251 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:08.549685 13378 raft_consensus.cc:2368] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 9. Current term is 10.
I20260709 13:19:08.487699 16087 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:08.488895 16353 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:19:08.560815 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 75940.2us
I20260709 13:19:08.553956 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:08.552501 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:08.554414 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:08.437995 16252 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:08.565114 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:08.565881 13383 raft_consensus.cc:2393] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 6: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:08.567224 12673 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:08.568409 12746 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:08.570333 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:08.550420 15557 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:08.635243 15557 raft_consensus.cc:697] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:08.577245 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:08.577768 16360 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:08.579629 16330 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:08.574788 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:08.586190 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:08.588119 16289 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:08.589221 16127 raft_consensus.cc:3055] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:08.591406 16252 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:08.591876 12776 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:08.595647 16284 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:08.595818 16282 consensus_queue.cc:1059] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:08.597420 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:08.599592 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 60806.6us
I20260709 13:19:08.599869 16010 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:08.606328 16290 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:08.608856 15952 consensus_queue.cc:1059] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:08.608919 16306 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:08.609697 16313 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:08.663300 16313 raft_consensus.cc:697] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:08.664331 16313 consensus_queue.cc:237] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:08.669688 12774 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2308) took 2474 ms. Trace:
I20260709 13:19:08.670094 16308 raft_consensus.cc:993] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530: : Instructing follower 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:19:08.670616 16308 raft_consensus.cc:1081] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Signalling peer 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:19:08.670308 12774 rpcz_store.cc:276] 0709 13:19:06.195140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.208362 (+ 13222us) service_pool.cc:224] Handling call
0709 13:19:06.248346 (+ 39984us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:06.248750 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.248789 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:08.264488 (+2015699us) spinlock_profiling.cc:243] Waited 22.2 ms on lock 0x7b1800460900. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2932a6 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6bcaba11 00007f4c6bcbb372 00007f4c6d3eda01 00007f4c6d3b243a
0709 13:19:08.277469 (+ 12981us) log.cc:844] Serialized 26 byte log entry
0709 13:19:08.467311 (+189842us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:08.467370 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:08.467495 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:08.669219 (+201724us) raft_consensus.cc:1685] finished
0709 13:19:08.669255 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:08.669597 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1037,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":40061,"spinlock_wait_cycles":52970624,"thread_start_us":183595,"threads_started":1,"wal-append.queue_time_us":369020}
I20260709 13:19:08.611032 12630 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:08.612385 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:08.676661 12627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56832 (request call id 2495) took 1404 ms. Trace:
I20260709 13:19:08.615474 15946 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:08.677345 15946 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:08.678083 16270 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:08.616633 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:08.679200 12782 raft_consensus.cc:2393] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:08.618103 12631 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7"
dest_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:56832
I20260709 13:19:08.679792 12631 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:08.680245 12631 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:08.619719 16305 log.cc:927] Time spent T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 1.483s	user 0.004s	sys 0.008s
W20260709 13:19:08.620371 16366 log.cc:927] Time spent T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:19:08.575345 16311 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:08.572568 16353 raft_consensus.cc:697] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:08.626020 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:08.630504 13349 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:08.631597 15812 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:08.553202 16198 raft_consensus.cc:697] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:08.639842 16001 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:08.641002 12628 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:08.654745 12672 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:08.653110 16127 raft_consensus.cc:740] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:08.659454 16271 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:08.659492 16282 raft_consensus.cc:3055] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:08.661994 16284 raft_consensus.cc:697] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:08.571862 12746 leader_election.cc:400] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 9. Current term is 10.
I20260709 13:19:08.727301 12746 leader_election.cc:403] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 13:19:08.727782 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 157805us
I20260709 13:19:08.678917 12781 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:19:08.677033 12627 rpcz_store.cc:276] 0709 13:19:07.271844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.275821 (+  3977us) service_pool.cc:224] Handling call
0709 13:19:08.676514 (+1400693us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b540002c954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:08.676609 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":935390592}
I20260709 13:19:08.702857 12776 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
W20260709 13:19:08.719532 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 89158.6us
W20260709 13:19:08.729867 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 67014.9us
I20260709 13:19:08.731891 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:08.721585 16308 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:08.723387 12629 raft_consensus.cc:1240] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:08.701825 16311 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:08.721469 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 66798.7us
I20260709 13:19:08.661850 16087 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:08.737103 12672 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:08.732946 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 152525us
I20260709 13:19:08.739208 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:08.739394 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:08.739884 12775 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:19:08.741535 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:08.724359 16315 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:08.727185 16282 raft_consensus.cc:740] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:08.655550 16324 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:08.644714 15557 consensus_queue.cc:237] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:08.770308 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:08.748481 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:08.764103 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:08.773701 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:08.772516 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:08.774623 12782 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:08.776698 16236 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:08.777323 16236 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:08.777107 16282 consensus_queue.cc:260] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:08.777932 16236 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:08.778688 16282 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:08.778935 16324 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:08.783566 16109 raft_consensus.cc:993] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530: : Instructing follower 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:19:08.784261 16109 raft_consensus.cc:1081] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Signalling peer 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:19:08.786022 16353 consensus_queue.cc:237] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:08.787945 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 89128.7us
I20260709 13:19:08.734350 16308 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:08.738006 16311 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:08.736860 16237 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:08.740784 16251 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:08.795944 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:08.798374 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:08.810006 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:08.811538 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:08.798633 12702 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13. Ops: [8.9-12.11]
I20260709 13:19:08.799582 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:08.800479 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:08.814774 12700 raft_consensus.cc:2410] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:19:08.817473 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:08.769186 16001 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:08.819015 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:08.768468 15878 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:08.821069 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:08.766404 15998 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:08.793874 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:08.826823 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:08.831138 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:08.832731 12704 raft_consensus.cc:2393] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:08.833529 12747 leader_election.cc:400] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 9. Current term is 10.
I20260709 13:19:08.793644 16308 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:08.835373 12705 raft_consensus.cc:2393] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 11: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:08.835587 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:08.827800 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:08.834035 12747 leader_election.cc:403] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:19:08.836274 12709 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:19:08.827137 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:08.834223 13377 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:08.836285 12783 raft_consensus.cc:2368] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10.
I20260709 13:19:08.838117 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:08.838052 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:08.828116 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:08.806596 16324 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:08.834529 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:08.842000 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:08.843667 12704 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:08.846662 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:08.789848 16198 consensus_queue.cc:237] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:08.841810 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:08.840049 16363 log.cc:927] Time spent T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:19:08.852825 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:08.838764 12700 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:08.844753 16315 raft_consensus.cc:697] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:19:08.857535 16368 log.cc:927] Time spent T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:19:08.857750 16315 consensus_queue.cc:237] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:08.859822 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:08.860633 12705 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:08.849354 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:08.861779 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:08.862830 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:08.866180 12703 raft_consensus.cc:1240] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:08.869695 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:08.869139 16284 consensus_queue.cc:237] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:08.867228 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:08.874509 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:08.875150 12700 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:08.876945 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:08.840266 15370 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:19:08.845194 16251 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:08.843423 16324 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:08.840801 15878 raft_consensus.cc:697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:08.889973 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:08.875765 12747 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:08.884593 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:08.861402 12779 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:08.894289 12703 raft_consensus.cc:1240] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:08.893646 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:08.896613 16330 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:08.899792 16241 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:08.900331 16241 raft_consensus.cc:697] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:08.901152 16241 consensus_queue.cc:237] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:08.902598 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:08.905207 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:08.905846 12778 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:08.913367 16010 raft_consensus.cc:3055] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:08.913962 16010 raft_consensus.cc:740] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:19:08.914222 16376 log.cc:927] Time spent T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:19:08.915472 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:08.845309 16360 consensus_queue.cc:1059] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:19:08.933060 16360 consensus_queue.cc:1243] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer 522fd2ba0e9f402691b5ef29af8579cd log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:19:08.934322 16360 raft_consensus.cc:3055] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:08.934737 16360 raft_consensus.cc:740] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:08.935770 16360 consensus_queue.cc:260] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:08.883566 16001 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:08.894632 16324 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:08.893076 15946 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:08.942059 16156 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.943018 16156 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:08.945773 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:08.946588 12627 raft_consensus.cc:1240] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:08.887135 16306 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:08.916791 12777 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:08.917519 12779 raft_consensus.cc:1240] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:08.828807 16127 consensus_queue.cc:260] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:08.918985 15878 consensus_queue.cc:237] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:08.966364 12776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36478 (request call id 2320) took 1088 ms. Trace:
I20260709 13:19:08.966815 12776 rpcz_store.cc:276] 0709 13:19:07.878410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.881113 (+  2703us) service_pool.cc:224] Handling call
0709 13:19:08.966193 (+1085080us) spinlock_profiling.cc:243] Waited 703 ms on lock 0x7b54002cc754. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:08.966301 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1616649600}
I20260709 13:19:08.967555 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:08.969379 13384 pending_rounds.cc:85] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: Aborting all ops after (but not including) 8
I20260709 13:19:08.970049 13384 pending_rounds.cc:107] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:19:08.970326 13384 raft_consensus.cc:2889] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:08.922135 16087 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:08.924985 15812 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:08.927865 16270 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:08.929668 12597 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:08.931438 12700 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
W20260709 13:19:08.986069 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.956144 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:08.986943 12778 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:08.973515 12775 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:08.990998 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:19:08.992928 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 63365.5us
I20260709 13:19:08.991597 13379 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:08.997826 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:08.998235 12629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2510) took 1177 ms. Trace:
I20260709 13:19:08.998698 12629 rpcz_store.cc:276] 0709 13:19:07.820565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.378736 (+558171us) service_pool.cc:224] Handling call
0709 13:19:08.379027 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:08.732466 (+353439us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:08.998025 (+265559us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b5400049cd4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:08.998181 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":791364864}
I20260709 13:19:09.002280 12777 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:09.003825 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:09.004026 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:09.007290 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:09.011540 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:09.006526 12597 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:08.998982 16360 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:09.021049 12597 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:08.998059 16234 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:09.023227 12597 leader_election.cc:400] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10.
I20260709 13:19:09.023909 12597 leader_election.cc:403] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:19:09.000932 16069 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:09.025955 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:09.025301 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 131986us
I20260709 13:19:08.968639 16010 consensus_queue.cc:260] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:09.026000 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:08.994450 16379 consensus_queue.cc:1059] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:19:09.035166 12597 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:09.035498 13348 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:09.043715 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:09.037140 12703 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:19:09.046454 12747 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:09.047361 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:09.048537 12776 raft_consensus.cc:2410] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:19:09.048575 12747 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:09.049817 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:09.041955 12597 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:09.051605 12778 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:09.052312 13348 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:09.040198 12631 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:09.055773 16010 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:09.055858 16168 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:09.033954 15592 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:09.057446 16281 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:09.081400 16281 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:09.081990 16290 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:09.082811 16290 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:09.084281 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:09.085067 12705 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:09.081885 16281 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:09.055827 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:09.031037 16361 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:09.023196 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:09.058252 13348 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:09.096701 16310 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:09.097175 16310 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:09.097607 16310 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:09.077387 16156 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:09.100696 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:09.102226 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:09.057312 16100 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:09.102571 16308 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:09.102983 16100 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:09.103418 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:09.104698 12702 raft_consensus.cc:1240] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:09.063508 16383 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:09.066370 16378 consensus_queue.cc:1059] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:09.106504 16378 raft_consensus.cc:3055] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:09.106882 16378 raft_consensus.cc:740] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:09.107772 16378 consensus_queue.cc:260] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:09.109189 16378 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:09.059363 16236 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:09.060672 12597 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:19:09.111426 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 50835.1us
I20260709 13:19:09.112115 12782 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:09.070614 15592 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:09.037787 15931 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10.
I20260709 13:19:09.112884 15592 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:09.112716 16310 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:09.082059 12628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2511) took 1259 ms. Trace:
I20260709 13:19:09.113709 12628 rpcz_store.cc:276] 0709 13:19:07.822530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.386820 (+564290us) service_pool.cc:224] Handling call
0709 13:19:08.387087 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:08.722172 (+335085us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:09.081806 (+359634us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b5400159fd4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:09.081993 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":583343360}
I20260709 13:19:09.057416 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:09.115512 12776 raft_consensus.cc:2410] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:19:09.113282 15931 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:09.051182 15998 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:09.056797 12777 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:09.088534 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:09.071403 16321 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:09.089497 16361 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:09.103506 16100 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:09.111471 16236 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:09.066128 12631 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:09.067291 16330 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:09.137779 12781 raft_consensus.cc:1275] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 5. (term mismatch)
I20260709 13:19:09.138258 12781 pending_rounds.cc:85] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 4
I20260709 13:19:09.138542 12781 pending_rounds.cc:107] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 13:19:09.138814 12781 raft_consensus.cc:2889] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:09.126870 12703 raft_consensus.cc:2393] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 6: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:09.112557 12597 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:09.131583 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:09.085911 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.135476 12776 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:36478
I20260709 13:19:09.148918 12597 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:09.141259 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:09.125308 15931 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:09.130210 16001 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:19:09.142062 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.104713 16308 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:09.143304 16310 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:09.146116 16237 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 9. Current term is 10.
I20260709 13:19:09.167498 12598 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:09.136849 16324 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:09.168992 16311 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:09.171356 12597 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:09.188959 13379 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:09.171936 15998 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:09.175765 12672 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:09.176831 16107 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:09.190150 12597 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:09.139765 16251 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 9. Current term is 10.
I20260709 13:19:09.134812 16361 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:09.130043 16321 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:09.125741 16383 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:09.147593 16330 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:09.187054 12598 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:09.209854 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:09.175385 15931 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:09.224567 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:09.208180 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:09.214694 16321 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:09.222486 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:09.210747 16309 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:09.199661 16271 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:09.190856 16270 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:09.371979 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:09.215032 16290 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:09.218462 15998 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:09.216850 12705 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
W20260709 13:19:09.276635 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 226859us
I20260709 13:19:09.414273 12747 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:09.253865 12597 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:09.308173 12708 raft_consensus.cc:1240] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:09.313964 13378 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:19:09.316480 16100 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:09.250036 16308 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:09.228892 16383 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:09.322289 16360 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.348408 16198 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:09.350040 12632 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:19:09.199828 16069 consensus_queue.cc:1059] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 13:19:09.375202 16330 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:09.191419 16152 consensus_queue.cc:1059] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:19:09.360960 16353 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:09.432551 12777 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20db0f202ea546699789c2325d241b35"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:36478
W20260709 13:19:09.386395 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.433274 12777 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:09.433670 12777 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:09.401708 12630 raft_consensus.cc:1240] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:09.406649 12781 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:09.407508 12776 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:09.436324 12776 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:09.400022 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:19:09.391896 16389 log.cc:927] Time spent T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:19:09.384536 16316 raft_consensus.cc:993] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530: : Instructing follower 3d539f8d95d343d8b1696803f18b30d2 to start an election
W20260709 13:19:09.376547 16392 log.cc:927] Time spent T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.247s	user 0.003s	sys 0.000s
W20260709 13:19:09.425323 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 171551us
I20260709 13:19:09.449179 16330 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:09.442227 16395 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:09.450757 16311 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:09.452693 16152 raft_consensus.cc:3055] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:09.453101 16152 raft_consensus.cc:740] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:09.453745 15812 raft_consensus.cc:3055] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:09.454227 15812 raft_consensus.cc:740] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:09.455201 15812 consensus_queue.cc:260] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:09.456620 15812 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:09.456986 12700 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:19:09.446884 16353 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:09.459661 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.460217 16353 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:09.432621 16237 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:09.461426 16237 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:09.463815 16237 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:09.464556 16395 raft_consensus.cc:697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:09.357846 16271 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:09.432164 12775 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 9. Current term is 10. Ops: [7.8-9.9]
I20260709 13:19:09.423609 16087 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:09.432122 15998 raft_consensus.cc:697] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:09.423614 16290 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:09.483394 16383 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:09.465509 16395 consensus_queue.cc:237] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:09.484032 16100 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:09.483740 16152 consensus_queue.cc:260] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:09.488404 12781 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:19:09.468267 16271 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:09.489045 12777 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:09.490391 12776 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:09.437109 16360 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:09.309140 16107 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.122s
I20260709 13:19:09.504247 15998 consensus_queue.cc:237] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:09.506094 12777 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:09.483469 16290 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:09.404794 16281 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:09.485762 16152 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:09.518795 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 331785us
I20260709 13:19:09.531939 12628 raft_consensus.cc:1275] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:19:09.532286 13349 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:09.538447 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:09.538846 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:09.539455 12781 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:09.540671 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:09.533313 12598 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:09.501199 16324 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:09.534898 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 119225us
I20260709 13:19:09.544885 12781 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
I20260709 13:19:09.545429 12634 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:09.550464 12598 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:09.554090 12598 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:09.558086 12598 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:09.510433 16361 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:09.555562 16313 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.023s	sys 0.023s
I20260709 13:19:09.570616 13349 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:09.572875 12598 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:09.552954 16198 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:09.578698 12627 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:09.557430 15998 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:09.579203 16281 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:09.579730 16361 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:09.580356 16290 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:09.580637 16401 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:09.567869 15950 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:09.584196 13378 raft_consensus.cc:1240] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:09.576543 12598 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:09.551918 16370 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:09.552059 15557 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
W20260709 13:19:09.571424 16313 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.025s	sys 0.025s
I20260709 13:19:09.586001 12597 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:09.589237 12598 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:09.590037 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:09.594460 12777 raft_consensus.cc:2393] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 11: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:09.552024 15592 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:09.567611 16400 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:09.594599 13380 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:09.617995 16405 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:09.602633 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:09.621796 12777 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:09.579421 16397 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:09.593480 16361 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:09.603461 16281 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:09.648697 16281 raft_consensus.cc:697] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:09.649600 16281 consensus_queue.cc:237] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:09.590718 15950 consensus_queue.cc:1048] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
W20260709 13:19:09.640125 16393 log.cc:927] Time spent T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:19:09.640674 15592 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:09.642601 16127 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:09.645977 16360 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:09.638448 12777 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:19:09.660219 12783 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:09.633945 16069 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:19:09.669152 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 86129.9us
W20260709 13:19:09.661365 16388 log.cc:927] Time spent T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
W20260709 13:19:09.665546 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 80994.3us
I20260709 13:19:09.671912 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:09.670064 13348 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:09.674544 16400 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:09.675199 16400 raft_consensus.cc:697] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 10 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:09.674834 12777 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:09.676792 13380 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:09.676352 16400 consensus_queue.cc:237] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:09.678803 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:09.684048 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:09.684156 16406 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:09.685423 16406 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:09.688763 16406 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:09.689976 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:09.695784 15931 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:09.669605 16069 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:09.698994 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 109846us
I20260709 13:19:09.700402 12598 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:09.704903 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:09.708877 16152 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:09.718181 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:09.718933 13380 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:09.720566 12631 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:09.706969 16069 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:09.709789 16152 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:09.720139 16290 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:19:09.708250 16251 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:09.740808 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:09.750489 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:09.750540 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:09.755224 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:09.777140 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:09.777288 12777 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:09.777725 12704 raft_consensus.cc:3055] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:09.758646 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:09.727097 12746 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:09.727209 16152 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:09.783252 16402 raft_consensus.cc:2764] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:19:09.783174 16290 consensus_queue.cc:1059] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:09.784389 16234 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:09.784739 12707 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:09.784920 16234 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:09.731129 16286 raft_consensus.cc:3055] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:09.788966 16286 raft_consensus.cc:740] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:09.790773 12776 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:09.790485 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 63466.1us
I20260709 13:19:09.778317 12704 raft_consensus.cc:740] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:09.795604 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:09.723037 12598 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:09.745414 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:09.798832 12708 raft_consensus.cc:2393] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 10: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:09.746456 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:09.741904 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:09.800580 12702 raft_consensus.cc:2393] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 11: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:09.793995 12704 consensus_queue.cc:260] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:09.726662 16241 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.819s	user 0.017s	sys 0.001s
I20260709 13:19:09.799080 16324 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:09.739290 16165 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.804702 16286 consensus_queue.cc:260] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:09.805136 16321 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:09.807169 16271 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:09.810381 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 69090.4us
I20260709 13:19:09.810895 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:09.811275 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 103590us
W20260709 13:19:09.812222 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 89277.1us
I20260709 13:19:09.815789 12747 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:09.789821 16316 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.735248 15931 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:09.826651 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 154590us
I20260709 13:19:09.817715 12707 raft_consensus.cc:3055] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:09.834619 12707 raft_consensus.cc:740] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:19:09.810081 16241 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.906s	user 0.017s	sys 0.001s
I20260709 13:19:09.830466 12597 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:09.825341 12598 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:09.838179 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:09.839794 12597 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:09.843115 16409 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:09.843799 16409 raft_consensus.cc:697] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:09.846494 16357 consensus_queue.cc:1059] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:09.847317 16357 raft_consensus.cc:3055] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:09.847652 16357 raft_consensus.cc:740] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:09.849959 15946 consensus_queue.cc:1059] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:09.850690 15946 consensus_queue.cc:1243] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 10.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:19:09.851613 15946 raft_consensus.cc:3055] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:09.851969 15946 raft_consensus.cc:740] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:09.852761 15946 consensus_queue.cc:260] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:09.852972 16109 consensus_queue.cc:1059] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:09.854604 15946 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:09.857748 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.860404 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:09.860950 12776 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:09.844852 16409 consensus_queue.cc:237] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:09.848558 16357 consensus_queue.cc:260] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:09.864415 16357 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:09.867933 12776 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:09.870069 12597 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:09.835405 12707 consensus_queue.cc:260] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:09.774860 16251 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:09.871609 12707 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:09.813750 16402 raft_consensus.cc:3055] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:09.873479 13376 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10. Ops: []
W20260709 13:19:09.880127 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 54903.5us
I20260709 13:19:09.812898 16324 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:09.933336 12598 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:09.939153 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:09.939679 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:09.940040 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:09.943091 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:09.930495 16402 raft_consensus.cc:740] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:09.945712 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:19:09.947367 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 724920us
I20260709 13:19:09.948487 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:09.836539 16313 raft_consensus.cc:1081] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Signalling peer 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:19:09.961359 16361 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:09.961645 16109 raft_consensus.cc:3055] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:09.961952 16361 raft_consensus.cc:697] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:09.962069 16109 raft_consensus.cc:740] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:09.872241 15557 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:09.962774 15557 raft_consensus.cc:697] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:09.833390 16316 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:09.962915 16361 consensus_queue.cc:237] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:09.964865 12783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36488 (request call id 1994) took 1110 ms. Trace:
I20260709 13:19:09.964998 16316 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:09.965255 12783 rpcz_store.cc:276] 0709 13:19:08.854560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.855247 (+   687us) service_pool.cc:224] Handling call
0709 13:19:09.964714 (+1109467us) spinlock_profiling.cc:243] Waited 689 ms on lock 0x7b540044c5d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:09.964808 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1585338112}
I20260709 13:19:09.836908 16087 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:09.813544 16290 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.119s
I20260709 13:19:09.839350 16282 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:09.825667 16152 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:09.959470 16271 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:09.833892 16400 consensus_queue.cc:1059] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.128s
I20260709 13:19:09.963016 16109 consensus_queue.cc:260] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:09.817361 16410 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:09.967496 12709 raft_consensus.cc:2410] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 13:19:09.874455 16308 consensus_queue.cc:1175] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "19cd864c053744bebd96093a7982537e" responder_term: 9 status { last_received { term: 9 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 9 index: 6 } }
I20260709 13:19:09.990195 12634 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:19:09.971861 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:09.994413 12628 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:09.972043 12707 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:09.973836 16284 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:09.976822 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:09.930642 16360 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:09.937032 15931 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:09.937827 16324 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:09.987865 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:09.826032 16165 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:09.970397 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:09.950618 12627 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:19:09.996513 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:09.997397 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:10.006933 12633 raft_consensus.cc:2393] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 11: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:10.012962 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:09.967025 15950 consensus_queue.cc:1059] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:19:09.986850 16383 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:10.018674 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:10.022589 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:09.991791 16282 raft_consensus.cc:697] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:10.025048 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:09.978685 15557 consensus_queue.cc:237] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:09.979410 16401 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:19:10.057756 16069 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:10.060765 16069 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:10.070578 16413 log.cc:927] Time spent T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 13:19:10.008904 16360 raft_consensus.cc:697] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:10.072393 16360 consensus_queue.cc:237] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:10.015036 16087 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:10.074889 16087 raft_consensus.cc:697] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:10.076831 12632 raft_consensus.cc:2368] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 11. Current term is 12.
I20260709 13:19:10.083562 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:10.084187 12625 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:10.087459 12628 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:10.089772 16010 consensus_queue.cc:1059] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:10.091065 16010 consensus_queue.cc:1048] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:10.094944 16087 consensus_queue.cc:237] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:10.096916 16423 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:10.077502 16374 log.cc:927] Time spent T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 1.223s	user 0.005s	sys 0.000s
I20260709 13:19:10.098474 16355 consensus_queue.cc:1059] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:10.099555 16355 raft_consensus.cc:3055] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:10.100029 16355 raft_consensus.cc:740] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:10.106689 16423 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:10.109091 16423 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:10.109773 16152 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:10.114111 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:10.114919 12627 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:10.115540 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.123054 16282 consensus_queue.cc:237] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:10.125320 16152 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:10.125890 16152 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:10.101079 16355 consensus_queue.cc:260] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:10.128410 16355 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:10.036562 16402 consensus_queue.cc:260] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:10.129102 16152 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:10.129828 16402 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:10.131913 16284 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.132457 16284 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:10.134807 16165 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:10.135308 16284 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:10.093153 16010 raft_consensus.cc:3055] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:10.136605 16290 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.136909 16010 raft_consensus.cc:740] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:10.137260 16290 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:10.138167 16010 consensus_queue.cc:260] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:10.139780 16290 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:10.139916 16010 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:10.141296 16321 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:10.142458 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.141983 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:10.143569 12628 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:19:10.145992 12625 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
W20260709 13:19:10.153175 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.037489 16383 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:10.165335 16383 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:10.152060 16412 log.cc:927] Time spent T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:19:10.174511 12627 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
W20260709 13:19:10.172921 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 207239us
I20260709 13:19:10.177966 13349 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:19:10.204295 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.204643 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:19:10.205279 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:10.205487 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 257158us
I20260709 13:19:10.206315 12628 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:19:10.215850 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 205411us
I20260709 13:19:10.202785 16410 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.224195 12672 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:19:10.227680 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.217417 16251 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:10.254463 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 228426us
I20260709 13:19:10.288198 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:10.222354 16400 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:19:10.278227 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.289889 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:10.293004 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 68914.8us
I20260709 13:19:10.293303 12633 raft_consensus.cc:1275] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:19:10.297744 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:10.298275 13380 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 10: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:10.298298 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:10.298465 12706 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:19:10.299255 12625 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:10.304194 12672 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:10.235206 16165 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:10.311251 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:10.314749 12747 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:10.319422 12778 raft_consensus.cc:2393] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 10: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:10.323119 12775 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:19:10.306586 16234 consensus_meta.cc:287] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.521s	user 0.014s	sys 0.007s
I20260709 13:19:10.332026 12704 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:10.331859 12747 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:10.307861 16251 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:10.333807 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:10.310119 16087 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.332456 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:10.335580 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:10.365767 12631 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:19:10.366082 12630 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:10.337146 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:10.228693 16410 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:10.342543 12629 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: []
W20260709 13:19:10.362047 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 139241us
I20260709 13:19:10.363895 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:10.367738 12747 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:10.318292 16236 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:10.380419 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:10.389935 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.341173 16383 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:10.374053 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:10.377123 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 72997.3us
I20260709 13:19:10.334142 16433 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:19:10.372959 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 84919.8us
I20260709 13:19:10.373045 16087 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:10.396412 16087 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:10.386735 16290 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:10.397181 16087 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:10.398749 16087 raft_consensus.cc:697] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:10.399971 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:10.384054 16311 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:10.331462 16234 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:10.404395 12672 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:10.412305 12672 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:10.405558 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:10.400022 15946 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:10.385368 16428 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:10.412088 12673 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:10.393306 15878 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:10.417717 12672 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:10.424868 12672 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:10.418700 16311 raft_consensus.cc:697] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:10.386333 16236 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:10.383771 16251 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:10.482643 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 102340us
W20260709 13:19:10.489595 12701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1903) took 1649 ms (client timeout 1516 ms). Trace:
W20260709 13:19:10.490046 12701 rpcz_store.cc:269] 0709 13:19:08.839758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.841783 (+  2025us) service_pool.cc:224] Handling call
0709 13:19:10.489274 (+1647491us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54002481d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:10.489490 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3332103424}
W20260709 13:19:10.492029 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 94332.7us
I20260709 13:19:10.426365 16165 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:10.410130 15370 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:19:10.425837 16127 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:10.431939 16313 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:19:10.549870 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 72851us
I20260709 13:19:10.488420 16370 raft_consensus.cc:3055] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:10.434293 15952 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:10.418145 16342 consensus_queue.cc:1048] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.119s
W20260709 13:19:10.569123 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 56904.4us
W20260709 13:19:10.543085 16392 log.cc:927] Time spent T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:19:10.423840 15946 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:10.491386 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:10.579754 16241 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:10.598613 16241 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:10.601052 16241 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:10.594815 16429 log.cc:927] Time spent T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
I20260709 13:19:10.596191 13376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 676) took 1288 ms. Trace:
I20260709 13:19:10.603403 13376 rpcz_store.cc:276] 0709 13:19:09.308143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.311533 (+  3390us) service_pool.cc:224] Handling call
0709 13:19:09.313777 (+  2244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:09.930645 (+616868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.595840 (+665195us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b54002f6054. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:10.596106 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1286795520}
I20260709 13:19:10.604183 16236 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:10.608547 16109 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:10.609865 16311 consensus_queue.cc:237] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:10.612758 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:10.614249 16410 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:10.549606 16428 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:10.620759 16428 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:10.622336 16234 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:10.432122 15878 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:10.654548 12704 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:10.569269 16370 raft_consensus.cc:740] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:10.537191 16087 consensus_queue.cc:237] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:10.541762 16306 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:09.511409 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:10.728612 12782 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:10.710209 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:10.865623 12709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38496 (request call id 438) took 1848 ms (client timeout 1650 ms). Trace:
W20260709 13:19:10.901984 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 61991us
I20260709 13:19:10.738111 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:10.918049 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 349530us
I20260709 13:19:10.890511 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:10.764376 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:10.990280 12783 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:10.920866 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:10.909042 12774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.001s	sys 0.008s
W20260709 13:19:10.994277 12774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.001s	sys 0.009s
I20260709 13:19:10.729691 12775 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:19:10.995016 12774 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2338) took 1947 ms. Trace:
W20260709 13:19:10.982668 12709 rpcz_store.cc:269] 0709 13:19:09.017031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.017182 (+   151us) service_pool.cc:224] Handling call
0709 13:19:10.865309 (+1848127us) spinlock_profiling.cc:243] Waited 933 ms on lock 0x7b54001b0654. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:10.865507 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2145910016}
W20260709 13:19:10.594641 16440 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:19:10.673285 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 182570us
W20260709 13:19:10.593549 16444 log.cc:927] Time spent T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 13:19:10.800516 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:10.914484 16360 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.839s	user 0.042s	sys 0.082s
I20260709 13:19:11.015652 12747 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:10.912554 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:11.022704 12598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.045s	sys 0.128s
W20260709 13:19:11.079078 12598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.059s	sys 0.168s
I20260709 13:19:10.587085 16168 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.710505 16383 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:10.995621 12774 rpcz_store.cc:276] 0709 13:19:09.047553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.123479 (+ 75926us) service_pool.cc:224] Handling call
0709 13:19:09.148546 (+ 25067us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:09.148822 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:10.292883 (+1144061us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b18003d6000. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6cf5a4bc 00007f4c6a2a848b 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c2455 00007f4c70daa3c7 00007f4c6d3bc9a7 00007f4c6d3cc0bf
0709 13:19:10.293010 (+   127us) log.cc:844] Serialized 14 byte log entry
0709 13:19:10.994673 (+701663us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:10.994745 (+    72us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:10.994789 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.994865 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:10.994965 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11545,"prepare.queue_time_us":41595,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":23782,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"spinlock_wait_cycles":56603008,"thread_start_us":726947,"threads_started":3,"wal-append.queue_time_us":110841}
I20260709 13:19:11.080155 16168 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:11.080744 16383 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:10.735337 16100 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:10.833338 16010 consensus_meta.cc:287] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.693s	user 0.010s	sys 0.001s
W20260709 13:19:10.772409 16324 consensus_meta.cc:287] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.750s	user 0.004s	sys 0.006s
W20260709 13:19:10.814107 16449 log.cc:927] Time spent T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
W20260709 13:19:10.771075 16435 log.cc:927] Time spent T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.379s	user 0.000s	sys 0.000s
W20260709 13:19:10.863358 16399 log.cc:927] Time spent T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 1.165s	user 0.004s	sys 0.004s
W20260709 13:19:11.096975 12673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.055s	sys 0.095s
W20260709 13:19:11.097518 12673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.055s	sys 0.095s
I20260709 13:19:11.088636 12775 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: []
W20260709 13:19:10.857501 16451 log.cc:927] Time spent T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.060s	user 0.002s	sys 0.000s
I20260709 13:19:11.089818 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:11.089921 12782 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:10.597731 16354 consensus_queue.cc:1059] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.539s
W20260709 13:19:11.047356 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 59916us
W20260709 13:19:11.093137 12600 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.009s	sys 0.007s
W20260709 13:19:11.105299 12600 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.009s	sys 0.007s
I20260709 13:19:10.918893 16450 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.087126 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:10.997882 16241 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:11.097947 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 594977us
I20260709 13:19:10.767912 15946 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:10.766336 16425 log.cc:927] Time spent T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.304s	user 0.001s	sys 0.000s
I20260709 13:19:11.102421 13382 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
W20260709 13:19:11.079388 16360 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.004s	user 0.053s	sys 0.088s
I20260709 13:19:10.852286 16461 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:10.770597 16251 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:11.090565 13376 raft_consensus.cc:1275] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:19:10.920588 16236 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:10.840179 16402 consensus_meta.cc:287] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.710s	user 0.012s	sys 0.000s
W20260709 13:19:11.092571 13351 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.008s	sys 0.007s
W20260709 13:19:11.131615 13351 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.008s	sys 0.008s
W20260709 13:19:10.767028 16436 log.cc:927] Time spent T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.357s	user 0.000s	sys 0.001s
I20260709 13:19:11.014825 16001 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:10.998658 16306 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:11.022387 16452 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.107961 12776 raft_consensus.cc:1240] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:10.673527 16443 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:11.081463 16383 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:10.868585 16428 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:11.152331 16383 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:11.116182 16450 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:10.834241 15878 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:11.112793 12673 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:11.122949 12782 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:11.155006 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:11.155833 13376 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
I20260709 13:19:10.738379 16397 consensus_queue.cc:1059] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.371s
I20260709 13:19:11.090305 16432 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:11.126952 12783 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:11.089394 16100 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:11.162019 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 49330.2us
I20260709 13:19:11.136420 12775 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:36478
I20260709 13:19:11.112389 16409 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.140882 16306 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:11.165952 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:11.131951 16433 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:11.166688 12776 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:11.082854 16324 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:11.145741 16001 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:11.128361 16354 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:11.133507 16282 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.922s	user 0.016s	sys 0.026s
I20260709 13:19:11.163430 12775 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:11.174962 12775 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:11.176316 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:11.158867 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 143310us
I20260709 13:19:11.180004 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:11.178921 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:11.181423 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 75923.5us
W20260709 13:19:11.174787 16282 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.967s	user 0.017s	sys 0.026s
I20260709 13:19:11.116739 16241 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:11.173965 15878 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:11.158042 16308 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:11.182927 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 77262.3us
I20260709 13:19:11.135246 16236 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:11.156877 16270 consensus_queue.cc:1059] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:19:11.220534 16439 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:11.221086 16456 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:11.221297 16439 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:11.221596 16456 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:11.223609 16456 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:11.223985 16439 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:11.149657 16443 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:11.224746 16316 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:11.225392 16316 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:11.226246 16443 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:11.182178 16354 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:11.227800 16316 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:11.228806 16354 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:11.220609 16407 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:11.229955 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 120966us
I20260709 13:19:11.166669 16432 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:11.240151 12597 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:11.242565 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 63787.9us
W20260709 13:19:11.224342 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.243498 12747 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:11.242405 12775 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:11.233212 16432 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:11.287467 12598 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 2.019s (SENT) [suppressed 10 similar messages]
I20260709 13:19:11.133016 16461 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:11.201496 15878 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:11.365128 12775 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:11.174769 16409 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:11.368503 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 125114us
I20260709 13:19:11.381501 12747 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:11.367136 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:11.365783 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 123693us
I20260709 13:19:11.379984 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:11.385429 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:11.383898 12597 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:11.384028 12782 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:11.390516 12747 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:11.217206 16454 consensus_queue.cc:1059] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.206s
I20260709 13:19:11.396143 16428 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.398483 16454 raft_consensus.cc:3055] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:11.399686 16454 raft_consensus.cc:740] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:11.401280 16454 consensus_queue.cc:260] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:11.396926 16428 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:11.403199 16359 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:11.404042 16354 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:11.166290 16452 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:11.409128 16454 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:11.422049 12629 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:11.423221 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:11.410929 16359 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:11.417452 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:11.417853 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:11.378341 16321 raft_consensus.cc:3055] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:11.219234 16308 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:11.381451 15878 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:11.384644 16439 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:11.433167 12630 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:11.413064 16198 raft_consensus.cc:3055] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:11.445057 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:11.444948 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:11.447644 12782 raft_consensus.cc:1240] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 5. Current term is 7. Ops: []
I20260709 13:19:11.405998 16251 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:11.412549 16354 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:11.381234 16361 consensus_queue.cc:1059] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:19:11.467835 12633 raft_consensus.cc:2393] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 10: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:11.381992 15812 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 13:19:11.424126 16313 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.472922 16313 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:11.476732 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:11.477800 12634 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:11.429497 16236 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:11.429172 16168 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:11.432005 16321 raft_consensus.cc:740] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:11.479197 16313 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:11.490150 16472 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:11.463636 16198 raft_consensus.cc:740] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:11.493798 16462 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:11.494937 16457 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:11.494525 16198 consensus_queue.cc:260] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:11.496455 16198 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:11.497831 16431 log.cc:927] Time spent T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:19:11.498711 16354 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:11.464859 16433 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:11.499591 16236 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.508562 16469 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:11.508337 16236 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:11.509929 16466 log.cc:927] Time spent T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:19:11.510607 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:11.512044 16433 raft_consensus.cc:697] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:11.513331 16433 consensus_queue.cc:237] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:11.367854 16407 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:11.511220 13383 raft_consensus.cc:1813] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Handling vote request from an unknown peer 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:11.523571 13383 raft_consensus.cc:2393] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 6: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:11.444789 16355 consensus_queue.cc:1059] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:19:11.525149 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:11.527469 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:11.488233 16397 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.531661 16321 consensus_queue.cc:260] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:11.532974 16321 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:11.535609 16450 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:11.517732 16452 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:11.537668 16308 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:11.538795 16001 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:11.538885 16324 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:11.539707 16100 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:11.540453 16428 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:11.542217 16359 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:11.467164 16439 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:11.554119 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 266677us
I20260709 13:19:11.551565 16397 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:11.567068 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 176639us
I20260709 13:19:11.551734 13376 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
W20260709 13:19:11.550851 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 258137us
I20260709 13:19:11.525259 16472 raft_consensus.cc:697] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:11.583173 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1584) took 1286 ms. Trace:
I20260709 13:19:11.573961 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:11.587414 16100 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:11.588008 12747 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:11.588060 16100 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:11.590404 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:11.511201 16236 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:11.583806 12702 rpcz_store.cc:276] 0709 13:19:10.296428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.296548 (+   120us) service_pool.cc:224] Handling call
0709 13:19:10.296834 (+   286us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:10.308574 (+ 11740us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:10.325403 (+ 16829us) log.cc:844] Serialized 14 byte log entry
0709 13:19:11.152703 (+827300us) spinlock_profiling.cc:243] Waited 6.63 ms on lock 0x7b180038e600. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf5223a 00007f4c6d3bca2a
0709 13:19:11.521317 (+368614us) log.cc:844] Serialized 14 byte log entry
0709 13:19:11.522421 (+  1104us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:11.523143 (+   722us) log.cc:844] Serialized 52 byte log entry
0709 13:19:11.523313 (+   170us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:11.523359 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:11.523451 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:11.582883 (+ 59432us) raft_consensus.cc:1685] finished
0709 13:19:11.582909 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.583085 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28553,"prepare.queue_time_us":16788,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":31,"raft.queue_time_us":114790,"raft.run_cpu_time_us":425,"raft.run_wall_time_us":27853,"spinlock_wait_cycles":15244672,"thread_start_us":246523,"threads_started":2,"wal-append.queue_time_us":1170818}
I20260709 13:19:11.595978 12747 leader_election.cc:400] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 11. Current term is 12.
I20260709 13:19:11.596436 12747 leader_election.cc:403] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:19:11.559345 16270 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:19:11.597891 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:11.598692 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:11.597020 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 213228us
W20260709 13:19:11.609372 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.610577 12775 raft_consensus.cc:1240] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:11.601720 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:11.612381 12702 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:19:11.614059 12776 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:11.614173 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:11.614841 12775 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:19:11.589107 16100 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:11.623040 12776 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:19:11.621178 12775 raft_consensus.cc:1240] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:19:11.622285 13383 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:19:11.625236 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:11.593796 16470 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:11.627547 12709 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:19:11.629998 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:11.632225 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:11.638933 12704 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:11.639559 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:11.640133 12629 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:19:11.640884 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:11.641021 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:11.641415 12629 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
W20260709 13:19:11.641835 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.634709 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:11.607993 16324 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:11.642570 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:11.643038 12775 raft_consensus.cc:2410] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 11 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:19:11.608232 16001 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 11. Current term is 12.
I20260709 13:19:11.648298 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:11.649128 13385 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:19:11.652030 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:11.652606 12629 raft_consensus.cc:3055] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:11.652891 12629 raft_consensus.cc:740] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:11.654359 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:11.653934 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:11.659829 12704 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:11.661906 12631 raft_consensus.cc:2393] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 11: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:11.654851 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:11.664633 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:11.665480 12709 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:19:11.675652 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:11.619148 16400 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:19:11.643162 16481 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:11.656751 12629 consensus_queue.cc:260] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:11.681831 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:11.694501 12709 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:11.694635 12629 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:11.682922 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:11.695302 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:11.683656 12701 raft_consensus.cc:1275] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:19:11.696082 12625 raft_consensus.cc:2393] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 10: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:11.684963 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:11.685694 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:11.652141 16469 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.690847 16461 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:11.678676 16355 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.714169 12709 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:19:11.695845 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:11.684275 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:11.619848 16270 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:11.647691 15812 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:11.699612 12631 raft_consensus.cc:1275] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:19:11.722231 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:11.700426 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:11.703438 16471 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:11.667321 16324 raft_consensus.cc:697] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:11.663157 16409 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:11.720566 12703 raft_consensus.cc:2393] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 10: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:11.713606 16152 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:11.693068 16481 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:11.739346 13349 leader_election.cc:336] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.650s (SENT)
I20260709 13:19:11.739284 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:11.740871 12631 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
W20260709 13:19:11.742033 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 82893.2us
W20260709 13:19:11.747988 13349 leader_election.cc:336] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.761s (SENT)
I20260709 13:19:11.757082 16457 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:11.709578 16479 consensus_queue.cc:1059] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:19:11.647940 16470 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:11.762868 16270 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:11.764042 16439 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:11.727607 15812 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:11.719302 16355 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:11.667001 16472 consensus_queue.cc:237] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:11.772516 12707 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
W20260709 13:19:11.774294 13349 leader_election.cc:336] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.882s (SENT)
I20260709 13:19:11.793896 13349 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:11.784308 13381 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:11.724822 16469 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:11.791040 16407 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:11.798501 16306 consensus_meta.cc:287] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.619s	user 0.012s	sys 0.000s
I20260709 13:19:11.799985 12746 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:11.796952 16321 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:11.742405 16152 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:11.805853 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 53831.7us
W20260709 13:19:11.806365 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.810854 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:11.810978 13382 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:11.812685 16481 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:11.812952 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:11.782111 16324 consensus_queue.cc:237] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:11.813678 16219 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:11.815517 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:11.798683 15952 raft_consensus.cc:3055] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:11.807296 16270 raft_consensus.cc:3055] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:11.819428 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:11.821237 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:11.823390 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:11.814255 16219 raft_consensus.cc:697] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:11.843966 15931 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:11.790490 16100 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:11.846858 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.846567 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:11.846029 15931 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:11.849843 16439 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:11.850184 16491 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:11.851065 16107 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.850795 16491 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:11.802937 15878 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:11.824643 15952 raft_consensus.cc:740] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:11.857234 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:11.806025 16306 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:11.842496 16270 raft_consensus.cc:740] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:11.858969 12631 raft_consensus.cc:1240] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:11.850419 16439 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:11.806958 16407 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:19:11.858453 16100 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:11.873358 16485 log.cc:927] Time spent T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:19:11.873852 15592 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:11.853901 16479 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.870355 12597 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:11.871667 16241 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:11.872709 16490 log.cc:927] Time spent T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:19:11.873149 16152 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:11.875584 16355 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:11.880703 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:11.867367 16470 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:11.883942 12702 raft_consensus.cc:2368] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 11. Current term is 12.
I20260709 13:19:11.877028 12629 raft_consensus.cc:2410] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:19:11.884799 13349 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:11.891207 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:11.891964 12776 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:19:11.893056 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:11.894937 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:11.895514 12774 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:11.896778 12783 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:19:11.883122 16469 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:11.899619 12782 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:11.862637 15952 consensus_queue.cc:260] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:11.904561 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.905278 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:11.906167 15931 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:11.906384 16495 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:11.882655 16370 consensus_queue.cc:260] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:11.911811 13385 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 10: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:11.912734 16480 consensus_queue.cc:1059] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:19:11.913108 12627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 555) took 1709 ms. Trace:
I20260709 13:19:11.857781 16107 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:11.917032 16430 consensus_queue.cc:1059] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:19:11.922266 16430 raft_consensus.cc:3055] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:11.922632 16430 raft_consensus.cc:740] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:11.922976 16107 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:11.874985 16219 consensus_queue.cc:237] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:11.923565 16430 consensus_queue.cc:260] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:11.926285 15952 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:11.921247 12627 rpcz_store.cc:276] 0709 13:19:10.203918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.205795 (+  1877us) service_pool.cc:224] Handling call
0709 13:19:10.206098 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:10.414723 (+208625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:10.414758 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:10.415193 (+   435us) log.cc:844] Serialized 26 byte log entry
0709 13:19:10.536254 (+121061us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:10.581532 (+ 45278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.833734 (+252202us) spinlock_profiling.cc:243] Waited 4.11 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:11.912751 (+1079017us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b54002ebdd4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:11.912808 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:11.912868 (+    60us) raft_consensus.cc:1685] finished
0709 13:19:11.912883 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:11.913043 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21265,"raft.queue_time_us":21400,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":16670,"spinlock_wait_cycles":439396992,"thread_start_us":120725,"threads_started":1,"wal-append.queue_time_us":314159}
I20260709 13:19:11.917097 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:11.878005 15592 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:11.929278 12783 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:19:11.918711 16270 consensus_queue.cc:260] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:11.927851 16491 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:11.928910 16495 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:11.915995 15812 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:11.934499 16483 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:11.940945 15878 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:11.942452 16306 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:11.903833 16479 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:11.943354 16370 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:11.948038 16495 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:11.958644 16270 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:11.969837 16469 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.970505 16469 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:11.971807 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 79204.7us
I20260709 13:19:11.972601 16469 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:11.934996 15592 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:11.973495 12597 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:11.979313 15812 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:11.980630 12597 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:11.981549 15812 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:11.982007 15812 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:11.982978 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:11.982349 15812 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:11.990399 12597 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:11.991200 16479 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:11.998311 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.999266 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:12.003588 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:11.993525 16321 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:12.008416 16441 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:19:12.009546 15592 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:12.011384 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:12.012813 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:12.013468 12783 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:11.989128 16497 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:12.016346 16497 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:12.019207 16497 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:12.021481 12774 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:19:12.023795 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:12.031873 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:12.032496 12774 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:19:12.017194 16479 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:12.035429 12597 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:12.044798 12597 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:19:12.049228 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.046198 16107 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:12.054877 16107 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:12.055279 16107 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:12.055603 13384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 688) took 1384 ms. Trace:
I20260709 13:19:12.056003 13384 rpcz_store.cc:276] 0709 13:19:10.671305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.672759 (+  1454us) service_pool.cc:224] Handling call
0709 13:19:10.673111 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:11.230140 (+557029us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:11.230171 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:11.230215 (+    44us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:11.230248 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.055381 (+825133us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b5400498c54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:12.055425 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:12.055547 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1280489216}
I20260709 13:19:12.051770 16395 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:12.058835 16395 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:12.059237 16395 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:12.061846 15878 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.062528 15878 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:12.005694 16480 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:19:12.042584 16479 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:12.078182 12633 raft_consensus.cc:1240] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 9. Current term is 10. Ops: []
W20260709 13:19:12.091831 16496 log.cc:927] Time spent T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 13:19:12.096149 16306 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.096783 16306 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:12.098999 16306 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:12.099602 16306 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.100199 16306 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:12.104046 16286 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:12.065624 15878 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:12.107297 16354 raft_consensus.cc:3055] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:12.107831 16354 raft_consensus.cc:740] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:12.108637 16354 consensus_queue.cc:260] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:12.111223 16479 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:12.104543 15812 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:12.052721 16069 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.073s
W20260709 13:19:12.188138 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:12.187602 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.215193 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:12.211158 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.206244 12747 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:12.202514 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:12.223392 12626 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
W20260709 13:19:12.223294 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.202864 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:12.214756 12704 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:19:12.202140 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:12.219834 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:12.215621 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.210466 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:12.231997 12627 raft_consensus.cc:2410] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:19:12.232372 12701 raft_consensus.cc:2410] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:19:12.219463 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.168889 13379 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:12.210153 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.210078 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.219101 12700 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:19:12.224975 12673 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:12.113245 16270 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:12.235930 12703 raft_consensus.cc:2410] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:19:12.207859 16397 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:12.228116 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.236727 12747 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:12.237479 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.123581 16439 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:12.238783 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.143198 16354 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:12.176918 15878 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.226076 16252 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:12.229843 16234 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:12.242022 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:12.242738 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:12.243124 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:12.244766 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.258922 12703 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:12.262042 12673 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:12.245256 16241 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:12.246057 12747 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:12.236450 16379 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.267246 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:12.247043 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:12.247888 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:12.194170 16306 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:12.285456 12632 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:12.198300 16453 raft_consensus.cc:3055] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:12.293156 16453 raft_consensus.cc:740] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:12.294049 16453 consensus_queue.cc:260] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:12.250402 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.295876 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:12.297012 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.252864 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:12.258335 12708 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:12.238150 16219 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:12.269869 12626 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:12.270174 16127 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:12.250478 15952 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:12.266382 16495 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:12.298745 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:12.275014 16479 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:12.275049 16395 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:12.268425 16234 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:12.276785 16107 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:12.254881 16252 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:12.277966 16484 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:12.278230 16241 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:12.278702 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:12.258977 15878 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:12.336184 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:12.282341 16379 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:12.283285 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.271806 16497 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:12.297544 12705 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:12.297962 13377 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 10 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:12.298365 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.312451 13384 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:12.300048 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:12.313346 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.313203 12632 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:12.304286 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:12.338477 13380 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:12.338778 13382 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:12.308735 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:12.339141 12673 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:12.339447 12776 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:12.336944 16241 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:12.340493 16241 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:12.340826 16497 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:12.341461 16234 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:12.280575 16010 raft_consensus.cc:2955] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Committing config change with OpId 4.5: config changed from index 4 to 5, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:12.342584 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:12.308751 12708 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:12.309147 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:12.300431 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:12.310426 16127 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:12.344463 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.316085 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.303350 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:12.302029 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:12.345216 12701 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:12.338570 12704 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 6.
I20260709 13:19:12.317700 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:12.336416 13349 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:12.324004 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:12.325169 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:12.271850 12517 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 981fb2f435e24d06a81fc02cf4591cc5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 19cd864c053744bebd96093a7982537e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:19:12.330994 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.332199 13376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51762 (request call id 645) took 1088 ms. Trace:
I20260709 13:19:12.310904 15952 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:12.348461 12775 raft_consensus.cc:2393] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 10: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:12.334625 12709 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
W20260709 13:19:12.335067 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 56448.7us
I20260709 13:19:12.349464 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:12.349715 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.335495 12626 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:12.336715 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:12.350775 13378 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:12.351876 12673 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:12.352855 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.353818 15878 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:12.354837 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:12.355434 12700 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:12.355765 15952 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:12.356706 16430 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:12.357642 16306 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:12.358305 12625 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:19:12.358405 12782 raft_consensus.cc:2393] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 13: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:12.359582 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:12.337590 12633 raft_consensus.cc:1240] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:12.308394 12783 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12. Ops: []
W20260709 13:19:12.306420 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.346150 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:12.365191 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:12.365912 12701 raft_consensus.cc:2376] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8. Re-sending same reply.
I20260709 13:19:12.366475 12747 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:12.367238 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.347185 12777 raft_consensus.cc:2393] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:12.368701 12701 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:19:12.369583 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:12.370692 12701 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:12.347147 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:12.347330 12517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 121392us
I20260709 13:19:12.347422 13376 rpcz_store.cc:276] 0709 13:19:11.244304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.366778 (+122474us) service_pool.cc:224] Handling call
0709 13:19:12.332033 (+965255us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b54005d9654. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:12.332141 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":387454720}
I20260709 13:19:12.380470 12700 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:12.386256 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:12.354657 16310 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:12.381417 16454 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.389801 16100 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:12.390025 16495 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:12.390394 16100 raft_consensus.cc:697] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:12.391024 12780 raft_consensus.cc:2393] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 9: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:12.391292 16100 consensus_queue.cc:237] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:12.393357 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:12.393747 16311 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:12.394799 12630 raft_consensus.cc:1240] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:12.397843 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:12.389122 16454 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:12.398280 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:12.398419 12708 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:12.335598 16252 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:12.398802 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:12.399942 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:12.400943 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:12.403563 13379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51762 (request call id 646) took 1118 ms. Trace:
I20260709 13:19:12.403947 13379 rpcz_store.cc:276] 0709 13:19:11.292115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.364917 (+ 72802us) service_pool.cc:224] Handling call
0709 13:19:12.403398 (+1038481us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b5400512f54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:12.403490 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1799845248}
I20260709 13:19:12.400013 16454 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:12.406745 16395 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:12.407974 16306 raft_consensus.cc:697] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:19:12.408473 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 56649.4us
I20260709 13:19:12.408874 16306 consensus_queue.cc:237] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:12.409492 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:12.410064 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.410888 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:12.412720 16282 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:12.414434 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:12.415055 16423 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.415522 16423 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:12.417061 16469 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:12.398828 12703 raft_consensus.cc:2376] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12. Re-sending same reply.
I20260709 13:19:12.419827 12631 raft_consensus.cc:1240] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:12.419257 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:12.420214 12701 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:12.420585 12628 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:12.421146 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:12.421224 12672 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:12.422137 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:12.423245 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:12.423631 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:12.424312 12701 raft_consensus.cc:2368] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7.
I20260709 13:19:12.424433 16357 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:12.424944 16454 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:12.425395 16355 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:12.417423 16423 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:12.428409 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:12.430115 16107 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:12.334460 16484 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:12.344359 16219 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:12.383561 16407 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:12.431164 12780 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 9. Current term is 10. Ops: [7.8-9.9]
I20260709 13:19:12.434588 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:12.470959 12781 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:19:12.437825 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:12.439512 12708 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:12.448977 12782 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:19:12.445120 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.453456 16355 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:12.462625 16423 consensus_queue.cc:1048] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:12.453681 16357 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:12.348593 16127 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:12.465682 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:12.440075 12706 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:12.468438 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.466747 16219 raft_consensus.cc:697] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:12.482997 12709 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:12.453030 12707 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:12.708653 12626 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:19:12.711760 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:12.611578 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:12.613492 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.732424 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.732587 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.733439 12706 raft_consensus.cc:2393] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 7: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:12.733850 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:12.734967 12706 raft_consensus.cc:3055] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:12.735507 12747 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:12.735824 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.734064 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.735518 12706 raft_consensus.cc:740] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:12.739523 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.737979 12706 consensus_queue.cc:260] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:12.742161 12747 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:12.467182 16484 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:12.752526 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:12.665385 16453 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:12.769408 15998 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.770066 15998 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:12.720172 16452 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:12.771122 16452 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:12.772433 15998 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:12.773092 16452 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:12.641115 15952 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:12.475072 16379 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:12.774501 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.792047 12783 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:12.792807 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:12.793099 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:12.793541 12634 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
I20260709 13:19:12.794627 12704 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:19:12.720741 16355 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:12.471527 15812 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:12.770432 16469 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:12.804498 12778 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:12.719904 16456 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:12.780545 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.783821 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:12.784578 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.785903 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:12.814913 12782 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:12.815614 12775 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:19:12.789036 12777 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:19:12.816419 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:12.791379 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:12.774837 16234 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:19:12.800807 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:12.826387 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:12.826956 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.827085 12704 raft_consensus.cc:1275] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
W20260709 13:19:12.828159 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.828382 12700 raft_consensus.cc:2368] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12.
I20260709 13:19:12.829061 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:12.829717 12632 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:12.830071 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:12.830725 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:12.831081 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:12.832230 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:12.805720 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.463877 16454 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:12.807291 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:12.810694 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:12.768114 16311 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:12.798681 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:12.836865 12778 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:12.821707 12703 raft_consensus.cc:2368] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7.
I20260709 13:19:12.817325 13380 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:19:12.803937 16469 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:12.842324 12703 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:12.843637 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:12.844786 12628 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:12.795470 16458 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:12.846340 12627 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:12.742348 16241 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:12.838964 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:12.849298 12703 raft_consensus.cc:1240] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:12.840442 12702 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:19:12.850124 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:12.851889 12703 raft_consensus.cc:1240] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:12.814040 16479 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:12.855959 12702 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:19:12.852603 12707 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:19:12.857807 12702 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:19:12.861296 12703 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:19:12.863729 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:12.847572 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:12.832626 16234 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:12.868316 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:12.848404 16469 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:12.873956 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:12.849045 16402 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:12.875098 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:12.849296 16282 raft_consensus.cc:697] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:12.825309 16379 consensus_queue.cc:1059] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:19:12.874455 12707 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:12.883441 16357 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:12.826387 16456 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:12.885949 12775 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:12.890424 12778 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:12.891605 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:19:12.893193 16430 consensus_meta.cc:287] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.536s	user 0.009s	sys 0.001s
I20260709 13:19:12.897929 16423 consensus_queue.cc:1059] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:12.898797 16423 consensus_queue.cc:1243] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Peer 522fd2ba0e9f402691b5ef29af8579cd log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:19:12.856437 16458 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:12.904239 16458 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:12.906562 12782 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:19:12.907927 16115 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.909901 16115 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:12.914719 12775 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:12.914976 16241 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:12.916841 12598 leader_election.cc:336] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.516s (SENT)
I20260709 13:19:12.873848 16310 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:12.876281 16234 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:12.941282 13384 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:12.948354 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:12.952239 12782 raft_consensus.cc:1275] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:12.929221 16310 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:12.841377 16219 consensus_queue.cc:237] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:12.967522 12672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.013s	sys 0.013s
W20260709 13:19:12.967988 12672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.013s	sys 0.013s
W20260709 13:19:12.969080 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 546325us
I20260709 13:19:12.970032 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:12.979951 12672 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:12.968638 16469 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:12.984205 12672 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:12.968329 15998 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.964540 16310 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:12.986788 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:12.988346 12707 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:19:12.985790 12672 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:12.988164 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:12.988163 12673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.036s	sys 0.088s
W20260709 13:19:12.989445 12673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.036s	sys 0.088s
W20260709 13:19:12.991145 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 579846us
I20260709 13:19:12.989336 12700 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:12.997705 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:13.035915 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:12.989480 12672 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:13.037402 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:13.038046 12626 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:13.042171 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 52783us
I20260709 13:19:13.042621 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:13.043429 12707 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:19:13.043603 12672 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:13.037937 12631 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:19:13.050591 12626 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:12.986382 15998 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:13.053122 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:13.105163 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 51476.1us
I20260709 13:19:13.106596 12597 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:13.107626 16479 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:13.108258 16479 raft_consensus.cc:697] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:13.109385 16479 consensus_queue.cc:237] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:13.035512 16483 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:13.003331 15952 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:13.129647 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:12.982102 16282 consensus_queue.cc:237] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:13.072495 16430 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:13.150916 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.051594 16397 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.095s
I20260709 13:19:13.136148 16469 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:13.180713 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 74188.1us
I20260709 13:19:13.150838 15952 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:13.199095 15998 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:13.217990 16508 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:13.128158 16483 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:13.152031 16454 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:13.222462 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.226676 12597 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:19:13.225847 16252 consensus_meta.cc:287] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.827s	user 0.011s	sys 0.000s
I20260709 13:19:13.226953 15952 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:13.245858 16508 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:13.159539 16430 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:13.252203 12597 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:13.205029 16234 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:13.257228 16252 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:13.257658 15998 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:13.257254 16454 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:13.258762 15952 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:13.261894 16354 consensus_meta.cc:287] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 1.008s	user 0.012s	sys 0.000s
I20260709 13:19:13.260533 16508 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:19:13.273025 12749 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.010s	sys 0.017s
I20260709 13:19:13.274132 16480 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.268033 16397 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:13.284554 12708 raft_consensus.cc:2410] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 13:19:13.273450 16100 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.004s	sys 0.024s
I20260709 13:19:13.271023 16407 raft_consensus.cc:3055] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:13.286890 16452 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.287386 16480 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:13.270663 16453 consensus_queue.cc:1048] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
W20260709 13:19:13.298187 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 481818us
I20260709 13:19:13.284694 16508 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:13.284111 15998 raft_consensus.cc:697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:13.287758 16354 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:19:13.300073 16100 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.004s	sys 0.024s
I20260709 13:19:13.291184 16397 raft_consensus.cc:697] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:19:13.292057 16010 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.012s	sys 0.018s
W20260709 13:19:13.301759 16306 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.004s	sys 0.026s
W20260709 13:19:13.312556 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.297389 16407 raft_consensus.cc:740] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:19:13.319141 12748 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.008s	sys 0.023s
W20260709 13:19:13.358681 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 315114us
I20260709 13:19:13.375627 12672 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
W20260709 13:19:13.379786 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:13.312947 16010 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.012s	sys 0.018s
W20260709 13:19:13.384285 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 65569.3us
W20260709 13:19:13.319754 12748 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.008s	sys 0.023s
W20260709 13:19:13.314169 16306 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.004s	sys 0.026s
I20260709 13:19:13.384956 12673 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:13.301962 16452 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:13.388955 16100 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:13.419299 16100 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:13.421418 16100 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:13.389163 12702 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:19:13.422382 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:13.423606 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:13.390342 16306 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:13.424305 16306 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.424335 12627 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:19:13.424811 16306 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:13.425977 16456 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:13.427907 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.428623 16402 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.429021 16115 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:13.390431 16509 consensus_queue.cc:1059] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:13.429292 16402 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:13.430048 16509 raft_consensus.cc:3055] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:13.430418 16509 raft_consensus.cc:740] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:13.430558 12708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1952) took 1171 ms. Trace:
I20260709 13:19:13.431203 16509 consensus_queue.cc:260] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:13.391899 12673 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:13.432443 16509 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:13.392668 16507 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:13.433205 16507 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:13.433614 16507 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:13.434078 12778 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:13.391757 16510 consensus_queue.cc:1059] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:13.435035 13379 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:19:13.435253 16456 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:13.435660 16456 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.394260 12672 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:13.413244 15998 consensus_queue.cc:237] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:13.301640 16310 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:13.423820 12630 raft_consensus.cc:1240] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:13.389724 16354 consensus_queue.cc:1059] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:13.431614 12704 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:19:13.437887 12704 pending_rounds.cc:85] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 6
I20260709 13:19:13.438135 12704 pending_rounds.cc:107] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:19:13.412528 16234 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:13.438477 12704 raft_consensus.cc:2889] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:13.436034 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:13.435972 16456 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:13.430927 12708 rpcz_store.cc:276] 0709 13:19:12.259689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.733850 (+474161us) service_pool.cc:224] Handling call
0709 13:19:13.430345 (+696495us) spinlock_profiling.cc:243] Waited 547 ms on lock 0x7b5400109ad4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:13.430498 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1258653952}
I20260709 13:19:13.444782 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:13.431658 16402 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:13.454398 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:13.457957 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:13.459009 13379 raft_consensus.cc:2410] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:19:13.459702 16107 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.008s	sys 0.008s
I20260709 13:19:13.436942 16432 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:13.468297 12775 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.021s	sys 0.035s
W20260709 13:19:13.470300 12775 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.021s	sys 0.035s
W20260709 13:19:13.460053 16107 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.008s	sys 0.008s
I20260709 13:19:13.389113 16397 consensus_queue.cc:237] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:13.448743 16252 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:13.452185 16483 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:13.482491 16321 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.009s	sys 0.008s
I20260709 13:19:13.469169 16430 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:13.515291 12598 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.012s	sys 0.008s
W20260709 13:19:13.515646 12703 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.010s	sys 0.008s
W20260709 13:19:13.515898 12747 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.010s	sys 0.029s
I20260709 13:19:13.449829 16290 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:19:13.517858 16252 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:13.521481 16511 log.cc:927] Time spent T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:19:13.531206 12598 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.012s	sys 0.008s
W20260709 13:19:13.517608 16321 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.010s	sys 0.008s
W20260709 13:19:13.531859 12703 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.011s	sys 0.008s
I20260709 13:19:13.577327 12706 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:13.456702 16407 consensus_queue.cc:260] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:13.588836 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 650262us
W20260709 13:19:13.531556 12747 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.010s	sys 0.029s
W20260709 13:19:13.623242 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 870725us
W20260709 13:19:13.546890 16484 consensus_meta.cc:287] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.755s	user 0.006s	sys 0.006s
W20260709 13:19:13.515091 15812 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.007s	sys 0.011s
W20260709 13:19:13.606572 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.619622 16310 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:13.545668 16430 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:19:13.620880 16355 consensus_meta.cc:287] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.803s	user 0.009s	sys 0.000s
I20260709 13:19:13.493633 16432 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:13.632965 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:13.635838 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.634838 12747 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:13.629738 15812 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.007s	sys 0.011s
I20260709 13:19:13.641973 12776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 485) took 1207 ms. Trace:
I20260709 13:19:13.642549 12747 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:13.626701 16529 raft_consensus.cc:2764] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:19:13.643698 12776 rpcz_store.cc:276] 0709 13:19:12.434512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.434617 (+   105us) service_pool.cc:224] Handling call
0709 13:19:12.434899 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:12.435288 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:12.435324 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:12.436085 (+   761us) log.cc:844] Serialized 26 byte log entry
0709 13:19:12.731971 (+295886us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b1800321b80. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf520d1 00007f4c6d36da0f
0709 13:19:13.165094 (+433123us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:13.165154 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:13.165254 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:13.638205 (+472951us) raft_consensus.cc:1685] finished
0709 13:19:13.638229 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:13.641899 (+  3670us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":87,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":7772,"spinlock_wait_cycles":23822720,"thread_start_us":432894,"threads_started":1,"wal-append.queue_time_us":881575}
I20260709 13:19:13.599643 16252 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:13.648190 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 254019us
I20260709 13:19:13.631948 16430 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.653414 16515 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:13.659170 16515 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:13.659556 16515 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:13.660207 16252 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:13.662147 16452 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:13.664250 16127 consensus_meta.cc:287] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.924s	user 0.010s	sys 0.000s
I20260709 13:19:13.664758 16127 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:13.641350 16290 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:19:13.637924 16480 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:13.642282 16441 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:13.647547 16219 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:19:13.674803 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 241893us
I20260709 13:19:13.684999 12706 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:19:13.689955 12705 raft_consensus.cc:2393] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:13.630383 16484 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:13.701098 12706 pending_rounds.cc:85] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 8
I20260709 13:19:13.703531 12706 pending_rounds.cc:107] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:19:13.707029 12706 raft_consensus.cc:2889] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:13.672602 12672 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:13.639165 16355 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:13.711078 12747 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:13.706913 12673 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:13.665512 16432 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:13.710899 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 268860us
I20260709 13:19:13.712028 12706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 453) took 1500 ms. Trace:
I20260709 13:19:13.669920 16430 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:13.721094 12780 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:19:13.718823 12673 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:13.718241 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 255502us
I20260709 13:19:13.724364 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:13.729385 12709 raft_consensus.cc:2368] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11.
I20260709 13:19:13.729398 12747 leader_election.cc:400] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 11. Current term is 12.
I20260709 13:19:13.730038 12747 leader_election.cc:403] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:13.725651 12597 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:19:13.736002 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.736073 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:13.737038 12626 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:13.719347 12706 rpcz_store.cc:276] 0709 13:19:12.211792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.734486 (+522694us) service_pool.cc:224] Handling call
0709 13:19:12.734835 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:13.577022 (+842187us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b5800611cd0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d30e0aa 00007f4c6d390447 00007f4c6d3a8fe7 00007f4c6d3aa7de 00007f4c6d3b4b49 00007f4c6d3b5416 00007f4c6d3b1d8f 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4
0709 13:19:13.709645 (+132623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:13.711973 (+  2328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":523663232}
I20260709 13:19:13.687796 16441 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:13.781744 12597 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:13.791172 12705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1947) took 1542 ms (client timeout 1961 ms). Trace:
W20260709 13:19:13.791810 12705 rpcz_store.cc:269] 0709 13:19:12.248385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.337530 (+ 89145us) service_pool.cc:224] Handling call
0709 13:19:13.790924 (+1453394us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b540011c1d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:13.791102 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3072862080}
I20260709 13:19:13.731796 16452 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:13.752518 16432 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:13.802865 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.809655 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:13.772502 16234 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:13.812132 12701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 454) took 1600 ms. Trace:
W20260709 13:19:13.813030 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.772226 16107 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:13.819115 12701 rpcz_store.cc:276] 0709 13:19:12.212042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.754038 (+541996us) service_pool.cc:224] Handling call
0709 13:19:12.754451 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:12.754702 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:13.292716 (+538014us) log.cc:844] Serialized 14 byte log entry
0709 13:19:13.807948 (+515232us) log.cc:844] Serialized 14 byte log entry
0709 13:19:13.811702 (+  3754us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:13.811767 (+    65us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:13.811826 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:13.811922 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:13.812066 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12985,"prepare.queue_time_us":763843,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":96,"raft.queue_time_us":156219,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":4741,"thread_start_us":998039,"threads_started":3,"wal-append.queue_time_us":347105}
I20260709 13:19:13.723075 16516 consensus_queue.cc:1059] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
W20260709 13:19:13.825539 12598 leader_election.cc:336] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 2.019s (SENT)
I20260709 13:19:13.726402 16219 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.081s
W20260709 13:19:13.806864 16514 log.cc:927] Time spent T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 13:19:13.845026 12598 leader_election.cc:336] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 2.080s (SENT)
I20260709 13:19:13.845757 12598 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:13.824049 16234 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:13.850190 16234 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:13.815939 16290 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 11. Current term is 12.
I20260709 13:19:13.812904 16510 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:13.817168 16480 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:13.821606 16357 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:13.821956 16010 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:13.840329 16270 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:13.823751 16107 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.845376 16282 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:13.806931 16306 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:13.892095 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 437725us
W20260709 13:19:13.891422 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.813082 16432 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:13.884774 15812 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:13.889446 16510 raft_consensus.cc:697] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:19:13.896452 16512 log.cc:927] Time spent T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.254s	user 0.000s	sys 0.001s
I20260709 13:19:13.896180 16107 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:13.894112 16469 raft_consensus.cc:3055] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:13.895730 16282 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.902697 16379 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:13.857242 12673 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:13.858906 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 186429us
W20260709 13:19:13.868702 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 139449us
I20260709 13:19:14.062027 12672 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:14.019990 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:14.023737 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:14.096549 12630 raft_consensus.cc:2410] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
W20260709 13:19:13.902860 13349 leader_election.cc:336] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.727s (SENT)
I20260709 13:19:14.060425 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:14.024488 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:19:14.042403 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:14.099406 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 196478us
W20260709 13:19:14.100082 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:14.101199 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:14.043392 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 186139us
I20260709 13:19:14.102515 12672 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:14.054773 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:14.045483 12707 raft_consensus.cc:1240] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
W20260709 13:19:14.104869 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 50618.9us
I20260709 13:19:14.070653 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:14.107214 12707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1613) took 1124 ms. Trace:
I20260709 13:19:14.110347 12707 rpcz_store.cc:276] 0709 13:19:12.983000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:13.425978 (+442978us) service_pool.cc:224] Handling call
0709 13:19:13.426285 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:14.105072 (+678787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.106770 (+  1698us) spinlock_profiling.cc:243] Waited 618 ms on lock 0x7b540006cf54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:14.107125 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1421867904}
W20260709 13:19:14.049283 12709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1606) took 1592 ms (client timeout 1781 ms). Trace:
W20260709 13:19:14.113217 12709 rpcz_store.cc:269] 0709 13:19:12.456312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.735571 (+279259us) service_pool.cc:224] Handling call
0709 13:19:14.048686 (+1313115us) spinlock_profiling.cc:243] Waited 993 ms on lock 0x7b54001b0654. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:14.048996 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2282847616}
I20260709 13:19:14.073901 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:14.115525 12782 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:14.075294 12783 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:14.046964 12703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1611) took 1073 ms. Trace:
I20260709 13:19:14.057360 12775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 2076) took 1130 ms. Trace:
I20260709 13:19:14.117157 12703 rpcz_store.cc:276] 0709 13:19:12.973420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.973550 (+   130us) service_pool.cc:224] Handling call
0709 13:19:12.973928 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:12.974190 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:13.600454 (+626264us) log.cc:844] Serialized 14 byte log entry
0709 13:19:14.045649 (+445195us) log.cc:844] Serialized 14 byte log entry
0709 13:19:14.046029 (+   380us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:14.046105 (+    76us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:14.046237 (+   132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.046407 (+   170us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.046671 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":544070,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":34,"thread_start_us":1069727,"threads_started":2,"wal-append.queue_time_us":329892}
I20260709 13:19:14.118139 12775 rpcz_store.cc:276] 0709 13:19:12.926384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.926866 (+   482us) service_pool.cc:224] Handling call
0709 13:19:12.927250 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:12.927491 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:13.472708 (+545217us) log.cc:844] Serialized 14 byte log entry
0709 13:19:14.056435 (+583727us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:14.056570 (+   135us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:14.056705 (+   135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.056869 (+   164us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.057106 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16762,"prepare.queue_time_us":676163,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":84706,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":3883,"thread_start_us":1092597,"threads_started":3,"wal-append.queue_time_us":422716}
I20260709 13:19:14.030931 12532 catalog_manager.cc:5697] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 4 to 5, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 5 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:14.124393 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:13.960306 16432 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:13.956499 16456 consensus_meta.cc:287] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.514s	user 0.005s	sys 0.005s
I20260709 13:19:14.142038 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:14.142957 16456 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:13.955304 16357 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:13.927703 16484 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:14.147200 16357 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:13.931263 16379 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:14.157655 16555 log.cc:927] Time spent T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 13:19:13.924830 16515 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:13.907955 16127 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:14.162142 16270 consensus_queue.cc:1059] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:14.163224 16354 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:14.163928 16354 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:14.164400 16354 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:14.165809 15998 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.166853 15998 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:14.170159 16564 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:14.171351 16564 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:14.171554 16306 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:13.940624 16509 consensus_meta.cc:287] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.508s	user 0.008s	sys 0.004s
I20260709 13:19:14.174315 16537 consensus_queue.cc:1059] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:14.175348 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:14.178045 16539 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.178294 16561 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:14.179126 16561 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:14.179672 16561 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:14.179325 16539 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:13.967069 16423 raft_consensus.cc:3055] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:14.182219 16423 raft_consensus.cc:740] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:14.183110 16540 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:14.183921 16423 consensus_queue.cc:260] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:14.187090 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:14.187196 16569 log.cc:927] Time spent T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 13:19:14.190037 16069 consensus_queue.cc:1059] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:14.191973 16069 raft_consensus.cc:3055] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:14.192518 16069 raft_consensus.cc:740] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:13.970355 16407 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:14.194132 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.195189 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:13.954964 16510 consensus_queue.cc:237] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:14.199067 16565 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:14.199652 16565 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:14.200107 16565 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:13.916354 16533 log.cc:927] Time spent T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 13:19:14.201725 16423 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:14.202050 16564 consensus_queue.cc:1059] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:14.203632 16516 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:13.974263 16528 raft_consensus.cc:3055] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:14.205102 16564 consensus_queue.cc:1243] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 639ab2db265f4747bb7afb93e833f530 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:19:14.205350 16528 raft_consensus.cc:740] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:14.207134 16528 consensus_queue.cc:260] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:14.208441 16528 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:14.219082 12626 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:14.219904 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:19:13.949488 16507 consensus_meta.cc:287] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.515s	user 0.008s	sys 0.000s
I20260709 13:19:14.220669 12630 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:14.221580 16507 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:14.222255 16515 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:14.224195 16507 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:13.954277 16441 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:14.226277 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:19:14.161955 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.921424 16282 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:14.228435 12628 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:14.219193 16559 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:14.234361 12626 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:19:14.177117 16537 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:14.232856 12532 catalog_manager.cc:5697] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 4 to 7, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 7 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:14.233799 16355 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:14.243047 12532 catalog_manager.cc:5697] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 6 to 8, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 8 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:14.251694 16539 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:14.252493 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:14.253970 16379 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:14.255405 12629 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:13.918639 16469 raft_consensus.cc:740] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:14.259073 16469 consensus_queue.cc:260] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:14.261620 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 121145us
I20260709 13:19:14.262367 12627 raft_consensus.cc:1240] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:14.262820 16469 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:14.266712 16552 consensus_queue.cc:1059] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:14.269801 16559 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:14.270445 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:14.270742 16552 raft_consensus.cc:3055] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:14.271356 16552 raft_consensus.cc:740] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:14.271853 12633 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 7: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:14.272790 16559 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:14.272614 16552 consensus_queue.cc:260] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:14.275681 12629 raft_consensus.cc:1240] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:14.276552 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:14.280320 12630 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
W20260709 13:19:14.245113 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 88084.8us
I20260709 13:19:14.282855 12625 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
W20260709 13:19:14.290630 16508 consensus_meta.cc:287] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.984s	user 0.007s	sys 0.010s
I20260709 13:19:14.291545 16508 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:14.292647 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:14.294142 16508 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:14.295837 16456 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:14.231496 16540 raft_consensus.cc:3055] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:14.297487 16379 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:14.301939 16540 raft_consensus.cc:740] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:14.302443 16379 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:14.303386 16540 consensus_queue.cc:260] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:14.308785 16069 consensus_queue.cc:260] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:14.310639 16069 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:14.311764 15998 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:14.313939 16537 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:19:14.316949 12529 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54792 (request call id 226) took 29830 ms (client timeout 15000 ms). Trace:
I20260709 13:19:14.318692 16379 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:14.322422 12517 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6609d5f93ab048558ea46e511ef3b2ec with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 19cd864c053744bebd96093a7982537e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:14.245247 12532 catalog_manager.cc:5697] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 5 to 7, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 7 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:14.326529 16354 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:14.328990 16354 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:14.328689 12532 catalog_manager.cc:5697] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 3 to 5, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 5 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:19:14.337690 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 168175us
I20260709 13:19:14.338860 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:14.339747 12625 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:14.339996 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:14.340781 12775 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:14.339066 16552 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:14.343868 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 61016us
I20260709 13:19:14.346629 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:14.347602 16430 consensus_meta.cc:287] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.578s	user 0.007s	sys 0.012s
I20260709 13:19:14.348011 13349 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:14.348106 16430 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:14.349555 16521 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:14.350018 16521 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:14.350158 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.350436 16521 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:14.330155 16567 log.cc:927] Time spent T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.191s	user 0.001s	sys 0.001s
I20260709 13:19:14.318212 16561 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:14.356843 12747 leader_election.cc:400] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11.
I20260709 13:19:14.389472 12747 leader_election.cc:403] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:14.390525 12628 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
W20260709 13:19:14.365751 12774 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.002s	sys 0.019s
W20260709 13:19:14.359150 15952 consensus_meta.cc:287] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 1.075s	user 0.017s	sys 0.000s
W20260709 13:19:14.395857 12774 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.002s	sys 0.021s
I20260709 13:19:14.395750 12673 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:14.367007 16234 consensus_meta.cc:287] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.516s	user 0.014s	sys 0.003s
I20260709 13:19:14.305461 16540 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:14.399091 12625 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:19:14.350775 16430 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:14.402263 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:19:14.404577 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.404503 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:14.401203 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:14.351131 16432 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:14.407565 12673 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:14.373445 16357 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:14.360765 16521 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:14.415247 12775 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:19:14.416885 12783 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:19:14.359712 16107 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:14.401050 16379 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:14.413811 12626 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:14.407900 16234 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:14.412107 16354 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:19:14.422289 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 72779.2us
I20260709 13:19:14.426003 13382 raft_consensus.cc:2393] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 11: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:14.424659 16458 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:14.431138 16219 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:19:14.431489 16458 raft_consensus.cc:697] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 11 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:14.433645 16456 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:14.433861 12631 raft_consensus.cc:2393] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 9: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:14.434278 16456 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:14.422559 16508 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:14.429446 16354 raft_consensus.cc:697] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:14.408092 15952 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:14.442481 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 88969.6us
I20260709 13:19:14.444058 16107 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:14.447672 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:14.445518 16379 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:14.448503 16430 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.448714 16479 raft_consensus.cc:3055] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:14.449188 16479 raft_consensus.cc:740] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:14.449746 16354 consensus_queue.cc:237] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:14.450260 16479 consensus_queue.cc:260] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:14.451651 16479 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:14.449071 16430 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:14.432901 16458 consensus_queue.cc:237] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:14.428056 16432 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:14.439091 12532 catalog_manager.cc:5697] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:14.463768 16321 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.464586 16321 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:14.442051 16537 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.446141 16508 raft_consensus.cc:697] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:19:14.317380 12529 rpcz_store.cc:269] 0709 13:18:44.486103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:44.494370 (+  8267us) service_pool.cc:224] Handling call
0709 13:18:45.006188 (+511818us) spinlock_profiling.cc:243] Waited 64.4 ms on lock 0x7b540011eed4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31c090 00007f4c6d3baf3a 00007f4c731711cc 00007f4c7326f37e 00007f4c717302a4 00007f4c71730232 00007f4c71730161 00007f4c717300dc 00007f4c7172f392 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:14.316665 (+29310477us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:06.472959 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:07.155645 (+682686us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:07.197661 (+ 42016us) write_op.cc:435] Acquired schema lock
0709 13:19:07.200623 (+  2962us) tablet.cc:662] Decoding operations
0709 13:19:07.550554 (+349931us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:07.558813 (+  8259us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:07.560133 (+  1320us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:19:07.634329 (+ 74196us) tablet.cc:701] Row locks acquired
0709 13:19:07.634377 (+    48us) write_op.cc:260] PREPARE: finished
0709 13:19:07.634577 (+   200us) write_op.cc:270] Start()
0709 13:19:07.634731 (+   154us) write_op.cc:276] Timestamp: P: 1783603147634506 usec, L: 0
0709 13:19:07.634755 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:19:07.635440 (+   685us) log.cc:844] Serialized 6362 byte log entry
0709 13:19:08.577333 (+941893us) op_driver.cc:464] REPLICATION: finished
0709 13:19:08.857653 (+280320us) write_op.cc:301] APPLY: starting
0709 13:19:08.857774 (+   121us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:08.858086 (+   312us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:08.858246 (+   160us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:08.865157 (+  6911us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:08.865530 (+   373us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 13:19:08.865601 (+    71us) write_op.cc:312] APPLY: finished
0709 13:19:08.866161 (+   560us) log.cc:844] Serialized 113 byte log entry
0709 13:19:08.866360 (+   199us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:08.866521 (+   161us) write_op.cc:454] Released schema lock
0709 13:19:08.866600 (+    79us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:08.867008 (+   408us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:08.867036 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":8301,"reported_tablets":51,"spinlock_wait_cycles":148019072,"child_traces":[["op",{"apply.queue_time_us":256986,"apply.run_cpu_time_us":4727,"apply.run_wall_time_us":17870,"num_ops":16,"prepare.queue_time_us":441744,"prepare.run_cpu_time_us":66131,"prepare.run_wall_time_us":1303279,"raft.queue_time_us":740182,"raft.run_cpu_time_us":18500,"raft.run_wall_time_us":199288,"replication_time_us":924423,"thread_start_us":1053700,"threads_started":4,"wal-append.queue_time_us":103203,"wal-append.run_cpu_time_us":43272,"wal-append.run_wall_time_us":2128209}]]}
I20260709 13:19:14.478075 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:14.478229 12626 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
W20260709 13:19:14.479890 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 567432us
I20260709 13:19:14.485778 12532 catalog_manager.cc:5697] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 4 to 6, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 6 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:14.489986 16234 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:14.491262 16456 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:14.492501 16432 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:14.492733 16379 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:14.495117 12673 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:14.496189 16355 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:14.496662 16355 raft_consensus.cc:697] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:14.498414 16355 consensus_queue.cc:237] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:14.504808 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.505920 16484 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:14.511519 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:14.515820 15952 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:14.518133 16321 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:14.518860 16430 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:14.477440 16357 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:14.520949 16484 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:14.522692 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:14.529424 16508 consensus_queue.cc:237] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:14.550278 12774 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 2063) took 1758 ms. Trace:
I20260709 13:19:14.530823 16282 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:14.535076 12532 catalog_manager.cc:5697] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:14.537889 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:14.538113 12598 leader_election.cc:336] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 2.142s (SENT)
W20260709 13:19:14.538602 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 253043us
W20260709 13:19:14.541088 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 66335.2us
I20260709 13:19:14.475772 16537 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:14.542358 16521 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:14.508142 16565 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:14.543356 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:14.563355 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:14.551555 12774 rpcz_store.cc:276] 0709 13:19:12.791465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.795691 (+  4226us) service_pool.cc:224] Handling call
0709 13:19:12.796093 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:12.796537 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:19:13.286449 (+489912us) log.cc:844] Serialized 14 byte log entry
0709 13:19:13.724619 (+438170us) log.cc:844] Serialized 14 byte log entry
0709 13:19:13.963151 (+238532us) spinlock_profiling.cc:243] Waited 4.53 ms on lock 0x7b180028ee80. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c53f3 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:19:14.461303 (+498152us) log.cc:844] Serialized 14 byte log entry
0709 13:19:14.502611 (+ 41308us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:14.503316 (+   705us) log.cc:844] Serialized 26 byte log entry
0709 13:19:14.505453 (+  2137us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:14.505494 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.510422 (+  4928us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:14.510464 (+    42us) raft_consensus.cc:1685] finished
0709 13:19:14.510475 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.550222 (+ 39747us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13561,"prepare.queue_time_us":747822,"prepare.run_cpu_time_us":166,"prepare.run_wall_time_us":17640,"raft.queue_time_us":122025,"raft.run_cpu_time_us":494,"raft.run_wall_time_us":38850,"spinlock_wait_cycles":10411392,"thread_start_us":1559144,"threads_started":3,"wal-append.queue_time_us":530205}
I20260709 13:19:14.542310 15952 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:14.566501 16357 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.571403 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:14.572476 16537 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:14.577817 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:14.596976 12629 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:14.578585 16282 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:14.549937 16561 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:14.549783 16484 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:14.593979 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:14.593988 12597 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:14.625702 16219 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11.
W20260709 13:19:14.646101 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:14.647408 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 45984.1us
I20260709 13:19:14.648855 12672 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:14.650151 16479 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:14.650698 16479 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:14.651113 16479 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:14.653756 16219 consensus_queue.cc:1059] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:14.654836 16219 raft_consensus.cc:3055] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:14.655215 16219 raft_consensus.cc:740] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:14.656632 16219 consensus_queue.cc:260] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:14.658201 16458 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:14.625741 16357 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:14.661916 16219 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:14.662418 16357 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:14.663075 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:14.666807 12630 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:14.672174 12673 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:14.673054 16573 consensus_queue.cc:1059] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:14.673499 12597 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:14.674847 16573 raft_consensus.cc:3055] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:14.675241 16573 raft_consensus.cc:740] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:14.675813 16542 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:14.675932 16453 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:14.676321 16573 consensus_queue.cc:260] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:14.672238 12707 raft_consensus.cc:1275] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:19:14.679261 16573 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:14.683822 16537 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:14.686323 16484 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:14.687582 16565 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:14.694060 16547 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:14.694702 16547 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:14.695111 16547 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:14.709446 12774 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:14.711064 12707 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:14.713979 13380 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:14.709527 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:14.715381 13382 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:14.682574 12672 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:14.718243 12629 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:19:14.723215 12781 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:14.723107 13349 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:19:14.723852 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.724416 16458 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:14.724999 16458 raft_consensus.cc:697] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:14.726102 16458 consensus_queue.cc:237] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:14.723491 16270 raft_consensus.cc:2764] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:19:14.731559 13382 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:14.718649 16484 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:14.765604 16069 consensus_queue.cc:1059] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:19:14.766183 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 83709.5us
I20260709 13:19:14.770696 16480 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:19:14.769766 16423 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:14.774714 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.787846 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:14.793047 12597 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:14.793926 12703 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:14.806177 16115 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:14.783456 15998 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:19:14.810489 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:14.812083 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:14.812963 12704 raft_consensus.cc:2410] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:19:14.814811 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:14.815003 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:14.815789 12700 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:14.843770 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:14.844897 12709 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:19:14.846189 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:14.846902 12704 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:14.847368 16321 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:14.848559 16321 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:14.851039 16479 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:14.853152 12704 raft_consensus.cc:1275] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:14.854905 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:14.857527 16379 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:19:14.861397 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:14.863360 16576 log.cc:927] Time spent T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:19:14.864813 16321 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:14.866321 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:14.867538 12701 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:14.876756 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.879074 12634 raft_consensus.cc:1275] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:19:14.879280 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 293211us
I20260709 13:19:14.860245 12705 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:14.880565 12706 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
W20260709 13:19:14.881490 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.888990 12700 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:14.898411 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:14.920037 12775 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12. Ops: []
W20260709 13:19:14.918855 12598 leader_election.cc:336] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.816s (SENT)
I20260709 13:19:14.921617 12781 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:14.921142 12707 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:14.927544 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:14.943257 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:14.954931 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:14.958863 12706 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:19:14.968191 12708 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:19:14.971408 16479 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:14.972723 16565 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.973413 16565 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:14.976344 16565 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:14.912343 16508 consensus_queue.cc:1048] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:19:14.981719 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:14.981962 16570 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:14.982391 12775 raft_consensus.cc:1240] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:14.982726 12780 raft_consensus.cc:1240] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:14.981724 16561 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.998183 12709 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:15.002305 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.011333 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:15.011983 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:15.012423 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:15.012923 12777 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
I20260709 13:19:15.012444 12776 raft_consensus.cc:2410] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:19:15.013767 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:15.013912 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:15.014917 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:15.017263 12703 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:15.014734 12777 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
I20260709 13:19:14.984722 16561 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:15.019337 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 100564us
I20260709 13:19:14.929764 16219 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:19:15.027437 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.022280 12779 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:19:15.022652 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:15.030956 12780 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:19:15.033370 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:15.035218 12701 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:19:15.056777 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:15.095717 12778 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:15.025763 16540 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.014613 16479 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:19:14.982604 16570 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:15.099859 12709 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:19:15.099267 12705 raft_consensus.cc:1275] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:19:15.023419 16270 consensus_queue.cc:1059] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:19:15.105262 12707 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:15.094938 12777 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:19:15.110441 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:15.112828 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:15.116597 12776 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:15.030501 16219 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:15.026178 16547 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:15.112358 16540 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:15.148274 12704 raft_consensus.cc:2368] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11.
I20260709 13:19:15.131378 16115 raft_consensus.cc:697] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:15.229941 16521 consensus_queue.cc:1059] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:15.232795 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:15.221632 16069 raft_consensus.cc:3055] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:15.239887 16069 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:15.241001 16069 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:15.242439 16069 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:15.251786 16219 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:15.262477 12776 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:19:15.263849 12780 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:15.264154 16355 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:15.372282 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.269440 16540 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:15.271523 16547 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:15.405640 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.400103 16561 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:15.409029 16570 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:15.397575 16355 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:15.438723 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.436709 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:15.439482 12630 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:19:15.439888 12776 raft_consensus.cc:2410] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:19:15.278757 16115 consensus_queue.cc:237] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:15.442251 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:15.443576 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:15.446749 12705 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:15.435313 16321 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:15.449992 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:15.450882 12709 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:15.460129 12707 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:15.444177 16540 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.464964 12703 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:19:15.466951 12780 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12. Ops: []
W20260709 13:19:15.473263 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.466455 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:15.482777 12705 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:15.466377 16589 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.491057 16107 raft_consensus.cc:3055] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:19:15.497112 16582 log.cc:927] Time spent T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.271s	user 0.000s	sys 0.002s
I20260709 13:19:15.470041 16423 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.505748 16598 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.506386 16598 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:15.509038 16598 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:15.517324 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 296567us
I20260709 13:19:15.467324 16540 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:15.519053 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:15.457363 16321 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:15.524299 12597 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:15.528174 16599 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:15.529036 16599 raft_consensus.cc:697] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:15.529827 16479 consensus_queue.cc:1059] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:15.530184 16599 consensus_queue.cc:237] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:15.525096 12598 leader_election.cc:336] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.931s (SENT)
I20260709 13:19:15.536935 12707 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:15.503796 16107 raft_consensus.cc:740] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:15.527628 13348 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:15.554318 16355 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:15.554253 16107 consensus_queue.cc:260] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:15.555524 16601 raft_consensus.cc:3055] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:15.556025 16601 raft_consensus.cc:740] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:15.519151 16423 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:15.556959 16601 consensus_queue.cc:260] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:15.558157 16423 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:15.559492 12598 leader_election.cc:336] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.973s (SENT)
W20260709 13:19:15.559916 16584 log.cc:927] Time spent T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.342s	user 0.000s	sys 0.002s
I20260709 13:19:15.555639 16107 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:15.561075 12598 leader_election.cc:336] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 2.029s (SENT)
I20260709 13:19:15.526134 16589 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:15.563990 16589 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:15.561502 12598 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:15.558167 16601 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:15.578310 12598 leader_election.cc:336] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 2.046s (SENT)
I20260709 13:19:15.578846 12598 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:15.580168 12598 leader_election.cc:336] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 2.088s (SENT)
I20260709 13:19:15.580583 12598 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:15.581840 16540 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:15.573706 16589 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:15.575506 16423 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.587437 16573 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:15.587224 16594 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:19:15.604923 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.611837 12777 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:15.630558 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:15.602846 16423 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:15.632643 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 114480us
W20260709 13:19:15.643589 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 63441.7us
I20260709 13:19:15.652762 13348 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:15.657642 16604 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:19:15.659507 16603 log.cc:927] Time spent T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb: Append to log took a long time: real 0.061s	user 0.001s	sys 0.001s
W20260709 13:19:15.661968 12598 leader_election.cc:336] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 2.136s (SENT)
I20260709 13:19:15.670526 16423 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:15.671188 16607 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:15.688843 16321 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:19:15.694473 15998 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:15.690282 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:15.696146 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.691936 16270 raft_consensus.cc:3055] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:15.723093 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:15.724280 12634 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:15.724783 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:15.729741 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 88878.1us
I20260709 13:19:15.731091 13348 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:15.732246 13348 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:15.705421 16270 raft_consensus.cc:740] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:15.749172 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:15.750061 12775 raft_consensus.cc:2410] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:19:15.753086 12778 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:15.744222 16379 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:15.761818 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 112489us
I20260709 13:19:15.737978 16321 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:15.762072 12634 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
W20260709 13:19:15.756690 16542 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.053s	sys 0.075s
I20260709 13:19:15.773751 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:15.728703 15998 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:15.769929 16542 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.053s	sys 0.075s
I20260709 13:19:15.763293 16458 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:15.771963 16321 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:15.780133 16609 log.cc:927] Time spent T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
W20260709 13:19:15.794006 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 61837.4us
I20260709 13:19:15.795001 13348 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:15.799176 15998 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:15.800786 16561 consensus_queue.cc:1048] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:15.801193 12747 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:15.786865 16458 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:15.801965 16458 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:15.802599 16561 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:15.802958 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:15.803925 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:15.804771 16561 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:15.805606 16570 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:19:15.810317 16590 log.cc:927] Time spent T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.397s	user 0.000s	sys 0.003s
W20260709 13:19:15.810573 16591 log.cc:927] Time spent T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.400s	user 0.003s	sys 0.000s
I20260709 13:19:15.810586 16570 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:15.811522 16570 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:15.807147 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:15.805873 16598 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:15.815770 16598 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:15.816174 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.816190 16598 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:15.819599 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:15.820487 12626 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:19:15.820597 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:15.819866 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:15.805843 16561 consensus_queue.cc:1048] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:19:15.789919 16321 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:15.823488 12517 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6609d5f93ab048558ea46e511ef3b2ec with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 19cd864c053744bebd96093a7982537e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:15.826236 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:15.827132 12629 raft_consensus.cc:2393] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 11: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:15.827679 16379 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:15.828584 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:15.829213 12781 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:19:15.830111 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:15.832831 12673 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:15.833364 16598 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:15.834614 16540 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:15.834684 12673 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:15.835261 16540 raft_consensus.cc:697] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:15.835808 16598 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:15.805529 12747 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:15.819615 12774 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:15.837527 13348 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:15.836230 16540 consensus_queue.cc:237] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:15.846100 16558 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:15.846530 12628 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:19:15.846946 16558 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:15.847469 16558 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:15.853708 16606 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:19:15.868673 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 63347us
I20260709 13:19:15.857435 16115 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.869943 16573 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.871330 16115 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:15.873878 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:15.874539 12779 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:15.878898 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:15.879769 13380 raft_consensus.cc:2393] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 12: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:15.880738 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:15.881958 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:15.871443 16573 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:15.887054 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 49629us
I20260709 13:19:15.889426 12782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 2100) took 1170 ms. Trace:
I20260709 13:19:15.894557 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:15.895746 16601 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:15.883265 16379 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:19:15.890913 16598 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:15.902915 16598 raft_consensus.cc:697] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:15.904301 16598 consensus_queue.cc:237] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:15.891489 12782 rpcz_store.cc:276] 0709 13:19:14.719161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.731419 (+ 12258us) service_pool.cc:224] Handling call
0709 13:19:14.773538 (+ 42119us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:14.790913 (+ 17375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:14.790944 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:14.838122 (+ 47178us) log.cc:844] Serialized 26 byte log entry
0709 13:19:15.131728 (+293606us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:15.131779 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:15.131884 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:15.809132 (+677248us) spinlock_profiling.cc:243] Waited 2.93 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:15.883148 (+ 74016us) raft_consensus.cc:1685] finished
0709 13:19:15.883173 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:15.889361 (+  6188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6978,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":3981,"spinlock_wait_cycles":6747520,"thread_start_us":292968,"threads_started":1,"wal-append.queue_time_us":459154}
I20260709 13:19:15.890396 16484 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:15.924784 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:15.925473 12774 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:15.916157 16561 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:15.928911 12597 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:15.913811 16521 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:15.793419 16270 consensus_queue.cc:260] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:15.930545 12779 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:15.890555 16321 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:15.889201 16595 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:19:15.936647 12778 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:19:15.937000 12778 pending_rounds.cc:85] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 4
I20260709 13:19:15.926496 16573 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:15.900344 16570 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:15.938796 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:15.923311 16484 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:15.949028 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.937191 12778 pending_rounds.cc:107] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:19:15.953810 12778 raft_consensus.cc:2889] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:15.922802 16379 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:19:15.935940 16594 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:15.946188 16484 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:15.969687 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 87783.6us
I20260709 13:19:15.973902 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:15.936146 16542 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:19:15.954270 16430 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.971218 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:15.936529 16521 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:15.965159 16594 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:16.006547 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.007495 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:16.013974 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:16.014808 12778 raft_consensus.cc:2410] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:19:16.013598 12779 raft_consensus.cc:1240] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:16.012432 16561 raft_consensus.cc:697] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:16.026350 12747 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:16.004658 16115 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:15.964768 16589 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:19:16.010432 16594 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:16.032150 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:16.042690 12779 raft_consensus.cc:2393] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 13: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:16.044441 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:16.005283 16430 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:16.048979 16430 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:16.051296 16621 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:16.052248 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.042158 16453 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:19:16.053022 12700 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
I20260709 13:19:16.054328 16595 consensus_queue.cc:1048] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:16.055171 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:16.052275 16521 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:16.056032 13384 raft_consensus.cc:2393] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 9: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:16.056250 16621 raft_consensus.cc:697] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:16.054481 12747 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:16.010145 16599 consensus_queue.cc:1048] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:19:16.068943 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.063237 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:16.070537 12778 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:19:16.073689 12782 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:16.074844 16355 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:16.075452 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:16.075860 16458 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:16.077091 13380 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:16.057307 16621 consensus_queue.cc:237] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:16.077950 16595 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:16.078632 16595 raft_consensus.cc:697] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:16.079568 16595 consensus_queue.cc:237] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:16.075344 16355 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:16.072283 12704 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
I20260709 13:19:16.082594 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:16.073140 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:16.028710 16625 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:16.087779 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:16.088721 12779 raft_consensus.cc:2393] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 10: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:16.085147 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:16.074132 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 108216us
I20260709 13:19:16.090368 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.089993 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:16.092504 13384 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:19:16.098650 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:16.100119 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:16.100203 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:16.103953 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.112571 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:16.113346 13382 raft_consensus.cc:2393] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 12: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:16.114666 12778 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:19:16.115784 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:16.118597 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:16.119352 12700 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:16.071314 16598 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:19:16.121318 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:16.102895 16469 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:16.148185 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:16.150187 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:16.151365 13377 raft_consensus.cc:2368] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11.
I20260709 13:19:16.151839 16626 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:16.152307 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:16.149305 13384 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:16.152292 16626 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:16.163849 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.167804 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.169735 12707 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:16.172977 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:16.173796 12782 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:16.176738 12704 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 11: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:16.178278 16270 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:16.153555 16570 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:16.189607 13383 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:16.190271 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:16.119366 16484 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:16.166258 16626 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:16.200371 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 127301us
I20260709 13:19:16.206564 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.203795 12700 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
W20260709 13:19:16.211759 13349 leader_election.cc:336] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.522s (SENT)
I20260709 13:19:16.172595 15998 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.210991 12704 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:19:16.214257 12747 leader_election.cc:400] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11.
I20260709 13:19:16.176790 16355 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:16.220700 12747 leader_election.cc:403] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:19:16.222872 16633 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.223454 16633 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:16.223717 16627 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:16.224486 16627 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:16.227084 16633 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:16.228263 16627 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:16.218111 12630 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:19:16.231114 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:16.194288 16321 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:16.206395 16453 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:19:16.236261 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.237289 12704 raft_consensus.cc:2393] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 7: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:16.199697 16558 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:16.239528 16558 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:16.236255 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:16.243007 12779 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 8: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:16.163997 16561 consensus_queue.cc:237] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:16.176803 16594 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:16.200002 16625 raft_consensus.cc:697] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 11 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:16.250015 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.258074 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:16.258674 12701 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:16.259120 12774 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:16.259748 12705 raft_consensus.cc:1240] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:19:16.260766 12774 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:19:16.260779 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.262099 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:16.263195 12634 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:16.263621 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.273310 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:16.274541 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:16.254215 16321 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:16.276036 16598 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.275188 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:16.266665 16458 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:16.287039 16594 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:16.229449 15998 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:16.289932 15998 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:16.290710 16321 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:16.267741 16621 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.306396 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:16.277904 13349 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:16.299003 16540 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:16.308856 16621 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:16.312393 16631 log.cc:927] Time spent T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 13:19:16.313990 16621 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:16.267674 16636 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:16.314705 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:16.322583 12782 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:19:16.315804 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:16.324020 12634 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:16.277065 16602 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:16.321437 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:16.315656 13349 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.723s (SENT) [suppressed 43 similar messages]
I20260709 13:19:16.334878 12701 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:19:16.320947 16607 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:16.312427 16540 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:16.340648 16607 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:16.341066 16607 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:16.340157 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.339941 12597 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:16.343073 12701 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:19:16.356210 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:16.358559 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:16.359424 12701 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:16.360208 12634 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:19:16.360533 12630 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:16.362149 16484 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:16.364526 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.287042 16598 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:16.365170 12706 raft_consensus.cc:2376] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11. Re-sending same reply.
I20260709 13:19:16.366644 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:16.342296 16540 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:16.375154 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:16.376708 16634 consensus_queue.cc:1059] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:19:16.377694 16634 raft_consensus.cc:3055] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:16.378073 16634 raft_consensus.cc:740] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:16.378981 16634 consensus_queue.cc:260] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:16.380244 16634 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:16.383746 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.371747 16630 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:16.387162 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:16.390194 12706 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:16.387257 12705 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:16.381409 16636 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11.
I20260709 13:19:16.394088 12630 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
W20260709 13:19:16.387935 16640 log.cc:927] Time spent T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:19:16.399246 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:16.405344 12630 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 7: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:16.404052 12779 raft_consensus.cc:1275] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:19:16.398082 12701 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:16.385921 16642 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:16.410036 16642 raft_consensus.cc:697] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:16.411341 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:16.390033 16630 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:16.412146 16630 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:16.406252 12779 pending_rounds.cc:85] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 6
I20260709 13:19:16.413024 12779 pending_rounds.cc:107] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:19:16.413265 12779 raft_consensus.cc:2889] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:16.416837 16601 consensus_meta.cc:287] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.521s	user 0.005s	sys 0.004s
I20260709 13:19:16.342588 16602 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:16.417866 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:16.411123 16642 consensus_queue.cc:237] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:16.422940 12774 raft_consensus.cc:1275] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:16.424139 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:16.417320 16601 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:16.431704 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:16.432418 12782 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 13: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:16.425251 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:16.426132 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:16.420238 13349 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:16.427074 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:16.427457 12706 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:19:16.428083 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.428715 13383 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:16.434442 12705 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 11: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:16.428906 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:16.437062 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.438005 12706 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:16.438656 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.440965 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:16.368417 16625 consensus_queue.cc:237] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:16.441423 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:16.440582 13379 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:19:16.443243 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:16.443580 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:16.446192 12706 raft_consensus.cc:2410] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:19:16.439594 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:16.448629 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:16.450858 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.451418 12705 raft_consensus.cc:2376] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11. Re-sending same reply.
I20260709 13:19:16.453832 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.454732 12706 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:16.454942 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:16.455974 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.457980 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.459291 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.459847 12702 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:16.460667 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.461323 12702 raft_consensus.cc:2393] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
W20260709 13:19:16.466670 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.467180 16602 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:16.468560 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:16.443994 16458 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:16.469447 12702 raft_consensus.cc:2368] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8.
W20260709 13:19:16.470152 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.464337 16601 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:16.515443 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:16.516193 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 74813.4us
I20260709 13:19:16.517077 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:16.517226 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:16.517836 12630 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:19:16.518326 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:16.519399 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.520054 12703 raft_consensus.cc:2393] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 7: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:16.520653 16606 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:16.514745 16633 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.522861 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:16.457237 16321 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:16.527411 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:16.529513 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:16.530107 12626 raft_consensus.cc:2368] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11.
I20260709 13:19:16.525306 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.531311 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:16.531456 12705 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:19:16.532240 12634 raft_consensus.cc:2368] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11.
I20260709 13:19:16.538519 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.509349 16458 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:16.575939 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:16.568805 12625 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:19:16.547338 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:16.580330 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:16.614298 16355 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:16.609536 12598 connection.cc:256] RPC call timeout handler was delayed by -1.1052802700000939s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18100000000000002s for a response
W20260709 13:19:16.630206 12598 connection.cc:256] RPC call timeout handler was delayed by -1.0495352279999679s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2118s for a response
I20260709 13:19:16.631309 12705 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:16.621897 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:16.632552 12701 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:19:16.634104 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:16.634678 12709 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:16.635816 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.639433 12701 raft_consensus.cc:2393] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:16.600451 16508 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.530840 16633 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:16.648550 16626 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:16.605901 16628 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.652098 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:16.674721 12706 raft_consensus.cc:2368] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13.
I20260709 13:19:16.527205 16607 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:16.657706 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:16.634384 16458 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:16.660775 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:16.660750 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 258802us
I20260709 13:19:16.643013 16630 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:16.665000 12598 leader_election.cc:336] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.650s (SENT)
I20260709 13:19:16.664918 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:16.683450 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 49338.3us
I20260709 13:19:16.669251 12626 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:16.684019 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.685902 12625 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:19:16.673463 12709 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:16.681352 12634 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:16.688757 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:16.689540 12629 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:19:16.650404 16355 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:16.663607 16508 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:16.691143 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.806360 12775 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:16.779677 12779 raft_consensus.cc:2410] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:19:16.808693 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:16.752902 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:16.809518 12701 raft_consensus.cc:2393] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:16.810516 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:16.808910 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 156442us
I20260709 13:19:16.747400 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:16.812902 12627 raft_consensus.cc:2393] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:16.720268 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:16.754953 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 66960.9us
I20260709 13:19:16.777987 12628 raft_consensus.cc:2368] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 8.
I20260709 13:19:16.708374 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.698748 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:16.761938 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:16.830313 12625 raft_consensus.cc:2393] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 11: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:16.831243 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:16.834177 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:16.835942 12629 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:16.836968 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:16.838238 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:16.838913 12705 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
I20260709 13:19:16.839792 12747 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:16.841336 12626 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:16.844147 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:16.812028 16626 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:16.664813 16651 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.748365 12672 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:16.765774 16656 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.852238 12705 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:19:16.801282 16633 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:16.856675 16634 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:16.859200 16601 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:16.866047 16607 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:16.845873 16561 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
W20260709 13:19:16.868364 16643 log.cc:927] Time spent T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.495s	user 0.003s	sys 0.001s
I20260709 13:19:16.691186 16628 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:16.884286 12625 raft_consensus.cc:1275] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:19:16.888310 12626 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:16.884871 12625 pending_rounds.cc:85] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 6
I20260709 13:19:16.895429 12625 pending_rounds.cc:107] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:19:16.895740 12625 raft_consensus.cc:2889] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:16.897235 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 57531.8us
I20260709 13:19:16.898205 12747 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:16.899092 16595 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:16.901758 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:16.902554 16595 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:16.903079 16595 raft_consensus.cc:697] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:16.903970 16595 consensus_queue.cc:237] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:16.853039 16656 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:16.923837 12634 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:19:16.925999 12630 raft_consensus.cc:1240] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:19:16.885727 16540 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:16.885864 16634 raft_consensus.cc:697] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:19:16.942626 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 194381us
I20260709 13:19:16.944146 12672 leader_election.cc:400] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11.
I20260709 13:19:16.944564 12672 leader_election.cc:403] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:19:16.947785 16656 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:16.954653 12517 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6609d5f93ab048558ea46e511ef3b2ec with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 19cd864c053744bebd96093a7982537e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:16.963817 12708 raft_consensus.cc:1275] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:16.854756 16651 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:16.971151 12672 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:16.973870 12708 raft_consensus.cc:1240] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:16.978972 12672 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:16.981122 12706 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:19:16.951695 16458 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:16.994958 12708 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:19:16.995440 12672 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:16.995999 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 158757us
I20260709 13:19:16.997674 12708 pending_rounds.cc:85] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 5
I20260709 13:19:16.997999 12708 pending_rounds.cc:107] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:19:16.998310 12708 raft_consensus.cc:2889] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:16.976748 16540 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11.
I20260709 13:19:16.999292 12632 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:19:16.977609 16630 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:17.028195 12701 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:17.032820 12672 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:17.021004 16619 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:17.033517 16508 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:17.073081 12598 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:17.036880 16540 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:17.105134 16669 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.021384 16634 consensus_queue.cc:237] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:17.093804 16657 log.cc:927] Time spent T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
W20260709 13:19:17.128633 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 55660.3us
I20260709 13:19:17.106137 16637 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.070569 16665 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:17.083367 16642 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.107633 16625 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:17.129009 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 96279.1us
I20260709 13:19:17.120630 16508 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:17.147927 16355 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:17.134712 12672 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:17.128018 16069 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:17.128412 16628 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:17.151818 16651 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:17.155695 13378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 636) took 1144 ms. Trace:
I20260709 13:19:17.156095 13378 rpcz_store.cc:276] 0709 13:19:16.011330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.047230 (+ 35900us) service_pool.cc:224] Handling call
0709 13:19:17.154781 (+1107551us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b5800095198. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c7238bfc0 00007f4c72376929 00007f4c723763d2 00007f4c6b8edbd4
0709 13:19:17.155073 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:17.155255 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:17.155282 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:17.155318 (+    36us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:17.155351 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:17.155398 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:17.155493 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48305920}
I20260709 13:19:17.112267 16656 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:19:17.132202 16669 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:17.154855 16458 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:17.152464 16642 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:17.167590 16508 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:17.168219 16642 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:17.160944 12747 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:17.161393 12598 leader_election.cc:336] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.797s (SENT)
I20260709 13:19:17.158241 16069 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:17.163149 13379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51754 (request call id 731) took 1231 ms. Trace:
I20260709 13:19:17.134909 16637 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:17.170058 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.170354 12598 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:17.171536 12672 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:17.130932 16540 raft_consensus.cc:697] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:17.172758 12747 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:17.172672 13379 rpcz_store.cc:276] 0709 13:19:15.927954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.928090 (+   136us) service_pool.cc:224] Handling call
0709 13:19:17.159562 (+1231472us) spinlock_profiling.cc:243] Waited 501 ms on lock 0x7b5400465354. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:17.159666 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1152463616}
I20260709 13:19:17.152294 16665 raft_consensus.cc:697] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:17.175148 16069 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:17.190788 16665 consensus_queue.cc:237] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:17.193218 16662 log.cc:927] Time spent T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:19:17.194182 16633 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:17.195768 16633 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:17.196141 16633 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.196502 16633 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:17.158033 16625 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:17.198750 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 151035us
I20260709 13:19:17.199472 12781 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:17.203507 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:17.204628 16648 consensus_queue.cc:1059] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:17.205765 16667 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:17.205936 16671 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:19:17.206987 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:17.212505 12782 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:17.214104 16637 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:17.216226 16625 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:17.199849 16453 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:17.207252 16669 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:17.228050 16540 consensus_queue.cc:237] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:17.232151 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:17.233287 12701 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:17.208577 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:17.235062 12778 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:17.239413 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:17.240200 13381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51762 (request call id 664) took 1213 ms. Trace:
I20260709 13:19:17.240478 13381 rpcz_store.cc:276] 0709 13:19:16.026897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.054240 (+ 27343us) service_pool.cc:224] Handling call
0709 13:19:17.239154 (+1184914us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b5800095198. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c723780a0 00007f4c72377984 00007f4c6b8f26d4 00007f4c6b8f2662
0709 13:19:17.240146 (+   992us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33067904}
I20260709 13:19:17.241678 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:17.208907 16673 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:19:17.248044 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.246958 12597 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:17.247285 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:17.250260 12627 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:17.249722 12703 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:19:17.255800 13348 connection.cc:256] RPC call timeout handler was delayed by -1.0597412559998247s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2082s for a response
W20260709 13:19:17.256328 13348 leader_election.cc:336] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 2.077s (SENT)
I20260709 13:19:17.257074 12597 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:17.253480 12746 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:17.261178 12746 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:17.263538 12782 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:19:17.205497 12672 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:19:17.268460 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 68972.5us
I20260709 13:19:17.205497 16648 consensus_queue.cc:1243] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Peer 3d539f8d95d343d8b1696803f18b30d2 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:19:17.292433 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 131072us
W20260709 13:19:17.293797 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.308208 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:17.313542 13348 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:17.307942 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:17.317866 12634 raft_consensus.cc:2410] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:19:17.320108 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:19:17.321817 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 149128us
I20260709 13:19:17.325186 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:17.265196 16453 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:17.267803 16637 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:17.302541 16669 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:19:17.338649 12598 leader_election.cc:336] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.810s (SENT)
I20260709 13:19:17.326072 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:17.340281 12598 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:17.302834 16630 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:17.315915 16670 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.346506 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:17.333019 16606 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:17.344249 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:17.332892 16453 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:17.352171 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 146735us
I20260709 13:19:17.353235 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:17.302806 16671 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
W20260709 13:19:17.356381 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 94177.3us
I20260709 13:19:17.357740 12597 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:17.316139 16558 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:19:17.337814 16637 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:17.344965 16630 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.362300 12630 raft_consensus.cc:1240] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:17.367918 13348 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:17.387096 16607 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:17.374248 16637 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:17.393131 16678 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:17.381950 16630 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:17.367888 16542 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:17.365721 16540 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:17.397709 16656 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:19:17.363595 16469 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.192s	user 0.008s	sys 0.006s
I20260709 13:19:17.389408 13378 raft_consensus.cc:1240] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:17.383810 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:17.403326 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 80818.2us
I20260709 13:19:17.356503 16453 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:17.407501 12627 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:19:17.409137 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:17.415686 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:17.410027 13381 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
W20260709 13:19:17.418525 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 60868.8us
I20260709 13:19:17.418882 13380 raft_consensus.cc:1275] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:19:17.408653 16469 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.236s	user 0.008s	sys 0.006s
I20260709 13:19:17.405112 16627 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:17.404510 16607 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:17.357304 16670 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:17.426343 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:17.363057 16458 consensus_queue.cc:1059] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:19:17.412846 16634 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:17.381740 16604 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:19:17.419994 16606 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.424278 16469 raft_consensus.cc:697] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:17.430722 16571 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:17.458564 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 76175.2us
I20260709 13:19:17.435153 16627 raft_consensus.cc:697] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:19:17.486176 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 169634us
I20260709 13:19:17.431461 16540 consensus_queue.cc:1059] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:19:17.494031 16601 raft_consensus.cc:3055] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:17.494192 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:17.494524 16601 raft_consensus.cc:740] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:17.494877 16604 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.486212 16678 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:19:17.447083 16607 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:17.495419 16604 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:17.495952 16627 consensus_queue.cc:237] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:17.497092 16607 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:17.455250 16606 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:17.450398 16685 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:17.492654 16670 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:17.495056 12628 raft_consensus.cc:2410] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 13 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 4.
I20260709 13:19:17.500645 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:17.491099 16675 log.cc:927] Time spent T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
W20260709 13:19:17.504720 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 55978.4us
I20260709 13:19:17.513209 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:17.452641 16634 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:17.516517 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:17.516309 13349 leader_election.cc:336] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.682s (SENT)
I20260709 13:19:17.482363 16680 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:17.536118 16469 consensus_queue.cc:237] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:17.537518 16636 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:17.539817 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:17.489943 16669 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:19:17.540140 12672 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:17.543447 12672 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:17.544363 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:17.544970 12631 raft_consensus.cc:2376] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10. Re-sending same reply.
I20260709 13:19:17.513264 16687 consensus_queue.cc:1059] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:19:17.549000 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:17.504947 16558 consensus_queue.cc:1059] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:19:17.550770 12631 raft_consensus.cc:2376] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12. Re-sending same reply.
W20260709 13:19:17.551240 13349 leader_election.cc:336] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.836s (SENT)
I20260709 13:19:17.551858 13349 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:17.554399 12598 leader_election.cc:336] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.880s (SENT)
I20260709 13:19:17.555128 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:17.556128 12598 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:17.504688 16479 consensus_queue.cc:1048] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:19:17.559190 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:17.559121 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:17.531183 16656 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:17.463892 16571 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:17.556485 12672 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:19:17.567723 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 101787us
I20260709 13:19:17.566704 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:17.566954 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:17.559947 12628 raft_consensus.cc:2376] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10. Re-sending same reply.
I20260709 13:19:17.551067 16423 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:17.572227 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:17.572815 12628 raft_consensus.cc:2376] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12. Re-sending same reply.
I20260709 13:19:17.630827 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:17.673406 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 86162us
I20260709 13:19:17.677547 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:17.674371 12672 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:17.552153 16604 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:17.539445 16069 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:17.680363 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:19:17.682163 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 131112us
I20260709 13:19:17.574155 16670 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:17.579015 16665 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:17.708271 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:17.711797 12629 raft_consensus.cc:2368] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11.
I20260709 13:19:17.701488 13349 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:17.712577 12672 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:17.550135 16636 consensus_queue.cc:1048] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.150s
I20260709 13:19:17.714012 12672 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:17.714668 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:17.720043 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:17.721100 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:17.723752 12774 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:19:17.725860 12774 raft_consensus.cc:1240] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:17.727316 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:17.721434 13349 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:17.726956 12774 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:17.733982 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:17.733578 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:17.734674 12629 raft_consensus.cc:2393] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:17.734723 12779 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:17.737533 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:17.738215 12626 raft_consensus.cc:2376] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11. Re-sending same reply.
I20260709 13:19:17.736537 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:17.741267 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:17.555845 16601 consensus_queue.cc:260] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:17.741618 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:17.742956 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:17.743952 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:17.745174 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:17.717535 16355 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:17.715447 16625 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:17.747140 16677 log.cc:927] Time spent T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.286s	user 0.001s	sys 0.000s
I20260709 13:19:17.706738 16691 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:17.682766 16673 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:19:17.759110 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:17.758442 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:17.787791 12703 raft_consensus.cc:2376] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8. Re-sending same reply.
I20260709 13:19:17.788156 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:17.788710 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:17.782100 12777 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:19:17.789889 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:17.757865 12629 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:17.789812 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:17.715780 16656 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:17.768724 13349 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:17.733626 16540 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:17.771766 12634 pending_rounds.cc:85] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 3
I20260709 13:19:17.796757 12634 pending_rounds.cc:107] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:19:17.797091 12634 raft_consensus.cc:2889] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:17.759168 16679 log.cc:927] Time spent T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 13:19:17.745976 16607 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:17.779327 16634 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:17.779740 16069 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:17.778327 16652 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.808351 16652 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:17.759282 16664 consensus_queue.cc:1048] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.123s
I20260709 13:19:17.809028 16598 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:17.810956 16652 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:17.758765 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:17.798003 16630 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:17.811870 12700 raft_consensus.cc:2368] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8.
I20260709 13:19:17.812839 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:17.813498 12700 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:17.780980 16625 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:17.814067 12633 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 8: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:17.815809 16625 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:17.802851 16604 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:17.816450 16115 raft_consensus.cc:3055] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:17.816845 16607 raft_consensus.cc:697] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:17.817065 16115 raft_consensus.cc:740] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:17.782510 12783 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:17.757215 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:17.816919 12778 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:17.757589 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:17.820827 12702 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:19:17.790639 12703 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
W20260709 13:19:17.760592 16690 log.cc:927] Time spent T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 13:19:17.798364 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:17.757234 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:17.798751 12779 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
W20260709 13:19:17.778312 16689 log.cc:927] Time spent T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
I20260709 13:19:17.735504 16628 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:17.802124 12706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 1976) took 1492 ms. Trace:
I20260709 13:19:17.741462 16595 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:17.797647 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:17.855795 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:17.855968 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:17.780597 16571 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:17.763513 16636 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:17.860901 16595 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:17.861318 16636 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:17.863408 16595 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:17.804445 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:17.864383 12704 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 8: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:17.865358 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:17.866050 12704 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:19:17.866206 16115 consensus_queue.cc:260] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:17.867002 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:17.867502 16115 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:17.868209 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:17.868988 16107 raft_consensus.cc:3055] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:17.805745 12629 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:19:17.869393 16107 raft_consensus.cc:740] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:17.870708 12629 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:17.870338 16107 consensus_queue.cc:260] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:17.756764 16692 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:17.818234 16607 consensus_queue.cc:237] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:17.782194 12628 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:17.878453 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:17.881083 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:17.828627 12700 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
W20260709 13:19:17.801515 16633 consensus_meta.cc:287] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.605s	user 0.012s	sys 0.000s
I20260709 13:19:17.831666 16630 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:17.835469 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:17.838551 12708 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:17.846181 12701 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:17.871696 16107 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:17.894824 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:17.854807 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 128656us
I20260709 13:19:17.853524 12706 rpcz_store.cc:276] 0709 13:19:16.309627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.683668 (+374041us) service_pool.cc:224] Handling call
0709 13:19:17.801950 (+1118282us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b54000e24d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:17.802067 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":678186880}
I20260709 13:19:17.873261 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:17.874890 12783 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:19:17.883322 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:17.879230 12702 raft_consensus.cc:2376] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13. Re-sending same reply.
I20260709 13:19:17.795962 16453 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:17.888471 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:17.900926 12704 raft_consensus.cc:2368] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8.
I20260709 13:19:17.901376 12701 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 7: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
W20260709 13:19:17.851493 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.893281 12778 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
I20260709 13:19:17.906046 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:17.906764 12706 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:19:17.898161 12700 raft_consensus.cc:2376] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11. Re-sending same reply.
I20260709 13:19:17.898257 12747 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:17.910380 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:17.912096 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:17.916100 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 345822us
I20260709 13:19:17.863425 16636 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:17.837085 16637 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:17.919561 12783 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:19:17.930610 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:17.931376 12702 raft_consensus.cc:2368] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10.
I20260709 13:19:17.921237 12703 raft_consensus.cc:1275] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:17.933008 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:17.919538 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:17.933992 12704 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
I20260709 13:19:17.934360 12701 raft_consensus.cc:2376] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11. Re-sending same reply.
I20260709 13:19:17.920049 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:17.937180 12704 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:19:17.938059 12631 raft_consensus.cc:2393] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:17.937206 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:17.939113 12707 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:19:17.921833 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:17.941766 12708 raft_consensus.cc:2393] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:17.941881 12775 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:17.901386 16458 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:17.946342 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:17.946472 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:17.947149 13381 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:17.945461 12703 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:19:17.947696 12672 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:17.947096 13380 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:17.949664 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:19:17.945076 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.951459 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:17.899698 16692 raft_consensus.cc:697] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:17.953495 12780 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:17.953773 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:17.954281 12707 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:17.946712 16458 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:17.962178 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 407773us
I20260709 13:19:17.980878 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:17.989419 12704 raft_consensus.cc:1240] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:17.990810 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:17.991654 12779 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:17.894057 16633 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:17.986094 16458 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:18.010700 12747 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:17.961674 16700 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:18.019284 12776 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:19:17.982158 16453 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:17.918886 16713 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.100s
I20260709 13:19:17.998014 16652 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:18.025890 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:18.034410 12779 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:19:18.037814 16696 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:18.037940 16607 consensus_queue.cc:1059] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:18.043465 12747 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:17.987144 16673 consensus_queue.cc:1059] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:19:17.982489 16699 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.111s
I20260709 13:19:18.050751 12780 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
W20260709 13:19:18.055927 12598 leader_election.cc:336] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.930s (SENT)
I20260709 13:19:18.024343 16634 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:18.065250 12783 raft_consensus.cc:1240] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7. Ops: []
W20260709 13:19:18.066181 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 268636us
W20260709 13:19:18.075428 12598 leader_election.cc:336] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.961s (SENT)
I20260709 13:19:18.044831 16571 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:18.091359 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 48254us
I20260709 13:19:18.092888 12747 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:18.096565 16571 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:18.102622 12747 leader_election.cc:400] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11.
I20260709 13:19:18.106169 16706 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.155s
I20260709 13:19:18.103164 16634 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:18.106964 16633 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:18.105840 16717 raft_consensus.cc:3055] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:18.107537 16717 raft_consensus.cc:740] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:18.082983 16692 consensus_queue.cc:237] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:18.108435 16717 consensus_queue.cc:260] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:18.105532 16710 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:19:18.111495 12747 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:18.111830 16571 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:18.069540 16598 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:19:18.135870 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 60470.6us
W20260709 13:19:18.144577 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 186523us
W20260709 13:19:18.147756 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 79192.7us
W20260709 13:19:18.149609 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.148517 13349 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:18.161837 13349 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:18.148175 16633 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:18.165289 13349 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:18.165499 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:18.170356 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:18.174638 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:18.146790 13348 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:18.181041 12626 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:18.160363 16606 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:18.199996 12598 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:18.202948 13349 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:18.204435 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:18.198366 16692 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:18.200670 16606 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:18.195560 16479 consensus_queue.cc:1048] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.015s
W20260709 13:19:18.213987 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 67305us
I20260709 13:19:18.214491 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:18.232225 13348 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:19:18.232585 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 121197us
I20260709 13:19:18.234460 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:18.241724 16727 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:18.211796 16687 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:18.202311 16571 consensus_queue.cc:1059] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:19:18.267242 16571 raft_consensus.cc:3055] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:18.267647 16571 raft_consensus.cc:740] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:18.268467 16571 consensus_queue.cc:260] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:18.269984 12779 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: []
W20260709 13:19:18.276023 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.248584 16721 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:19:18.253839 16725 log.cc:927] Time spent T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:19:18.253788 16732 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:19:18.253731 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.259045 12629 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:19:18.262671 16706 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:18.263262 16682 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:18.263789 16696 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:18.276274 13381 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
W20260709 13:19:18.275485 16723 log.cc:927] Time spent T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:19:18.232388 12626 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:19:18.202131 16453 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:19:18.215088 16692 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:19:18.293210 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 93279.6us
I20260709 13:19:18.305346 12598 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:18.298321 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 66200.5us
I20260709 13:19:18.281234 16721 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:18.297811 16687 raft_consensus.cc:697] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:18.299686 16571 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:18.305089 16712 log.cc:927] Time spent T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.386s	user 0.002s	sys 0.003s
I20260709 13:19:18.286315 16732 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:18.306746 16692 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:18.293926 16634 consensus_queue.cc:1059] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:19:18.357868 16634 consensus_queue.cc:1243] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:19:18.358444 16634 raft_consensus.cc:3055] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:18.358803 16634 raft_consensus.cc:740] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:18.359625 16634 consensus_queue.cc:260] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:18.360919 16634 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:18.327140 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:18.327332 13381 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:19:18.327575 16469 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.328625 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:18.363056 16469 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:18.363426 16604 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:18.332998 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:18.364797 13380 raft_consensus.cc:2393] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 10: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:18.322227 16736 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:18.352103 16732 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:18.322757 16107 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:19:18.325668 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.322606 16721 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:18.352952 16709 consensus_queue.cc:1059] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:19:18.353919 16458 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:18.354490 16676 raft_consensus.cc:3055] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:19:18.368440 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 63165.3us
I20260709 13:19:18.379226 12598 leader_election.cc:400] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8.
I20260709 13:19:18.380802 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:18.378631 16676 raft_consensus.cc:740] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:18.382730 16676 consensus_queue.cc:260] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:18.383824 16676 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:18.365370 16469 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:18.381430 13383 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:18.392424 12598 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:18.394176 12598 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:18.418967 13379 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:18.395733 12598 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:18.386086 16678 raft_consensus.cc:3055] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:18.427392 12598 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:18.419755 16107 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:18.429546 12597 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:18.421540 16453 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:18.432749 16069 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:18.439401 12597 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:18.427587 16627 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:18.427714 16678 raft_consensus.cc:740] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:18.422700 16573 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:19:18.434096 16736 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:18.433507 16606 raft_consensus.cc:697] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:18.446605 12598 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:18.439210 16711 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:18.440231 16453 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:18.454273 16711 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:18.462242 16458 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:18.462529 16711 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:18.435599 16687 consensus_queue.cc:237] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:18.462738 12632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 617) took 1095 ms. Trace:
I20260709 13:19:18.463183 12632 rpcz_store.cc:276] 0709 13:19:17.367448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.367616 (+   168us) service_pool.cc:224] Handling call
0709 13:19:17.367897 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:18.461797 (+1093900us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:18.461826 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:18.461866 (+    40us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:18.461895 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.462483 (+   588us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54002ebdd4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:18.462527 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.462638 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2515248000}
I20260709 13:19:18.465616 12705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1652) took 2174 ms. Trace:
I20260709 13:19:18.466145 16727 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:18.466840 16727 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:18.466025 12705 rpcz_store.cc:276] 0709 13:19:16.291263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.894607 (+603344us) service_pool.cc:224] Handling call
0709 13:19:17.802559 (+907952us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:17.802921 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:17.802957 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:17.803523 (+   566us) log.cc:844] Serialized 26 byte log entry
0709 13:19:18.002015 (+198492us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:18.002059 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.002120 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:18.167785 (+165665us) raft_consensus.cc:1685] finished
0709 13:19:18.167805 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.465332 (+297527us) spinlock_profiling.cc:243] Waited 908 ms on lock 0x7b54000e24d0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:18.465538 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":886,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":1406,"spinlock_wait_cycles":2087324544,"thread_start_us":146252,"threads_started":1,"wal-append.queue_time_us":282987}
I20260709 13:19:18.470229 16700 raft_consensus.cc:3055] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:18.470636 16700 raft_consensus.cc:740] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:18.471449 16700 consensus_queue.cc:260] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:18.472671 16700 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:18.447492 16115 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:18.474006 16115 raft_consensus.cc:697] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:18.474920 16115 consensus_queue.cc:237] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:18.479878 13380 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:18.480068 16627 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:18.497605 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:18.481624 16736 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.499363 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:18.500634 13383 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 8: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:18.481840 16453 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:18.537712 12672 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:19:18.538064 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 58786us
I20260709 13:19:18.538906 12672 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:18.539390 12598 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:18.554332 16659 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:18.555200 16619 consensus_queue.cc:1059] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:19:18.556382 16738 log.cc:927] Time spent T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:19:18.540736 16673 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.558410 16673 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:18.561218 16673 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:18.561618 16625 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:18.550498 16547 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:19:18.562197 16625 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:18.564260 16625 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:18.566454 16637 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:18.566380 16721 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:18.567685 13376 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:18.571228 16671 raft_consensus.cc:3055] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:18.571606 16671 raft_consensus.cc:740] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:18.577183 12631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56838 (request call id 1561) took 1033 ms. Trace:
I20260709 13:19:18.544697 16648 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.577610 12631 rpcz_store.cc:276] 0709 13:19:17.543568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.566473 (+ 22905us) service_pool.cc:224] Handling call
0709 13:19:18.576971 (+1010498us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b5400370854. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:18.577118 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":849359744}
I20260709 13:19:18.578150 16648 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:18.578863 12672 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:18.580134 16648 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:18.580331 12672 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:18.551673 16627 raft_consensus.cc:697] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:18.580672 12628 raft_consensus.cc:1240] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:18.581334 16619 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:18.581421 16627 consensus_queue.cc:237] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:18.582068 16619 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:18.582646 16619 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:18.493425 16678 consensus_queue.cc:260] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:18.584722 16678 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:18.586167 16601 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:18.587607 16453 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:18.599727 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.600548 12628 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:19:18.602238 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:18.602864 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.601689 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:18.540136 16736 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:18.605022 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 65701.8us
I20260709 13:19:18.616982 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:18.617724 12628 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:18.642699 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:18.643478 12781 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:18.642722 12673 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:18.650012 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:18.654392 12781 raft_consensus.cc:1275] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:19:18.603789 16626 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:18.612897 16453 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:18.647645 16672 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:18.698726 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 118221us
I20260709 13:19:18.593225 16692 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:18.687835 16659 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.637012 16606 consensus_queue.cc:237] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:18.697808 16626 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:18.719411 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 100853us
I20260709 13:19:18.696382 16696 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
W20260709 13:19:18.722892 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 107030us
I20260709 13:19:18.733987 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:18.734761 12706 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:19:18.711244 16702 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:18.763741 12705 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:19:18.766997 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:18.767625 12705 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:19:18.771055 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:18.772891 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:18.769372 16736 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:18.774770 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:18.778730 12704 raft_consensus.cc:1275] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:19:18.780740 12704 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:19:18.784981 12704 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11. Ops: [8.5-10.6]
I20260709 13:19:18.705729 16711 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:18.787501 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 59448.6us
I20260709 13:19:18.719679 16676 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.791990 12598 leader_election.cc:400] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8.
I20260709 13:19:18.795840 13385 raft_consensus.cc:1240] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:18.807161 16626 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:18.704020 16672 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:18.815506 16672 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:18.816810 16676 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:18.817492 12706 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:18.818465 12707 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:19:18.819173 16676 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:18.819558 16721 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:18.821462 12598 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:18.822960 12598 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:18.823621 16571 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:18.824503 13378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 742) took 1369 ms. Trace:
I20260709 13:19:18.823814 12780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54346 (request call id 512) took 1099 ms. Trace:
I20260709 13:19:18.824887 13378 rpcz_store.cc:276] 0709 13:19:17.455600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.461235 (+  5635us) service_pool.cc:224] Handling call
0709 13:19:17.461576 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:17.497810 (+ 36234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:17.497840 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:17.497879 (+    39us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:17.497909 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.824259 (+1326350us) spinlock_profiling.cc:243] Waited 36 ms on lock 0x7b5400498c54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:18.824304 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.824451 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":82865024}
I20260709 13:19:18.825310 12780 rpcz_store.cc:276] 0709 13:19:17.724132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.724230 (+    98us) service_pool.cc:224] Handling call
0709 13:19:18.823612 (+1099382us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b54002cc254. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:18.823728 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":429868416}
W20260709 13:19:18.830199 16746 log.cc:927] Time spent T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 13:19:18.832955 16692 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:18.717909 16659 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:18.836112 16659 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:18.842043 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.847657 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:18.849936 12598 leader_election.cc:400] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10.
I20260709 13:19:18.854310 12598 leader_election.cc:403] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:18.839272 16740 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.791468 16747 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:19:18.882936 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.851356 16571 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:19:18.936808 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 294140us
I20260709 13:19:18.926584 12709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 474) took 2645 ms. Trace:
I20260709 13:19:18.878942 16115 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10.
W20260709 13:19:18.929697 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.957392 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:18.960700 12704 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
W20260709 13:19:18.960592 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 110832us
W20260709 13:19:18.971518 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.839479 16732 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:18.923256 16740 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:18.926862 16673 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:18.998049 16673 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:19.002440 16711 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:19.004860 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:19.006950 16606 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:19.008069 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:19.008838 12701 raft_consensus.cc:2393] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:19.009083 13348 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:19.009984 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:19.010941 13378 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:19.012362 12747 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:19.012519 12780 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:19:19.013240 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:19.013329 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:19.018631 13381 raft_consensus.cc:1240] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:19.019562 16732 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:19.024551 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:19.027545 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:19.031283 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:19:19.035019 16744 log.cc:927] Time spent T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 13:19:19.035758 16713 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:19.036623 16713 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:19.037976 16743 log.cc:927] Time spent T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.299s	user 0.001s	sys 0.000s
I20260709 13:19:19.041038 12777 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:19:19.046787 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:19.057885 12597 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:19.028595 16642 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:19.061023 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:19.039645 16710 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:19.039518 16648 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:19.063419 16698 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:19.069231 16479 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:19.068434 16678 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:19.069780 16627 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:19.067891 16642 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:19.070268 16717 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:19.070739 16625 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:19:19.072826 16700 consensus_meta.cc:287] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.600s	user 0.008s	sys 0.005s
I20260709 13:19:19.072590 16696 consensus_queue.cc:1048] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:19.072966 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:18.985813 16571 consensus_queue.cc:1048] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 13:19:19.073179 16637 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:19:19.077472 16547 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:19.021552 16453 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:19:19.052872 16687 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.053146 16713 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:19.046123 16595 consensus_queue.cc:1059] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
W20260709 13:19:19.148079 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 75168.1us
I20260709 13:19:19.061400 16672 consensus_queue.cc:1059] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:19:19.166098 12705 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:19.167776 12629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1580) took 1615 ms. Trace:
I20260709 13:19:19.125608 16589 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:19:19.146518 16727 consensus_meta.cc:287] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.679s	user 0.009s	sys 0.003s
W20260709 13:19:19.183923 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 126319us
I20260709 13:19:18.956810 12709 rpcz_store.cc:276] 0709 13:19:16.280760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.833227 (+552467us) service_pool.cc:224] Handling call
0709 13:19:16.846787 (+ 13560us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:19:16.850020 (+  3233us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:16.855458 (+  5438us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.579285 (+1723827us) spinlock_profiling.cc:243] Waited 6.53 ms on lock 0x7b18003049c0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6cf5a4bc 00007f4c6a2a848b 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c2455 00007f4c6d3aa060 00007f4c6d3bc716 00007f4c6d3cd287
0709 13:19:18.579374 (+    89us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:19:18.580119 (+   745us) log.cc:844] Serialized 104 byte log entry
0709 13:19:18.679320 (+ 99201us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:18.880889 (+201569us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.912485 (+ 31596us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.920697 (+  8212us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.925801 (+  5104us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.926087 (+   286us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.926216 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:18.926290 (+    74us) raft_consensus.cc:1685] finished
0709 13:19:18.926310 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.926507 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41551,"prepare.queue_time_us":4966,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":115339,"raft.run_cpu_time_us":547,"raft.run_wall_time_us":96405,"spinlock_wait_cycles":26164864,"thread_start_us":529479,"threads_started":4,"wal-append.queue_time_us":316360,"wal-append.run_cpu_time_us":2341,"wal-append.run_wall_time_us":1116561}
I20260709 13:19:19.198010 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:19.199824 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:19.199824 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:19.211814 12631 raft_consensus.cc:1240] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13. Ops: []
W20260709 13:19:19.226817 12517 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 6609d5f93ab048558ea46e511ef3b2ec with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 19cd864c053744bebd96093a7982537e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:19.219422 16671 consensus_queue.cc:260] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:19.175247 12629 rpcz_store.cc:276] 0709 13:19:17.552762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.871483 (+318721us) service_pool.cc:224] Handling call
0709 13:19:17.871826 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:18.288226 (+416400us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.167465 (+879239us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b540002ce54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:19.167700 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":889693056}
I20260709 13:19:19.223724 12776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2418) took 1267 ms. Trace:
I20260709 13:19:19.092581 16698 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:19.126420 16710 raft_consensus.cc:697] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 10 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:19.168865 16626 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.141396 16625 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:19.228200 12776 rpcz_store.cc:276] 0709 13:19:17.956888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.958854 (+  1966us) service_pool.cc:224] Handling call
0709 13:19:17.998453 (+ 39599us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:18.019854 (+ 21401us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.223440 (+1203586us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b54001aefd4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:19.223660 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47251840}
I20260709 13:19:19.225625 12634 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:19.120450 16115 consensus_queue.cc:1059] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.110s
I20260709 13:19:19.174165 16634 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:19.173175 16652 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.144326 16648 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:19.224303 16589 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:19.125146 16479 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:19.237969 16698 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:19.173969 16687 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:19.149596 16627 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:19.268239 16625 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:19.270015 16671 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:19.273360 16634 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:19.276293 16589 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:19.235755 16727 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:19.237031 16766 consensus_queue.cc:1048] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:19:19.276687 16652 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:19.282342 16706 consensus_queue.cc:1059] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.093s
I20260709 13:19:19.271849 16626 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:19.359566 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 46734.9us
I20260709 13:19:19.390930 12632 raft_consensus.cc:2410] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:19:19.370198 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:19.395908 12629 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:19:19.296671 16634 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:19.366834 16700 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:19.373843 16745 log.cc:927] Time spent T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.330s	user 0.001s	sys 0.000s
W20260709 13:19:19.366896 16756 log.cc:927] Time spent T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.244s	user 0.000s	sys 0.000s
W20260709 13:19:19.379458 16754 log.cc:927] Time spent T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
W20260709 13:19:19.368891 16753 log.cc:927] Time spent T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
W20260709 13:19:19.407437 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 343582us
I20260709 13:19:19.407603 12634 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:19:19.366334 16759 log.cc:927] Time spent T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:19:19.408375 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:19:19.372888 16748 log.cc:927] Time spent T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.392s	user 0.001s	sys 0.001s
I20260709 13:19:19.411595 13377 raft_consensus.cc:1240] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:19.375164 16687 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:19.377391 16732 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:19.378803 16648 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:19.358029 16767 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.419190 12673 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:19.433219 16697 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.445960 16764 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:19.446522 16764 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:19.449271 16693 raft_consensus.cc:3055] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:19.449770 16693 raft_consensus.cc:740] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:19.450863 16727 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:19.450636 16693 consensus_queue.cc:260] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:19.451308 12628 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:19.452862 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:19.452955 16626 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:19.454762 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:19:19.457471 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:19.458688 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.416666 16676 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:19.462385 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:19.467293 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:19.442878 16619 consensus_queue.cc:1059] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:19:19.470815 12628 raft_consensus.cc:2393] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:19:19.470870 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 234683us
I20260709 13:19:19.471909 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:19.472843 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:19.472550 12628 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:19:19.416168 16700 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:19.475836 12634 raft_consensus.cc:2368] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 11. Current term is 12.
I20260709 13:19:19.476230 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:19.477272 12625 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 6.
W20260709 13:19:19.478997 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 314023us
I20260709 13:19:19.481482 12628 raft_consensus.cc:1275] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:19:19.367900 16710 consensus_queue.cc:237] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:19.463860 16761 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:19.490677 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:19.441358 16732 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:19.473349 16713 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:19.497457 12776 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:19:19.507905 16700 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:19.454759 16767 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:19.508685 16771 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:19.510382 12634 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:19:19.491530 16619 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.511204 16626 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:19.495342 16761 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:19.572077 16761 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:19.556965 12781 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:19:19.573554 12781 raft_consensus.cc:606] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:19:19.575206 16771 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:19.495102 16627 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:19.578871 16732 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:19:19.556150 16713 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:19.471312 16676 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:19.516239 16479 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:19.516023 16589 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:19.518769 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:19.522110 16678 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:19.523974 12747 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:19.530740 16764 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:19.490880 12672 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:19.508502 12627 raft_consensus.cc:1240] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:19:19.483814 16673 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:19.489091 16642 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:19.462833 16697 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:19.569972 16619 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:19.597313 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 186152us
I20260709 13:19:19.600463 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:19.605746 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:19.609651 12631 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:19:19.595654 16598 raft_consensus.cc:2764] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 13:19:19.594275 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 103465us
W20260709 13:19:19.592960 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 69083.7us
I20260709 13:19:19.517258 16698 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:19.595888 16547 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:19.609860 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:19.630172 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:19.632257 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:19.632912 12708 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:19:19.642872 12672 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:19.602209 16771 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:19.591370 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 318849us
I20260709 13:19:19.647363 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:19.653831 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:19.656993 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:19.665786 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:19.667554 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:19.672765 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:19.673439 12706 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:19.676738 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:19.678108 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:19.678716 12701 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:19.680569 12704 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:19:19.685163 12702 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:19.685190 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:19.686846 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:19.686609 16696 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:19.656419 16717 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:19.664127 16652 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:19.693361 16696 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:19.710825 16697 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:19.672696 16642 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:19.712383 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:19.713531 12704 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:19.649503 16732 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:19:19.721138 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:19.723785 12701 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:19:19.725059 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:19.730264 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:19.731053 12700 raft_consensus.cc:2376] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13. Re-sending same reply.
I20260709 13:19:19.742359 12781 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:19.711648 16589 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:19.692283 16717 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:19.747500 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:19.752823 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:19.755087 12706 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:19:19.756539 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:19.757366 12706 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:19:19.758297 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:19.759825 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:19.737278 16598 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:19.764294 16767 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:19.808843 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 123045us
W20260709 13:19:19.807842 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 129492us
I20260709 13:19:19.764721 16717 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:19.768748 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:19.768862 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:19.776450 12598 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:19.767590 16672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.013s	sys 0.022s
I20260709 13:19:19.785360 12776 raft_consensus.cc:1240] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:19:19.794315 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:19.794741 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:19.795408 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 151707us
I20260709 13:19:19.797072 12707 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:19:19.816515 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:19.811343 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:19.815189 12774 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:19.732455 16642 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:19.822602 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:19.823297 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 47075.3us
I20260709 13:19:19.817306 12707 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:19:19.783209 16698 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:19.834580 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:19.830936 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 196609us
W20260709 13:19:19.815297 16672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.014s	sys 0.022s
I20260709 13:19:19.847647 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:19.848308 12598 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:19.847904 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:19.767602 16795 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:19:19.849781 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:19.851461 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 166338us
I20260709 13:19:19.852632 13378 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:19:19.852716 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:19.809616 16696 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:19.855911 12598 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:19.857393 12598 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:19.860893 12782 raft_consensus.cc:2393] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 11: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:19.793980 16779 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 13:19:19.833684 16764 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:19.870255 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:19.873459 13379 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:19.865851 12747 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:19.840736 16676 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:19.870599 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:19.879210 12774 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:19.882802 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:19.864815 16673 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:19.854264 16619 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:19.838294 16115 consensus_queue.cc:1059] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:19:19.887534 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:19.888682 16115 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:19.888891 16573 consensus_queue.cc:1059] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:19.889148 16598 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:19.875257 16696 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:19.890218 16696 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:19.891351 12747 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:19.891613 16696 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:19.892066 16598 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:19.892634 12747 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:19.893129 16696 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:19.865060 16634 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:19.893886 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:19.888629 12709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:19:19.895048 12747 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:19.896421 12747 leader_election.cc:400] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 11. Current term is 12.
I20260709 13:19:19.898869 12747 leader_election.cc:403] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:19:19.871672 16766 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:19.864632 16765 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:19.878129 16781 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.874958 16687 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:19.911144 13379 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:19.902966 16637 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:19.893407 16598 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:19.918473 13378 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:19:19.928836 12704 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
W20260709 13:19:19.931641 16780 log.cc:927] Time spent T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:19:19.909941 16775 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 11. Current term is 12.
I20260709 13:19:19.902982 16698 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:19.938959 16634 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:19.922202 16797 consensus_queue.cc:1048] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:19:19.922006 16766 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.941191 16766 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:19.943295 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 59204.1us
I20260709 13:19:19.918638 15998 raft_consensus.cc:3055] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:19.895563 16648 consensus_queue.cc:1059] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:19:19.903554 16740 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:19.968036 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:19.919772 16637 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:19.966420 16625 consensus_meta.cc:287] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.654s	user 0.011s	sys 0.000s
I20260709 13:19:19.974043 16625 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:19.975332 16547 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:19.976027 12709 raft_consensus.cc:606] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:19:19.976140 16625 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:19.976406 12709 raft_consensus.cc:614] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:19.935976 16696 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:19:19.980077 16779 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:19.980641 16779 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:19.982107 16642 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:19.983673 16766 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:19.966578 15998 raft_consensus.cc:740] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:19.985026 16713 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:19.985143 12703 raft_consensus.cc:1275] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:19:19.985594 12703 pending_rounds.cc:85] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 4
I20260709 13:19:19.985894 12703 pending_rounds.cc:107] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:19:19.986138 12703 raft_consensus.cc:2889] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:19.985535 15998 consensus_queue.cc:260] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:19.986747 15998 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:19.990932 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:19.991786 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:19.991115 13349 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:19.993793 12701 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:19:19.994141 13349 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:19.994684 16802 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:19.995287 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:19.995445 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:19.996156 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:19.999958 12632 raft_consensus.cc:2410] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:19:20.000766 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:19.998569 13378 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:20.003388 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:20.005816 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:20.007534 13349 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:20.009289 13349 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:20.010413 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:20.012284 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:19.972381 16637 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:20.015641 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:20.016222 12625 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:19:20.018877 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:20.020669 12630 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:19:20.022209 12629 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:20.014420 16710 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:19:19.922915 16781 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:20.011132 16606 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:20.013638 16740 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:19:20.017146 16755 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.016s	sys 0.031s
I20260709 13:19:20.037446 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:20.038348 12700 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:20.017347 16789 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:20.041790 12625 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:19:20.044998 13379 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:20.047645 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:20.048343 12627 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
I20260709 13:19:20.050179 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:20.021236 16648 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:19:20.052551 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:20.035009 16755 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.017s	sys 0.031s
I20260709 13:19:20.054170 16789 raft_consensus.cc:697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:20.057367 16789 consensus_queue.cc:237] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:20.053401 13349 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:20.053921 16797 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.163523 16797 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:20.153101 16717 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:20.156960 16678 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.065019 16648 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:20.157953 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
W20260709 13:19:20.158228 16807 log.cc:927] Time spent T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:19:20.158676 16775 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.112316 16740 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:20.095752 16625 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.178550 16625 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:20.181190 16625 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:20.162130 16779 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:20.051366 16761 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:20.054280 16633 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:20.246145 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:20.175987 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:20.177774 16453 raft_consensus.cc:3055] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:20.177397 16740 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:20.184139 16673 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:20.189922 16774 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:20.191480 16696 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:20.266636 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:20.267124 12700 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:19:20.267257 13377 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:20.191929 16561 raft_consensus.cc:3055] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:20.192332 16775 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:20.203609 16648 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:20.209735 12672 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:20.211912 16698 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:20.212476 16785 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:20.171449 16678 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:20.160792 16781 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:20.213968 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 43583.5us
I20260709 13:19:20.253837 16453 raft_consensus.cc:740] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:19:20.214335 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.231413 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:20.229502 16771 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:20.288451 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.236661 16776 raft_consensus.cc:3055] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:20.237846 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:20.290647 12704 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:20.251256 12630 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:19:20.251837 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:20.252631 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:20.255869 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:20.265625 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 153465us
I20260709 13:19:20.294106 12701 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:20.240792 16619 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:19:20.167104 16798 log.cc:927] Time spent T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.170s	user 0.002s	sys 0.000s
I20260709 13:19:20.263639 16673 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:20.261267 16740 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:20.311401 13381 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:20.286962 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:20.287348 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 77716.4us
I20260709 13:19:20.266709 16696 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:20.315759 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:20.316392 12707 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:20.298676 12777 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 11: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:20.286585 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:20.300578 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:20.302796 12598 leader_election.cc:336] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 2.118s (SENT)
I20260709 13:19:20.318444 12598 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:20.260573 16633 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:20.323171 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:20.269810 16561 raft_consensus.cc:740] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:20.335359 12701 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:20.317174 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:20.341529 13380 raft_consensus.cc:1240] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:20.290352 16776 raft_consensus.cc:740] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:20.344826 13348 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:20.310449 16700 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:20.348129 16671 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.348553 16767 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:20.349937 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:20.292037 16706 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:19:20.350947 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:20.386844 16775 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:20.387810 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:20.388715 12783 raft_consensus.cc:2368] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8.
I20260709 13:19:20.339972 16713 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:20.346025 16766 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:20.344219 16659 raft_consensus.cc:3055] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:20.354000 12775 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:19:20.354243 15998 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:20.301959 16771 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:20.355680 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:20.347450 12782 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:19:20.360239 12781 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
W20260709 13:19:20.362073 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 59309.6us
I20260709 13:19:20.408867 12777 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:19:20.270071 16774 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:20.323282 16696 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:20.366977 12779 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:19:20.366605 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:20.350265 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.368419 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:20.301200 16785 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:20.381938 16790 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:20.440136 13381 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:20.390831 12706 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:19:20.391330 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:20.397894 16713 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:20.393855 12707 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:19:20.352741 12704 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:19:20.339615 16779 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:20.445210 16779 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:20.405929 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:20.344127 16606 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:20.300402 16772 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:20.411489 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:20.326323 16619 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:20.354687 16573 consensus_queue.cc:1059] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.019s
W20260709 13:19:20.417279 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 170773us
I20260709 13:19:20.418227 12705 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:19:20.438640 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:20.337733 16747 consensus_queue.cc:1059] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:19:20.361094 16598 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:20.441954 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 50708.8us
I20260709 13:19:20.492424 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:20.351336 16791 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:20.443677 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:20.498773 12779 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:20.406834 16659 raft_consensus.cc:740] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:20.498893 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:20.502156 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:20.503371 12634 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:20.413362 15998 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:19:20.458511 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 106568us
I20260709 13:19:20.458770 12708 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:20.463712 12706 raft_consensus.cc:1240] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 13. Current term is 14. Ops: [11.7-13.8]
W20260709 13:19:20.466197 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 118756us
I20260709 13:19:20.407862 16775 consensus_queue.cc:1059] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:19:20.403795 16767 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:20.398839 16671 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:20.361042 16453 consensus_queue.cc:260] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:20.449106 16713 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:20.450646 16595 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:20.509456 15998 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:20.474895 16606 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:20.515255 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:20.517604 12777 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:19:20.506064 16115 raft_consensus.cc:3055] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:20.421579 16561 consensus_queue.cc:260] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:20.482883 12702 raft_consensus.cc:1240] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:19:20.506956 16792 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:20.508972 16573 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:20.524174 16571 consensus_queue.cc:1059] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:19:20.531067 16767 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:20.531217 16673 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:20.512169 16676 raft_consensus.cc:3055] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:20.513135 16761 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:20.532157 16659 consensus_queue.cc:260] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:20.541296 12634 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:19:20.541836 12779 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:20.505693 16598 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:20.449528 16776 consensus_queue.cc:260] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:20.490346 16772 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:20.537217 16698 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:20.535930 16595 raft_consensus.cc:697] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:20.538487 16115 raft_consensus.cc:740] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:20.508671 16791 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:20.540722 16785 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:20.553293 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:20.559113 12777 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:19:20.548355 16809 log.cc:927] Time spent T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 13:19:20.552290 16633 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:20.561120 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:20.557771 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 58976.1us
I20260709 13:19:20.542630 16797 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:20.547279 16561 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:20.545843 16573 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:20.573298 16698 raft_consensus.cc:697] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:19:20.583019 16820 log.cc:927] Time spent T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:19:20.583353 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:20.583778 16115 consensus_queue.cc:260] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:20.584352 16619 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:20.553185 16822 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:19:20.584404 16659 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:20.547504 16676 raft_consensus.cc:740] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:20.587934 16732 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:19:20.560425 16811 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
W20260709 13:19:20.589032 16814 log.cc:927] Time spent T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:19:20.594237 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:20.598392 13377 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:20.544721 16747 consensus_queue.cc:1059] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:19:20.581750 16678 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:20.584347 16598 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:20.638273 12634 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:20.594190 16776 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:20.556640 16821 consensus_queue.cc:1059] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.099s
I20260709 13:19:20.649545 16774 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:20.632966 16606 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:20.654450 16115 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:20.680781 12628 raft_consensus.cc:2393] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 13: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:20.725401 12746 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:20.713083 16816 log.cc:927] Time spent T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
I20260709 13:19:20.682613 16732 raft_consensus.cc:697] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:20.683367 16771 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:20.728721 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:20.736392 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 144068us
I20260709 13:19:20.736593 16840 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:19:20.735723 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 191889us
I20260709 13:19:20.743240 12672 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:20.676142 16595 consensus_queue.cc:237] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:20.744956 12672 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:20.734076 16698 consensus_queue.cc:237] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:20.726900 16727 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:19:20.755230 16671 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:20.792797 15998 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:20.797925 16573 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:20.776129 16772 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:20.820461 16637 consensus_meta.cc:287] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.798s	user 0.012s	sys 0.001s
I20260709 13:19:20.821226 16637 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:20.822010 12631 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:19:20.727651 16676 consensus_queue.cc:260] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:20.821743 16791 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:20.824857 16791 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:20.802033 16732 consensus_queue.cc:237] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:20.835475 12703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 2029) took 1170 ms. Trace:
I20260709 13:19:20.836561 12703 rpcz_store.cc:276] 0709 13:19:19.665139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.988681 (+323542us) service_pool.cc:224] Handling call
0709 13:19:19.988997 (+   316us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:20.236637 (+247640us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:20.236671 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:20.237252 (+   581us) log.cc:844] Serialized 52 byte log entry
0709 13:19:20.287352 (+ 50100us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:20.307867 (+ 20515us) log.cc:844] Serialized 14 byte log entry
0709 13:19:20.309427 (+  1560us) log.cc:844] Serialized 14 byte log entry
0709 13:19:20.309642 (+   215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:20.309714 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:20.832789 (+523075us) raft_consensus.cc:1685] finished
0709 13:19:20.835233 (+  2444us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:20.835411 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":261840,"prepare.queue_time_us":19626,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":248628,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":14960,"spinlock_wait_cycles":80649344,"thread_start_us":48364,"threads_started":1,"wal-append.queue_time_us":127245}
I20260709 13:19:20.823535 16713 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:20.825344 16791 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:20.850975 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 106122us
I20260709 13:19:20.872026 16791 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:20.872822 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:20.847963 16828 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.873031 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:20.861975 16843 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:20.873910 16828 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:20.866379 16637 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:20.832582 16727 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:20.876142 16727 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:20.878049 16747 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:20.878791 16747 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:20.879320 16747 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:20.882349 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:20.885839 13377 raft_consensus.cc:2393] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 11: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:20.891024 13349 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:20.883104 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:20.900485 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.903470 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:20.903818 16747 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:20.905143 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:20.930809 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:20.932150 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:20.933660 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:20.905416 16115 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:19:20.906773 16740 consensus_meta.cc:287] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.000s
I20260709 13:19:20.908069 16829 raft_consensus.cc:993] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2: : Instructing follower 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:19:20.879554 16727 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:20.910642 16747 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:20.915146 16717 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:20.919320 16673 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:20.920207 12528 catalog_manager.cc:5697] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 5 to 9, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 9 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:20.922065 16664 raft_consensus.cc:3055] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:20.923508 16453 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:20.924256 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.895056 15998 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:20.925947 12700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1697) took 1258 ms. Trace:
I20260709 13:19:20.925420 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:20.947052 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:20.979241 13378 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:19:20.949640 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:20.953140 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 62261.9us
I20260709 13:19:20.980160 13377 raft_consensus.cc:1813] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Handling vote request from an unknown peer 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:20.981456 12625 raft_consensus.cc:1240] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:20.959507 13380 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:20.950806 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:20.926527 16791 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:20.964979 12782 raft_consensus.cc:1240] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:20.893602 16828 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:20.885649 16696 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:20.965821 16848 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:20.966225 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:20.974687 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 196702us
I20260709 13:19:20.978268 12700 rpcz_store.cc:276] 0709 13:19:19.667420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:20.039158 (+371738us) service_pool.cc:224] Handling call
0709 13:19:20.065759 (+ 26601us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:20.267522 (+201763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:20.925704 (+658182us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b540034f154. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:20.925891 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":462553344}
I20260709 13:19:20.982299 13377 raft_consensus.cc:2410] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:19:20.979315 12628 raft_consensus.cc:2393] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 10: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:20.985100 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:20.987129 13382 raft_consensus.cc:2393] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 11: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:20.988865 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:20.990746 13384 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 13: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:20.971807 16115 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:20.992393 16791 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:20.993758 16115 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:20.994392 16696 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:20.994406 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:20.996014 16673 consensus_queue.cc:1059] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:20.979530 16831 raft_consensus.cc:1081] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Signalling peer 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:19:20.975428 16664 raft_consensus.cc:740] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:21.004076 16664 consensus_queue.cc:260] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:21.005288 16664 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:21.021842 12597 leader_election.cc:400] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8.
I20260709 13:19:21.022776 12709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:55238 (request call id 17) took 1328 ms. Trace:
I20260709 13:19:21.023187 12709 rpcz_store.cc:276] 0709 13:19:19.706547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.888449 (+181902us) service_pool.cc:224] Handling call
0709 13:19:21.022555 (+1134106us) spinlock_profiling.cc:243] Waited 80.3 ms on lock 0x7b540021edd4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3ac853 00007f4c7237a60a 00007f4c6b9007d4 00007f4c6b900762 00007f4c6b900691 00007f4c6b90060c 00007f4c6b8ff8c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:21.022667 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":184764672}
I20260709 13:19:21.022383 12597 leader_election.cc:403] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:20.969547 16740 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:21.026571 16804 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8.
I20260709 13:19:21.027144 13379 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:21.027918 16740 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:21.025421 16831 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:21.029093 16827 log.cc:927] Time spent T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.335s	user 0.005s	sys 0.000s
I20260709 13:19:21.029762 16831 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:21.030498 16804 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:21.030949 16642 raft_consensus.cc:3055] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:21.031391 16642 raft_consensus.cc:740] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:21.032263 16831 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:21.032347 16642 consensus_queue.cc:260] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:21.032785 16804 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:21.038734 16804 consensus_queue.cc:1243] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Peer 3d539f8d95d343d8b1696803f18b30d2 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:19:21.039108 16841 log.cc:927] Time spent T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.292s	user 0.003s	sys 0.000s
I20260709 13:19:21.033552 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:21.040948 16804 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:21.041496 12775 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:19:21.042174 12775 raft_consensus.cc:606] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:19:21.042508 12775 raft_consensus.cc:614] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:21.044827 12598 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:21.002274 16848 raft_consensus.cc:697] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:21.046550 16693 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:21.046990 16765 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:21.046485 16848 consensus_queue.cc:237] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:21.050310 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:21.050881 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.054468 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:21.056288 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:21.059413 12709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:19:21.060055 12709 raft_consensus.cc:606] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:19:21.060385 12709 raft_consensus.cc:614] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:21.050326 16713 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:21.063918 16642 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:21.045989 16453 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:21.065114 16453 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:21.067011 16825 log.cc:927] Time spent T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.379s	user 0.000s	sys 0.002s
I20260709 13:19:21.068773 16453 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:21.069235 16771 raft_consensus.cc:3055] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:21.079727 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:21.080138 16771 raft_consensus.cc:740] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:21.081125 16771 consensus_queue.cc:260] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:21.082376 16771 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:21.072588 16779 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:21.074909 12746 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:21.086908 16642 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:21.087319 16642 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.087641 16642 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:21.089044 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:21.089628 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:21.090368 12775 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 11: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:21.091010 12528 catalog_manager.cc:5697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 7 to 10, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 10 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:21.093640 12779 raft_consensus.cc:1240] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:21.094521 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:21.093070 12528 catalog_manager.cc:5697] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 3 to 11, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 11 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:21.096992 12528 catalog_manager.cc:5697] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 6 to 8, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 8 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:21.098503 12528 catalog_manager.cc:5697] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 4 to 7, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 7 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:21.100760 12528 catalog_manager.cc:5697] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index 4 to 5, term changed from 4 to 6, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66), VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New cstate: current_term: 6 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:19:21.105983 16823 log.cc:927] Time spent T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.448s	user 0.004s	sys 0.000s
W20260709 13:19:21.117112 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:21.118585 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.119300 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:21.120030 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.076031 12598 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:21.124769 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 48822.6us
I20260709 13:19:21.129714 12598 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:21.133864 12774 raft_consensus.cc:1217] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Deduplicated request from leader. Original: 7.4->[8.5-11.7]   Dedup: 8.5->[9.6-11.7]
I20260709 13:19:21.134523 12774 pending_rounds.cc:85] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 5
I20260709 13:19:21.134781 12774 pending_rounds.cc:107] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:19:21.135016 12774 raft_consensus.cc:2889] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:21.136615 12598 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:21.137710 16453 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:21.156380 16700 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:21.148587 16648 consensus_queue.cc:1048] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:19:21.132513 16627 consensus_queue.cc:1059] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.042s
W20260709 13:19:21.195793 16838 log.cc:927] Time spent T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.373s	user 0.001s	sys 0.002s
W20260709 13:19:21.234820 16837 log.cc:927] Time spent T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.456s	user 0.004s	sys 0.000s
I20260709 13:19:21.234637 16777 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:21.201244 16700 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:21.248054 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
W20260709 13:19:21.252282 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.253827 16751 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.254536 16751 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:21.258229 16751 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:21.259011 16751 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:21.259634 16751 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:21.262995 12708 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:19:21.265190 16751 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:21.273656 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:21.274411 12709 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
I20260709 13:19:21.277202 12709 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:19:21.278371 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:21.279031 12708 raft_consensus.cc:2376] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11. Re-sending same reply.
I20260709 13:19:21.280846 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:21.282763 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:21.283717 12597 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:21.284057 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:21.285326 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:21.285596 12778 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:21.285939 12779 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:21.284543 16627 consensus_queue.cc:1059] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:21.286727 16627 raft_consensus.cc:3055] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:21.287061 16627 raft_consensus.cc:740] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:21.287370 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:21.287843 16627 consensus_queue.cc:260] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:21.288965 16627 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:21.289047 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:21.290632 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:21.291230 12779 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:21.292434 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:21.293026 12704 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:21.298672 16676 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:21.300125 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:21.300690 16804 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:21.301201 16804 raft_consensus.cc:697] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:21.302527 16804 consensus_queue.cc:237] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:21.304841 12709 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:21.306625 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:21.318079 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:21.318859 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:21.319741 16766 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:21.320410 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:21.324697 12672 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:21.340288 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:21.365816 16851 log.cc:927] Time spent T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:19:21.364940 16642 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:21.369331 12672 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:21.370507 12708 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:19:21.351383 16822 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:21.371532 12704 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:19:21.373531 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:19:21.377962 16847 log.cc:927] Time spent T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 13:19:21.369140 16693 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:21.381946 12672 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:21.383496 12672 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:21.378284 16664 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:21.393966 12704 raft_consensus.cc:1240] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:21.394879 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:21.410126 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:21.385239 16797 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.001s	sys 0.017s
I20260709 13:19:21.417312 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:21.394889 16751 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:21.419862 16771 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.440618 16771 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:21.423521 16642 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:21.424743 12781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36488 (request call id 2171) took 1142 ms. Trace:
I20260709 13:19:21.425752 16849 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:19:21.448050 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.384101 16648 raft_consensus.cc:3055] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:21.386634 16693 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:21.397166 16664 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.435920 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:21.452862 13381 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:21.438828 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:21.454088 12705 raft_consensus.cc:2376] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8. Re-sending same reply.
I20260709 13:19:21.439061 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:21.440613 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:21.386857 16822 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:21.446585 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:21.446826 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:21.444350 12781 rpcz_store.cc:276] 0709 13:19:20.282650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:20.285321 (+  2671us) service_pool.cc:224] Handling call
0709 13:19:21.424539 (+1139218us) spinlock_profiling.cc:243] Waited 41.9 ms on lock 0x7b54001fde54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:21.424670 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96484096}
I20260709 13:19:21.447834 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:19:21.436237 16797 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.001s	sys 0.018s
I20260709 13:19:21.450757 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:21.451125 12701 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:21.448251 16751 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:21.451490 16706 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.451874 16648 raft_consensus.cc:740] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:21.462687 13349 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:21.458017 12779 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:21.466677 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:21.455688 12706 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:21.462322 12782 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:19:21.464263 12709 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:21.450212 16849 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:21.415665 16634 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:21.478652 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:21.478368 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:21.481377 12781 raft_consensus.cc:2393] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 9: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:21.458073 16664 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:21.483913 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:21.486378 12672 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:21.488263 12775 raft_consensus.cc:2410] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:19:21.493340 12672 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:21.478507 16795 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:19:21.495633 12672 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:21.478626 16717 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:21.465018 16710 consensus_queue.cc:1059] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:19:21.478271 16822 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.482676 16797 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:21.482925 16771 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:21.485863 16849 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:21.497797 16755 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:21.473686 16751 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:21.501386 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:21.500299 16843 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:21.476596 16706 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:21.502161 16845 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:19:21.489187 16821 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.012s	sys 0.016s
I20260709 13:19:21.486410 16634 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:21.516389 13381 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
W20260709 13:19:21.636204 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 136625us
I20260709 13:19:21.593493 16706 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:21.557370 16676 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:21.571542 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:21.518318 16700 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:21.561086 16693 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:21.563738 16740 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.579303 12631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56838 (request call id 1617) took 1168 ms. Trace:
I20260709 13:19:21.751380 12631 rpcz_store.cc:276] 0709 13:19:20.411129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:20.411238 (+   109us) service_pool.cc:224] Handling call
0709 13:19:21.579108 (+1167870us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b54003458d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:21.579214 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":750647680}
I20260709 13:19:21.516940 16766 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.555365 16795 raft_consensus.cc:3055] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:21.504864 16717 raft_consensus.cc:697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:21.573671 12528 catalog_manager.cc:5697] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 8 to 12, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 12 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:21.725625 16671 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:21.680840 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.508688 16648 consensus_queue.cc:260] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:21.582578 16751 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:21.511883 16797 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:21.565111 16853 log.cc:927] Time spent T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
W20260709 13:19:21.606058 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:21.564574 16854 log.cc:927] Time spent T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:19:21.509079 16822 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:21.555294 16845 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:21.766153 16634 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:21.766404 16845 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:21.767077 16717 consensus_queue.cc:237] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:21.746572 12672 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:22.055084 16869 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.771083 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:22.069080 12627 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:21.775063 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:21.748749 16676 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:21.747615 16779 raft_consensus.cc:2764] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:19:21.755810 16740 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:21.756089 16766 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:21.747148 16821 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.998s	user 0.016s	sys 0.020s
I20260709 13:19:21.753674 16795 raft_consensus.cc:740] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:21.782893 16797 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:22.207826 16795 consensus_queue.cc:260] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:21.780342 16664 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:22.210263 12700 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 10. Ops: []
I20260709 13:19:22.211334 16664 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:21.761902 16849 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:21.758594 16671 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:22.213358 16849 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:22.213490 16671 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:21.909015 16717 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:22.214272 16717 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:22.216509 16717 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:21.909999 16845 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:22.216974 16779 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.217648 16779 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:22.218487 16845 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:21.920419 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:21.928985 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:21.951866 12625 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.893s	user 0.009s	sys 0.007s
W20260709 13:19:22.220240 12625 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.161s	user 0.010s	sys 0.007s
I20260709 13:19:22.220659 16797 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:22.221096 12625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 646) took 1169 ms. Trace:
I20260709 13:19:22.213508 16772 consensus_queue.cc:1059] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:19:22.221275 16713 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:22.222072 16772 consensus_queue.cc:1243] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer 522fd2ba0e9f402691b5ef29af8579cd log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:19:22.222576 16717 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:22.221498 12625 rpcz_store.cc:276] 0709 13:19:21.051644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.057377 (+  5733us) service_pool.cc:224] Handling call
0709 13:19:21.057722 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:21.058106 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:21.058137 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:21.058690 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 13:19:22.220768 (+1162078us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:22.220812 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:22.220881 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:22.220921 (+    40us) raft_consensus.cc:1685] finished
0709 13:19:22.220933 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:22.221045 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1257,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"thread_start_us":1161607,"threads_started":1,"wal-append.queue_time_us":517922}
I20260709 13:19:22.223142 16717 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:21.976763 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:22.224261 16772 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.989184 16453 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:22.224380 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 247714us
I20260709 13:19:21.984318 16633 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:22.225898 16633 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:22.228621 16633 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:22.229319 16848 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:21.991027 16776 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:22.029665 16700 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.231271 16776 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:22.224821 16772 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:22.040413 16697 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:19:22.234891 12532 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54814 (request call id 220) took 34209 ms (client timeout 15000 ms). Trace:
I20260709 13:19:22.234653 16697 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:22.236151 16772 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:22.236804 16697 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:22.055714 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 309191us
I20260709 13:19:22.238039 12672 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:22.239269 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:22.239348 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:22.240007 16671 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:22.240607 12672 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:22.240753 16671 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:22.241189 16671 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:22.241536 16671 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:22.055835 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 919287us
I20260709 13:19:22.244405 16845 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:22.244599 16778 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:22.244969 12598 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:21.760089 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:22.246119 12631 raft_consensus.cc:2410] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:19:22.246248 16777 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:22.058334 16839 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:22.247025 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.060612 16843 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:22.064397 16627 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:22.247936 16627 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:22.248337 16627 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:22.249739 16732 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:22.250905 16671 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:22.251341 16776 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:22.253039 16671 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:22.068552 16869 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:22.255198 16869 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:21.785619 16710 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:22.104892 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.109933 12528 catalog_manager.cc:5697] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 8 to 9, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 9 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:19:22.117085 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.257333 16710 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:22.258172 12528 catalog_manager.cc:5697] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 3 to 9, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 9 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:22.259527 12673 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:22.259753 12528 catalog_manager.cc:5697] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:22.261157 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:22.261749 16843 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:22.262466 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:22.151563 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.263007 16849 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.263459 16849 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:22.263948 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:22.264508 12631 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:22.143929 12778 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:19:22.155964 12628 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:22.154160 12746 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:22.159286 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:22.265853 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:22.181535 16740 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:22.195511 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:22.266774 16845 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.266834 12708 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:19:22.267413 16845 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:22.192989 16766 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:21.766216 16822 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:22.270126 16822 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:22.261459 12528 catalog_manager.cc:5697] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 4 to 5, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 5 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:22.222337 16713 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:22.271360 12528 catalog_manager.cc:5697] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 1 to 9, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 9 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:22.272166 16633 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:22.272855 16633 raft_consensus.cc:697] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:22.273450 16713 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:22.275660 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:22.276297 12701 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:19:22.277227 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:22.278096 12703 raft_consensus.cc:2393] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 8: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:22.278566 16633 consensus_queue.cc:237] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:22.279477 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:22.280205 12673 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:21.953709 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:22.221128 16797 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:22.224884 16453 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:22.281042 12630 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:22.282608 16797 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:22.231338 16700 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:22.283334 12629 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:19:22.283727 12672 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:22.031474 16693 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:22.284883 16693 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:22.285344 16755 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:22.287123 16693 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:22.261986 12632 raft_consensus.cc:2393] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 10: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:22.262297 16843 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:22.290072 16845 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:22.265938 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 111899us
I20260709 13:19:22.259568 16710 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:22.268144 12626 raft_consensus.cc:1275] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:19:22.290531 16843 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:22.158226 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:22.247048 16839 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:22.292637 16700 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:22.272827 12528 catalog_manager.cc:5697] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 6 to 7, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 7 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:22.294454 16839 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:22.282663 16453 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:21.774029 16648 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:22.119669 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:22.297340 16713 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.298005 16713 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:22.300261 16713 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:22.203075 12707 raft_consensus.cc:3055] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:22.147650 16676 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:22.293969 12528 catalog_manager.cc:5697] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 4 to 6, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 6 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:22.301911 12707 raft_consensus.cc:740] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:22.306485 12673 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:22.307804 12528 catalog_manager.cc:5697] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:22.289949 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:22.309217 12700 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:19:22.208936 16795 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:22.302564 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:22.346755 12625 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:22.304275 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:22.304833 12672 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:19:22.346690 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.349241 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:22.350297 12597 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.798s (SENT) [suppressed 24 similar messages]
I20260709 13:19:22.220173 16779 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:22.308306 12707 consensus_queue.cc:260] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:22.352917 12673 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:22.364764 12707 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:22.365057 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.317235 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:22.366830 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:22.367350 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 78592.6us
I20260709 13:19:22.368228 12672 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:22.339332 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:22.339898 16697 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:22.341866 16740 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:22.342945 12528 catalog_manager.cc:5697] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 6 to 7, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 7 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:22.344506 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:22.351259 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:22.322695 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:22.354250 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:22.372344 12700 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:19:22.293222 16717 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:19:22.378175 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:22.367458 12703 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:22.351406 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:22.380192 12626 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:22.381072 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:22.382304 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:22.377207 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:22.384131 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:22.385892 12708 raft_consensus.cc:1240] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:22.386897 12528 catalog_manager.cc:5697] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 5 to 7, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 7 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:22.388404 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:22.372411 12625 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 7.
I20260709 13:19:22.329324 12631 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
W20260709 13:19:22.390396 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 47866.2us
I20260709 13:19:22.391953 12704 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:19:22.393275 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:22.394387 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:22.394829 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:22.401404 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:22.407764 12708 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:22.395383 12776 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:22.420219 12528 catalog_manager.cc:5697] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 6 to 10, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 10 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:22.421679 12700 raft_consensus.cc:2393] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 10: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:22.268183 16849 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:22.434131 12528 catalog_manager.cc:5697] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:22.437714 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:22.440781 12703 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
W20260709 13:19:22.235268 12532 rpcz_store.cc:269] 0709 13:18:48.025328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:48.066797 (+ 41469us) service_pool.cc:224] Handling call
0709 13:19:22.234789 (+34167992us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:14.928749 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:15.003612 (+ 74863us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:15.003666 (+    54us) write_op.cc:435] Acquired schema lock
0709 13:19:15.003686 (+    20us) tablet.cc:662] Decoding operations
0709 13:19:15.004822 (+  1136us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:15.004929 (+   107us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:15.004968 (+    39us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:19:15.097444 (+ 92476us) tablet.cc:701] Row locks acquired
0709 13:19:15.115288 (+ 17844us) write_op.cc:260] PREPARE: finished
0709 13:19:15.139527 (+ 24239us) write_op.cc:270] Start()
0709 13:19:15.156309 (+ 16782us) write_op.cc:276] Timestamp: P: 1783603155134594 usec, L: 0
0709 13:19:15.166751 (+ 10442us) op_driver.cc:348] REPLICATION: starting
0709 13:19:15.397920 (+231169us) log.cc:844] Serialized 3219 byte log entry
0709 13:19:15.823073 (+425153us) op_driver.cc:464] REPLICATION: finished
0709 13:19:15.952211 (+129138us) write_op.cc:301] APPLY: starting
0709 13:19:15.952321 (+   110us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:15.952572 (+   251us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:15.952720 (+   148us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:15.976151 (+ 23431us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:15.976481 (+   330us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:19:15.976528 (+    47us) write_op.cc:312] APPLY: finished
0709 13:19:15.976876 (+   348us) log.cc:844] Serialized 65 byte log entry
0709 13:19:16.007050 (+ 30174us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:16.018643 (+ 11593us) write_op.cc:454] Released schema lock
0709 13:19:16.025939 (+  7296us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:16.081104 (+ 55165us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:16.084360 (+  3256us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":817714,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":128826,"apply.run_cpu_time_us":7640,"apply.run_wall_time_us":81102,"num_ops":8,"prepare.queue_time_us":320518,"prepare.run_cpu_time_us":34023,"prepare.run_wall_time_us":667346,"raft.queue_time_us":162026,"raft.run_cpu_time_us":42151,"raft.run_wall_time_us":100950,"replication_time_us":656159,"thread_start_us":509384,"threads_started":4,"wal-append.queue_time_us":186201,"wal-append.run_cpu_time_us":28061,"wal-append.run_wall_time_us":1394709}]]}
I20260709 13:19:22.441970 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:22.420233 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:22.435153 12673 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:22.444687 12629 raft_consensus.cc:2410] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:19:22.442653 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:22.446789 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:22.428226 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:22.434020 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:22.447456 12702 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:19:22.443768 12701 raft_consensus.cc:1275] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:22.448155 12781 raft_consensus.cc:2393] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 9: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:22.450925 12673 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:22.386165 16772 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:22.375070 16755 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:22.488354 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:22.408360 16776 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.424345 12707 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:19:22.385979 16843 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:22.408390 16839 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:22.374696 16671 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:22.486045 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:22.474974 12630 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:19:22.465306 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:22.475309 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:22.490461 12707 pending_rounds.cc:85] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 8
I20260709 13:19:22.407960 16804 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:22.507834 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:22.510977 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:22.520210 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:22.496117 12528 catalog_manager.cc:5697] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 4 to 5, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 5 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:22.506102 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:22.520900 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:22.510104 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:22.511431 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:22.522215 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:22.521817 12528 catalog_manager.cc:5697] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 7 to 12, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 12 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:22.489087 12777 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:22.501642 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:22.524154 12702 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:22.520829 12779 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:22.525107 12630 raft_consensus.cc:1275] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:19:22.526209 12707 pending_rounds.cc:107] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:19:22.507123 12629 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:22.453472 12673 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:22.527902 12707 raft_consensus.cc:2889] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:22.529032 12707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 505) took 1273 ms. Trace:
I20260709 13:19:22.529687 12707 rpcz_store.cc:276] 0709 13:19:21.255860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:21.261769 (+  5909us) service_pool.cc:224] Handling call
0709 13:19:21.262186 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:22.202972 (+940786us) spinlock_profiling.cc:243] Waited 917 ms on lock 0x7b54000189a0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d329dc0 00007f4c6d3b41c0 00007f4c6d3b5231 00007f4c6d3b1d8f 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c
0709 13:19:22.528572 (+325600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:22.528962 (+   390us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2108118272}
I20260709 13:19:22.526441 12598 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:22.532855 12598 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:19:22.501935 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.536067 12628 raft_consensus.cc:2410] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:19:22.489754 16772 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:22.614586 16772 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:22.622301 16772 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:22.625240 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:22.408093 16697 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:22.626891 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 94159.7us
I20260709 13:19:22.634305 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:22.517179 16845 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:22.628659 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:22.539587 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:22.641711 12528 catalog_manager.cc:5697] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 5 to 7, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 7 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:22.643182 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:22.489311 16822 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:19:22.608821 16761 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:19:22.510495 12632 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce"
dest_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:56838
I20260709 13:19:22.495934 16732 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.145s
W20260709 13:19:22.725180 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 315580us
I20260709 13:19:22.685840 12703 raft_consensus.cc:2393] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
W20260709 13:19:22.695425 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.656761 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:22.735747 12631 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:19:22.736357 12632 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:22.644887 12782 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
W20260709 13:19:22.735770 12709 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.022s	sys 0.074s
W20260709 13:19:22.739907 12709 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.022s	sys 0.074s
I20260709 13:19:22.736824 12632 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:22.778365 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.534402 16777 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:22.797813 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.803045 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:22.807569 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:22.808298 12707 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:22.809248 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:22.810971 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:22.804404 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 454100us
I20260709 13:19:22.818636 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:22.819958 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:22.820442 12707 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:22.821231 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:22.822008 12707 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:22.537593 16804 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:22.823839 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:22.502219 16755 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:22.609053 16797 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:22.864070 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 513441us
I20260709 13:19:23.021824 13348 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:23.023491 16772 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:23.028168 16849 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:22.909008 12528 catalog_manager.cc:5697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 8 to 12, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 12 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:22.870853 12632 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:22.983709 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:22.802181 12655 debug-util.cc:398] Leaking SignalData structure 0x7b080027b480 after lost signal to thread 11520
W20260709 13:19:22.984032 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 530607us
I20260709 13:19:22.518581 16776 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:22.646068 16633 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:22.803403 16781 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.734990 16845 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:22.506174 16848 consensus_queue.cc:1048] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.308s
I20260709 13:19:22.737948 16822 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.053547 16822 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:22.684783 16766 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:23.062536 16766 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:23.063179 16766 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:22.538115 16869 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.335s
I20260709 13:19:22.698884 16843 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:22.518692 16671 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:23.006603 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:23.015606 12778 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:23.005875 12701 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
W20260709 13:19:23.046957 16627 consensus_meta.cc:287] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.795s	user 0.009s	sys 0.001s
W20260709 13:19:23.034013 12655 debug-util.cc:398] Leaking SignalData structure 0x7b08001bc720 after lost signal to thread 12566
I20260709 13:19:23.039902 12706 raft_consensus.cc:1275] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:19:23.180434 12655 debug-util.cc:398] Leaking SignalData structure 0x7b08003c19e0 after lost signal to thread 12658
W20260709 13:19:23.180850 12655 debug-util.cc:398] Leaking SignalData structure 0x7b0800283de0 after lost signal to thread 12659
I20260709 13:19:23.016852 12521 maintenance_manager.cc:419] P 480284ff372740a09dfd1f972d55d7eb: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033379
I20260709 13:19:23.044561 16781 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:23.045239 16845 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:23.045459 16633 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:23.069746 16710 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:23.069343 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:23.213466 12631 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:23.088096 16869 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.080998 12632 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:23.032784 12528 catalog_manager.cc:5697] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:19:23.131950 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.132177 12703 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:19:23.235222 12703 pending_rounds.cc:85] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 8
I20260709 13:19:23.235653 12703 pending_rounds.cc:107] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:19:23.235935 12703 raft_consensus.cc:2889] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:23.183496 12655 debug-util.cc:398] Leaking SignalData structure 0x7b08002df0a0 after lost signal to thread 12733
W20260709 13:19:23.237402 12655 debug-util.cc:398] Leaking SignalData structure 0x7b0800268b20 after lost signal to thread 12734
W20260709 13:19:23.238313 12598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.003s	sys 0.028s
W20260709 13:19:23.185107 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 112871us
W20260709 13:19:23.238787 12598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.003s	sys 0.028s
I20260709 13:19:23.064651 16779 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:23.111210 16671 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:23.037092 16751 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:23.108693 16804 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:23.043149 16797 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:23.247723 13349 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.001s	sys 0.012s
W20260709 13:19:23.248174 13349 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.001s	sys 0.013s
W20260709 13:19:23.243311 12672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.005s	sys 0.006s
I20260709 13:19:23.076555 16717 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.137s
I20260709 13:19:23.200554 16843 raft_consensus.cc:697] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:19:23.238632 12655 debug-util.cc:398] Leaking SignalData structure 0x7b08003cf900 after lost signal to thread 12807
W20260709 13:19:23.254081 12655 debug-util.cc:398] Leaking SignalData structure 0x7b08000f0d40 after lost signal to thread 12808
W20260709 13:19:23.255678 12655 debug-util.cc:398] Leaking SignalData structure 0x7b08003f4c40 after lost signal to thread 13410
I20260709 13:19:23.186945 16627 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:23.246515 12528 catalog_manager.cc:5697] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
W20260709 13:19:23.266320 12655 debug-util.cc:398] Leaking SignalData structure 0x7b08003e38c0 after lost signal to thread 16811
I20260709 13:19:23.239667 16693 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.283654 12631 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:23.242182 16777 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:23.285105 12674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.011s	sys 0.001s
I20260709 13:19:23.442571 12528 catalog_manager.cc:5697] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 4 to 6, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 6 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
W20260709 13:19:23.443454 12674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.012s	sys 0.001s
W20260709 13:19:23.249948 12672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.005s	sys 0.006s
W20260709 13:19:23.444686 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 787946us
I20260709 13:19:23.212858 16713 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:23.448568 16713 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:23.449582 16697 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:23.450145 16697 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:23.450764 16713 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:23.286778 12747 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.010s	sys 0.022s
W20260709 13:19:23.451361 12747 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.010s	sys 0.022s
I20260709 13:19:23.452387 16776 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:23.452966 16776 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:23.249665 12627 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.803s	user 0.009s	sys 0.003s
W20260709 13:19:23.453734 12627 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.007s	user 0.009s	sys 0.003s
I20260709 13:19:23.454278 13378 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:23.454533 12627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1628) took 1009 ms. Trace:
I20260709 13:19:23.455006 12627 rpcz_store.cc:276] 0709 13:19:22.445179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.445307 (+   128us) service_pool.cc:224] Handling call
0709 13:19:22.445637 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:22.445857 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:22.446097 (+   240us) log.cc:844] Serialized 14 byte log entry
0709 13:19:23.454159 (+1008062us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:23.454224 (+    65us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:23.454282 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.454354 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:23.454477 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1119,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"thread_start_us":1007741,"threads_started":1}
I20260709 13:19:23.247107 16778 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:23.456815 16778 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:23.457260 16886 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:23.457861 16886 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:23.458911 16778 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:23.459419 16778 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.459285 16713 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:23.459903 16778 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:23.460260 16886 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:23.457856 16777 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:23.460870 16671 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:23.290934 12655 debug-util.cc:398] Leaking SignalData structure 0x7b08003c66e0 after lost signal to thread 16820
I20260709 13:19:23.241693 16869 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:23.332127 16676 consensus_meta.cc:287] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.994s	user 0.010s	sys 0.002s
I20260709 13:19:23.322425 12673 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:23.291699 16821 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.439402 12634 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:23.367925 16843 consensus_queue.cc:237] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:23.441890 12702 consensus_meta.cc:287] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.908s	user 0.010s	sys 0.001s
W20260709 13:19:23.248667 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 513149us
I20260709 13:19:23.463358 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:23.464699 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:23.485179 13378 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 7.
I20260709 13:19:23.452394 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:23.470331 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:23.472533 12655 debug-util.cc:398] Leaking SignalData structure 0x7b08002a6c00 after lost signal to thread 16826
I20260709 13:19:23.485314 12702 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
W20260709 13:19:23.480719 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 158351us
I20260709 13:19:23.486493 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1729) took 1040 ms. Trace:
I20260709 13:19:23.465591 12598 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:23.486088 12655 debug-util.cc:398] Leaking SignalData structure 0x7b08003da000 after lost signal to thread 16846
I20260709 13:19:23.454447 16761 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.489925 16869 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:23.491870 16891 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:23.492270 16891 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:23.492602 16891 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:23.493422 12672 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:23.493937 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:23.494323 16849 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:23.494525 12778 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:23.467681 16889 log.cc:927] Time spent T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.181s	user 0.000s	sys 0.005s
I20260709 13:19:23.486829 12702 rpcz_store.cc:276] 0709 13:19:22.445970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.448658 (+  2688us) service_pool.cc:224] Handling call
0709 13:19:23.486421 (+1037763us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:23.503479 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:23.508085 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:23.501535 12598 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:23.517202 12598 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:23.518028 12598 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:23.518955 12598 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:23.450619 16697 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:23.476421 16874 log.cc:927] Time spent T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.263s	user 0.001s	sys 0.004s
I20260709 13:19:23.518816 12528 catalog_manager.cc:5697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 6 to 10, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 10 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:23.472282 16886 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:23.531833 12746 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:19:23.522782 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 70483us
I20260709 13:19:23.524405 12528 catalog_manager.cc:5697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:19:23.466388 16648 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.014s	sys 0.031s
I20260709 13:19:23.526852 12634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1625) took 1101 ms. Trace:
I20260709 13:19:23.532970 12747 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:23.453382 16776 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:23.535672 12778 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:23.533414 12634 rpcz_store.cc:276] 0709 13:19:22.425831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.426934 (+  1103us) service_pool.cc:224] Handling call
0709 13:19:22.427255 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.481787 (+1054532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.526572 (+ 44785us) spinlock_profiling.cc:243] Waited 915 ms on lock 0x7b540004a1d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:23.526779 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2105016576}
I20260709 13:19:23.537552 12672 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:23.542171 13349 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:23.496354 16664 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:23.543452 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:23.542759 12528 catalog_manager.cc:5697] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 5 to 6, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 6 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:23.544219 12672 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:23.466039 16693 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:23.549275 12672 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:23.522505 16779 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:23.557866 12528 catalog_manager.cc:5697] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:23.531922 16755 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:23.538677 16648 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.014s	sys 0.032s
I20260709 13:19:23.559265 12528 catalog_manager.cc:5697] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 5 to 9, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 9 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:23.555423 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 46338.6us
W20260709 13:19:23.564495 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 44738.9us
I20260709 13:19:23.545192 16453 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:23.543959 16778 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:23.482618 16676 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:23.540701 16892 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:23.573645 12704 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:19:23.566672 12747 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:23.514110 16761 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:23.564879 16895 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:23.546644 16627 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:23.546416 16897 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:23.561079 16875 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:23.601109 16901 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:23.621628 16901 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.621996 16901 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:23.622822 16895 raft_consensus.cc:697] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:23.623816 16895 consensus_queue.cc:237] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:23.624832 16903 log.cc:927] Time spent T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:19:23.630455 16888 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:23.631047 16888 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:23.578203 16634 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:23.557988 16848 consensus_queue.cc:1048] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:19:23.497359 16821 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:23.546950 16839 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:23.556128 16732 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:23.662883 16693 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:23.663976 16797 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:23.664422 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:23.664604 16821 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:23.618930 16845 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:23.668386 16845 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:23.670042 16905 log.cc:927] Time spent T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.116s	user 0.002s	sys 0.000s
I20260709 13:19:23.620664 16761 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:23.671386 16795 consensus_meta.cc:287] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 1.296s	user 0.015s	sys 0.000s
I20260709 13:19:23.662825 16732 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.672137 13381 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:23.600857 16892 consensus_queue.cc:1059] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 13:19:23.565634 16779 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:23.601830 16907 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:23.598507 16899 raft_consensus.cc:2764] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:19:23.611322 16902 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:23.675565 16697 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:23.676209 16902 raft_consensus.cc:3055] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:23.676441 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:23.676789 16902 raft_consensus.cc:740] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:23.615366 16648 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:23.677814 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:23.677918 16648 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.543390 16886 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:23.678567 16648 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:23.679351 16776 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:23.680634 16886 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:23.639923 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:23.611353 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:23.687927 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:23.689056 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:23.547020 16881 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:19:23.633802 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:23.662760 13376 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:19:23.634706 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:23.607227 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:23.642002 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:23.665095 12631 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:23.578104 12528 catalog_manager.cc:5697] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 7 to 9, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 9 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:23.615801 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:23.638846 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:23.643111 16911 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:23.702548 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:23.702346 16911 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:23.703948 16849 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:23.704540 16849 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:23.616961 16751 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:23.705789 16751 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:23.709072 16779 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:23.701701 16627 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.709841 16627 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:23.710891 16888 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:23.719596 13381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51768 (request call id 674) took 1402 ms. Trace:
I20260709 13:19:23.719817 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:23.721211 16627 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:23.724395 12673 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:23.726087 16917 log.cc:927] Time spent T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:19:23.725946 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:23.726800 12632 raft_consensus.cc:2393] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 11: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:23.728077 16839 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:23.728500 16839 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.728874 16839 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:23.705416 16911 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:23.729939 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:23.731084 12777 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:23.651036 16875 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:23.734551 16849 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:23.735041 12704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38496 (request call id 524) took 1342 ms. Trace:
I20260709 13:19:23.735589 12704 rpcz_store.cc:276] 0709 13:19:22.392942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.394518 (+  1576us) service_pool.cc:224] Handling call
0709 13:19:23.734822 (+1340304us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540021f2d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:23.734967 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2709005568}
I20260709 13:19:23.737542 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:23.646046 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:23.740129 12778 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:23.603487 16676 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:23.643244 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:23.638543 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:23.751456 12780 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:23.638944 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:23.752523 12782 raft_consensus.cc:3055] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:23.573930 16900 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:19:23.643682 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 86416.5us
I20260709 13:19:23.601049 16633 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:23.606635 16896 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.051s
W20260709 13:19:23.657949 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 60603.5us
I20260709 13:19:23.673281 16892 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:23.672103 16732 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:23.673954 16907 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.681803 16795 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.677547 13385 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 12: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:23.566485 16664 raft_consensus.cc:697] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:23.601652 16777 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:23.683662 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:23.677943 16902 consensus_queue.cc:260] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:23.684875 12708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 2055) took 1030 ms. Trace:
I20260709 13:19:23.808987 12628 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:23.807969 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:19:23.606966 16894 log.cc:927] Time spent T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:19:23.686780 16906 log.cc:927] Time spent T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:19:23.689965 16781 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:23.568166 16883 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.129s
I20260709 13:19:23.686781 16901 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:23.692386 16822 consensus_meta.cc:287] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.638s	user 0.014s	sys 0.001s
I20260709 13:19:23.694017 16804 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.697889 12775 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:19:23.699548 12779 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:23.742515 13380 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:23.720158 13381 rpcz_store.cc:276] 0709 13:19:22.317261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.317426 (+   165us) service_pool.cc:224] Handling call
0709 13:19:23.719362 (+1401936us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54004d2954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:23.719519 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2990774656}
I20260709 13:19:23.752874 12782 raft_consensus.cc:740] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:19:23.761606 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.763527 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:23.763715 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:23.767092 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:23.768580 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:23.768937 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:23.782172 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:23.848143 12631 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:23.782574 16697 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:23.680676 16671 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:23.785722 16899 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:23.786170 16885 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:23.665446 16634 raft_consensus.cc:697] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:23.786749 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:23.787737 12626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1626) took 1350 ms. Trace:
I20260709 13:19:23.788403 16751 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:23.789321 16676 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:23.790058 12516 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 4c13fa2aff324f65ad0bf6191ea68fab with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 3d539f8d95d343d8b1696803f18b30d2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:23.791574 16890 consensus_queue.cc:1059] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:23.769999 16900 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:23.800608 12747 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:23.785768 16766 consensus_meta.cc:287] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.722s	user 0.013s	sys 0.000s
I20260709 13:19:23.806301 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:23.811427 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:23.812616 12673 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:23.810665 12708 rpcz_store.cc:276] 0709 13:19:22.653995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.059468 (+405473us) service_pool.cc:224] Handling call
0709 13:19:23.059823 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:23.060156 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:23.060186 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:23.060680 (+   494us) log.cc:844] Serialized 26 byte log entry
0709 13:19:23.442813 (+382133us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:23.442851 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.442913 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:23.624102 (+181189us) spinlock_profiling.cc:243] Waited 42.3 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:23.684551 (+ 60449us) raft_consensus.cc:1685] finished
0709 13:19:23.684578 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:23.684809 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21259,"raft.queue_time_us":4606,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":57814,"spinlock_wait_cycles":97345792,"thread_start_us":381850,"threads_started":1,"wal-append.queue_time_us":484365}
I20260709 13:19:23.826514 16886 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:23.829939 16902 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:23.875008 16886 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:23.799163 16895 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.833491 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:23.877933 13378 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:23.877825 16895 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:23.850203 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 301358us
I20260709 13:19:23.879006 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:23.804942 16907 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:23.879844 16895 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:23.868256 16900 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:23.810285 16912 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:23.799188 16776 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:19:23.881762 16766 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:23.882148 16912 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:23.871599 16914 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:23.883708 12700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 2060) took 1228 ms. Trace:
I20260709 13:19:23.884068 16901 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:23.840714 16822 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:23.884116 12700 rpcz_store.cc:276] 0709 13:19:22.655162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.069639 (+414477us) service_pool.cc:224] Handling call
0709 13:19:23.070000 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:23.184976 (+114976us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:23.185023 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:23.200637 (+ 15614us) log.cc:844] Serialized 26 byte log entry
0709 13:19:23.348906 (+148269us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:23.348965 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.349062 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:23.601936 (+252874us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x7b48001b0258. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c722dbaa1 00007f4c722d2b37 00007f4c722d29ca 00007f4c723ca26f 00007f4c723c9f37 00007f4c723de55c 00007f4c723de508 00007f4c723de488 00007f4c723de443 00007f4c723dd6ec 00007f4c71307a96
0709 13:19:23.883361 (+281425us) raft_consensus.cc:1685] finished
0709 13:19:23.883384 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:23.883648 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":114552,"raft.queue_time_us":127895,"raft.run_cpu_time_us":3177,"raft.run_wall_time_us":400524,"spinlock_wait_cycles":5110016,"thread_start_us":100202,"threads_started":1,"wal-append.queue_time_us":325977}
I20260709 13:19:23.869496 16891 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:23.887408 12707 raft_consensus.cc:1240] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:23.887889 16891 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:23.889083 12707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1728) took 1466 ms. Trace:
I20260709 13:19:23.873492 16664 consensus_queue.cc:237] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:23.891501 16671 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:23.891939 16671 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.869139 16676 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:23.894160 16676 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:23.897886 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:23.898770 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:23.900144 16887 raft_consensus.cc:3055] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:23.900458 16895 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.901187 16895 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:23.901889 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:23.851490 16804 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:23.903188 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:23.903844 12778 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:23.904399 16804 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:23.904660 16821 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:23.867574 16885 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:23.905161 16821 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:23.905491 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:23.906244 12779 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:23.907012 16821 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:23.907119 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:23.907012 16885 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:23.864507 16899 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:23.854970 12782 consensus_queue.cc:260] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:23.909403 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:23.910090 12782 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:23.869056 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 68572.2us
I20260709 13:19:23.910763 12747 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:23.911542 12747 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:23.810890 16795 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:23.866330 12516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 76448.7us
I20260709 13:19:23.916622 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:23.874055 16634 consensus_queue.cc:237] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:23.804900 16892 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:23.892454 16671 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:23.920410 12746 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:23.926180 13377 raft_consensus.cc:2410] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:19:23.928906 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:23.900533 16887 raft_consensus.cc:740] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:23.903277 16895 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:23.912376 12672 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:23.941088 12672 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:23.942675 12672 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:23.870689 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 58165.9us
I20260709 13:19:23.889478 12707 rpcz_store.cc:276] 0709 13:19:22.422580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.822500 (+399920us) service_pool.cc:224] Handling call
0709 13:19:22.822713 (+   213us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:23.887820 (+1065107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.888864 (+  1044us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54002490d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:23.889026 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2448172800}
I20260709 13:19:23.915041 13349 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:23.858747 12626 rpcz_store.cc:276] 0709 13:19:22.436945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.440508 (+  3563us) service_pool.cc:224] Handling call
0709 13:19:22.499803 (+ 59295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:22.703029 (+203226us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:19:22.813643 (+110614us) log.cc:844] Serialized 14 byte log entry
0709 13:19:23.626179 (+812536us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b1800372100. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6cf5a4bc 00007f4c6a2a848b 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf5223a
0709 13:19:23.651414 (+ 25235us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:23.651467 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:23.651510 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.787461 (+135951us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b54002eb8d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:23.787515 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:23.787663 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24991,"prepare.queue_time_us":110901,"prepare.run_cpu_time_us":5907,"prepare.run_wall_time_us":45964,"raft.queue_time_us":36075,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"spinlock_wait_cycles":505304832,"thread_start_us":191144,"threads_started":1,"wal-append.queue_time_us":424597}
I20260709 13:19:23.946597 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:23.946717 12673 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:23.946954 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:23.947763 13379 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:23.912755 12747 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:23.948774 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:23.949007 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:23.949913 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:23.950101 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:23.950388 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:23.950455 12628 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:23.950937 12631 raft_consensus.cc:2410] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:19:23.877410 16886 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:23.921804 16899 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:23.922114 16693 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:23.921855 16821 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:23.859726 16697 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:23.819871 16648 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:23.891961 16849 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:23.930841 12782 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:23.937199 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:23.951095 13385 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 13: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:23.950120 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:23.952142 12707 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:19:23.958384 16761 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:23.923795 16822 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:23.952354 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:23.884354 16766 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:23.955597 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:23.956766 12706 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:23.957338 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:23.962136 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:23.964622 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:23.932597 16915 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:19:23.861133 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 68650.5us
I20260709 13:19:23.943653 16797 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:23.982250 12776 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:23.987893 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:24.008458 12775 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:23.990420 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:23.993355 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:24.010890 12706 raft_consensus.cc:2393] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 13: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:23.994351 12705 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:23.999361 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 50372.4us
I20260709 13:19:24.001931 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:23.923813 16912 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:23.958045 16904 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:23.987929 12779 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:23.982409 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:24.023036 12633 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:23.994048 12625 raft_consensus.cc:2393] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 12: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:24.006283 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:23.958590 16453 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:23.958163 16845 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:24.016072 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:23.996717 12597 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:24.030321 12597 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:24.031533 12597 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:23.989836 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:24.025153 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:24.026167 12634 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 11. Current term is 12. Ops: [11.12-11.12]
I20260709 13:19:23.961141 16751 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:24.026187 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:23.961310 16693 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:24.043551 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 64902.3us
I20260709 13:19:24.043639 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:24.053515 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:24.054128 12706 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:24.015772 12628 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:24.049089 12779 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:23.960958 16895 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:23.991503 16761 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.923830 16907 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:24.060680 12705 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:24.002753 16797 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:24.033402 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:24.073499 12774 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:24.075964 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:24.079128 12705 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:24.079937 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:24.080577 12705 raft_consensus.cc:2393] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 11: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:24.055758 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:23.994966 16915 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:24.018023 16848 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:19:24.078408 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 45101.9us
I20260709 13:19:24.084208 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:24.085301 12704 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 9: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:24.078109 12673 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:24.025789 16904 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:24.043639 16776 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:24.043829 16890 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:24.074729 16761 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:24.086588 12706 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:24.059504 16693 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:24.083783 16797 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:24.090163 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:24.084324 12597 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:24.052781 16453 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:24.055452 16920 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:24.091893 12632 raft_consensus.cc:1240] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:19:24.061645 16849 raft_consensus.cc:697] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:24.059907 16895 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:24.118959 16895 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:24.118888 16920 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:24.119364 16849 consensus_queue.cc:237] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:24.120129 16848 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.120714 16848 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:23.977540 16732 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:24.121551 16920 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:24.098304 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:24.098448 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:24.089373 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:24.144829 12704 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
W20260709 13:19:24.100958 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 48276.2us
I20260709 13:19:24.146060 12672 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:24.085529 16915 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:24.085634 16901 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:24.102697 12705 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:19:24.058014 16804 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.084161 16795 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:24.102866 12597 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:24.093251 16904 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:24.087754 16891 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:24.122208 13382 raft_consensus.cc:2368] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12.
I20260709 13:19:24.052380 16921 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:24.123718 12633 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:24.115931 16843 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:24.118099 16899 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:24.145097 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:24.266309 16901 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.197958 16732 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:24.200593 16822 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:24.197367 16671 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:24.246209 12709 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:24.272065 16776 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:19:24.290205 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 276726us
I20260709 13:19:24.250262 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:24.291312 12700 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:24.200016 16900 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:24.198724 16648 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:24.267059 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 164265us
I20260709 13:19:24.274989 16921 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.276302 12673 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:24.268491 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.280279 12706 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
W20260709 13:19:24.198951 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 52972.1us
I20260709 13:19:24.091442 16908 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:19:24.298955 12672 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:24.279603 16843 raft_consensus.cc:697] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:19:24.192842 16924 log.cc:927] Time spent T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:19:24.284418 16901 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:24.308017 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:24.310880 12775 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:24.308665 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:24.295316 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:24.295951 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:24.269207 16804 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:24.299297 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:24.097519 16875 consensus_queue.cc:1059] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:19:24.300366 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:24.300244 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:24.300726 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:24.322836 12782 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 5.
I20260709 13:19:24.281687 16899 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.302983 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:24.292327 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:24.305037 12634 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:24.312649 12783 raft_consensus.cc:1240] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:24.313706 13380 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:24.296603 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:24.309229 12672 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:24.315140 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:24.305006 12703 raft_consensus.cc:2368] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 13. Current term is 14.
I20260709 13:19:24.305233 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:24.330984 12777 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:24.323681 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:24.303604 16921 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:24.325358 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:24.302881 16776 raft_consensus.cc:697] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:24.337920 12672 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:24.307540 16892 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:24.305177 16822 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:24.314659 16697 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:24.321192 16700 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:19:24.348871 16839 consensus_meta.cc:287] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.620s	user 0.010s	sys 0.000s
I20260709 13:19:24.363653 16839 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:24.350821 16772 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:24.341933 16751 consensus_queue.cc:1059] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:19:24.365808 16843 consensus_queue.cc:237] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:24.366134 16887 consensus_queue.cc:260] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:24.367308 16887 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:24.351823 12781 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
W20260709 13:19:24.354171 16926 log.cc:927] Time spent T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 13:19:24.316365 16907 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:24.359826 12777 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:24.354954 16915 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:24.342214 16898 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:24.358086 16781 raft_consensus.cc:3055] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:24.332832 16899 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:24.305104 16732 consensus_queue.cc:1059] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:19:24.339046 16888 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:24.398854 16888 raft_consensus.cc:697] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 9 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:24.399677 16671 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:24.399824 16888 consensus_queue.cc:237] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:24.401338 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:24.398260 16751 raft_consensus.cc:3055] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:24.304380 16931 consensus_queue.cc:1059] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 13:19:24.418278 16931 raft_consensus.cc:3055] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:24.418747 16931 raft_consensus.cc:740] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:24.401940 12633 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:24.385308 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.404506 16751 raft_consensus.cc:740] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:24.374843 16904 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:24.384672 12700 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:24.361738 16700 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:24.371279 16822 raft_consensus.cc:697] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:24.390482 16781 raft_consensus.cc:740] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:24.434791 16740 raft_consensus.cc:3055] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:19:24.459623 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 130119us
I20260709 13:19:24.447124 12780 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:24.436462 16453 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:24.419734 16931 consensus_queue.cc:260] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:24.437287 16700 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:24.402711 16776 consensus_queue.cc:237] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:24.462381 12747 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:24.464540 16839 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:24.464099 16751 consensus_queue.cc:260] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:24.486531 16751 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:24.489880 16822 consensus_queue.cc:237] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:24.467469 16781 consensus_queue.cc:260] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:24.458928 16740 raft_consensus.cc:740] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:24.497937 16713 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:24.513886 16740 consensus_queue.cc:260] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:24.518874 16740 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:24.519120 16797 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:24.521163 16797 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:24.515290 16900 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:24.524636 16761 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:24.529808 16895 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:24.532137 16895 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:24.533180 16904 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:24.527205 16901 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:24.537976 16921 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:24.540300 16848 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:24.541185 16804 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:24.542704 16901 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:24.544732 16848 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:24.546278 16915 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:24.551239 16931 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:24.556288 16899 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:24.560659 16899 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:24.608924 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 270813us
I20260709 13:19:24.649362 16634 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.650527 12672 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:24.635466 16822 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.652663 12672 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:24.653153 16822 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:24.653949 16913 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:24.654250 16845 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:24.654654 16913 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:24.654739 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:24.655463 16913 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:24.656177 12626 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:24.657514 12672 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:24.659305 16845 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:24.617090 12707 raft_consensus.cc:2393] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 13: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:24.660025 16845 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:24.661119 16845 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:24.565138 16921 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:24.617276 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 119367us
I20260709 13:19:24.650528 16634 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:24.665967 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:24.619392 12627 raft_consensus.cc:1275] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:19:24.663676 12628 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:19:24.680590 16761 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:24.682061 12708 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 13: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:24.684334 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:24.684747 12709 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:19:24.685451 12634 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:19:24.685696 16940 consensus_queue.cc:1048] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:24.688035 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:24.687954 16940 consensus_queue.cc:1059] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:24.689263 16933 raft_consensus.cc:3055] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:24.693993 12625 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:24.499465 16693 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:24.689724 16933 raft_consensus.cc:740] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:24.702188 12776 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:24.702802 16933 consensus_queue.cc:260] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:24.704351 16933 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:24.707749 16761 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:24.708490 16878 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:24.714164 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:24.714277 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:24.716768 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:24.659821 16781 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:24.720485 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:24.721472 12629 raft_consensus.cc:2393] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 13: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:24.721383 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:24.723860 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:24.726192 12707 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:24.726671 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:24.725894 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:24.732823 12625 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:24.725955 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:24.741374 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:24.742422 12709 raft_consensus.cc:2393] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
W20260709 13:19:24.743549 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.747377 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:24.748345 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:24.748876 16951 log.cc:927] Time spent T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb: Append to log took a long time: real 0.133s	user 0.000s	sys 0.003s
I20260709 13:19:24.756716 16634 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:24.757659 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 60731.3us
I20260709 13:19:24.758940 12597 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:24.761492 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:24.763173 12627 raft_consensus.cc:2393] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:24.763466 12597 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:24.771368 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:24.777051 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 119925us
I20260709 13:19:24.783759 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:24.788098 12707 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:24.791067 16822 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:24.793095 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:24.799964 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:24.811159 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:24.811386 12707 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:24.737380 16693 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:24.814841 12634 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:24.818388 12626 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
W20260709 13:19:24.819391 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 56073.5us
W20260709 13:19:24.831300 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:24.833890 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.839066 13384 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:24.840715 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:24.845849 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:24.832854 16913 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:24.849648 13382 raft_consensus.cc:2393] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:24.854630 16887 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:24.865760 12673 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:24.867462 12673 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:24.868942 16761 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:24.869578 16761 raft_consensus.cc:697] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:24.870980 16761 consensus_queue.cc:237] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:24.873884 16891 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:24.874557 16891 raft_consensus.cc:697] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:24.875922 16891 consensus_queue.cc:237] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:24.882233 12747 connection.cc:256] RPC call timeout handler was delayed by -2.5980951999999888s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
W20260709 13:19:24.882781 12747 connection.cc:256] RPC call timeout handler was delayed by -2.5935951999999816s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1532s for a response
I20260709 13:19:24.858345 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:24.885950 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:24.885943 12747 connection.cc:256] RPC call timeout handler was delayed by -2.5755951999999525s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1552s for a response
W20260709 13:19:24.887948 12747 connection.cc:256] RPC call timeout handler was delayed by -2.3937951999998859s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1754s for a response
W20260709 13:19:24.888335 12747 connection.cc:256] RPC call timeout handler was delayed by -2.391995199999883s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1756s for a response
I20260709 13:19:24.889297 12707 raft_consensus.cc:2393] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:24.890969 12746 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:24.892287 16693 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:19:24.892830 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:24.893155 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 115147us
I20260709 13:19:24.892828 16693 raft_consensus.cc:697] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:24.895272 12672 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:24.895381 16693 consensus_queue.cc:237] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:24.896925 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:24.896273 16751 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:24.899547 16700 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:24.910269 16887 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:24.910852 16887 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:24.913852 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:24.914490 12747 connection.cc:256] RPC call timeout handler was delayed by -2.3892951999998786s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1759s for a response
W20260709 13:19:24.915088 12747 connection.cc:256] RPC call timeout handler was delayed by -2.3415952000000289s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18120000000000003s for a response
W20260709 13:19:24.915484 12747 connection.cc:256] RPC call timeout handler was delayed by -2.2893951999999445s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18700000000000003s for a response
W20260709 13:19:24.915867 12747 connection.cc:256] RPC call timeout handler was delayed by -2.0697952000000441s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2114s for a response
W20260709 13:19:24.916136 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 57786.7us
I20260709 13:19:24.913029 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:19:24.925287 12516 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 4c13fa2aff324f65ad0bf6191ea68fab with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 3d539f8d95d343d8b1696803f18b30d2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:19:24.927045 12747 connection.cc:256] RPC call timeout handler was delayed by -2.0328951999999845s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2155s for a response
W20260709 13:19:24.927560 12747 connection.cc:256] RPC call timeout handler was delayed by -1.9869951999999103s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22060000000000002s for a response
W20260709 13:19:24.928046 12747 connection.cc:256] RPC call timeout handler was delayed by -1.950995199999852s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22460000000000002s for a response
W20260709 13:19:24.928470 12747 connection.cc:256] RPC call timeout handler was delayed by -1.774780397999848s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2889s for a response
I20260709 13:19:24.930133 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:24.930958 12709 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:19:24.936169 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:24.937366 12778 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:24.930289 16700 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:24.937361 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:24.957777 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 60983.1us
I20260709 13:19:24.959038 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:24.960145 13382 raft_consensus.cc:2410] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:19:24.969885 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:24.979681 12634 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:19:24.968439 16913 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:24.985545 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:24.986220 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 58211.6us
I20260709 13:19:24.981662 16751 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:24.991611 12775 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:19:24.990170 12597 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:24.983099 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:25.018996 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:25.009027 16751 raft_consensus.cc:697] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:25.022645 12630 raft_consensus.cc:2368] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12.
I20260709 13:19:25.026340 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:25.027122 12597 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:25.027283 12783 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:19:25.028052 16776 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:25.028510 16776 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:25.028918 16776 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:25.029641 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:25.030666 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:25.031260 16886 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.031371 12777 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:25.032039 16886 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:25.034000 16941 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:25.034467 16886 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:25.034608 16941 raft_consensus.cc:697] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:25.034832 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:19:25.035238 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 55586.5us
I20260709 13:19:25.036378 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:25.035962 16941 consensus_queue.cc:237] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:25.037180 12775 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:25.037752 12598 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:25.037999 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:25.039235 16955 log.cc:927] Time spent T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:19:25.040710 16899 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:25.042376 16848 consensus_queue.cc:1059] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:25.044160 16848 raft_consensus.cc:3055] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:25.044595 16848 raft_consensus.cc:740] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:25.032871 16751 consensus_queue.cc:237] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:25.050029 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:25.051853 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:25.052758 12629 raft_consensus.cc:2393] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 12: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:25.061069 12778 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:19:25.063304 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:24.989507 16845 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:25.069200 12634 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:19:25.070380 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:25.070173 12673 leader_election.cc:400] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12.
I20260709 13:19:25.071223 12673 leader_election.cc:403] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:19:25.071419 16901 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:25.071869 16901 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:25.071962 12781 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:25.072288 16901 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:25.070837 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 43868.8us
I20260709 13:19:25.076460 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:25.074198 16913 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:25.096858 12597 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:25.098579 12597 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:25.100720 16921 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:25.101672 16921 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:25.102043 16921 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:25.102479 16921 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:25.104192 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:25.104521 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:25.104880 12779 raft_consensus.cc:2376] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12. Re-sending same reply.
I20260709 13:19:25.107014 16845 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:25.070135 16896 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.111114 13384 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 12: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
W20260709 13:19:25.113984 12633 consensus_meta.cc:287] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.561s	user 0.011s	sys 0.011s
I20260709 13:19:25.125770 12782 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:19:25.128159 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:25.124682 12633 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:25.129905 12777 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
W20260709 13:19:25.130512 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 44644.9us
I20260709 13:19:25.076928 16848 consensus_queue.cc:260] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:25.142463 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:25.130299 16896 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:25.161248 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:25.168445 16648 raft_consensus.cc:2764] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:19:25.172680 16899 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:19:25.173765 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.162127 16848 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:25.181212 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:25.181946 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:25.182443 12781 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
W20260709 13:19:25.184588 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 86083us
I20260709 13:19:25.185612 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:25.186597 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:25.187868 12778 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:25.188755 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:25.194725 16896 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:25.205751 16960 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:19:25.219178 12747 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:25.220093 16933 raft_consensus.cc:2764] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:19:25.230338 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:25.231092 12780 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:25.230917 16887 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:25.241287 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:25.249042 12630 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:25.249533 12775 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:25.248596 16921 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:25.233373 16887 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:25.254561 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:25.254959 13382 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:19:25.255360 13385 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 14
W20260709 13:19:25.276158 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 55626.2us
I20260709 13:19:25.269886 16913 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12.
I20260709 13:19:25.295215 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:25.297585 12597 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:25.297955 12777 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:25.305544 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:25.303656 12673 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:25.321821 16966 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:25.322273 12775 raft_consensus.cc:1275] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:19:25.322470 16899 consensus_queue.cc:1059] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:25.328920 16891 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:25.307353 16886 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.330034 16886 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:25.331450 16901 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:25.301280 16839 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:19:25.332366 16886 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:25.333187 16921 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:25.333787 12633 raft_consensus.cc:1240] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:25.339457 12777 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:25.333838 13349 leader_election.cc:400] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 13. Current term is 14.
I20260709 13:19:25.341701 12780 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:19:25.341760 13349 leader_election.cc:403] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:25.329819 16891 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:25.342883 12778 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:19:25.345031 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:25.345775 12778 raft_consensus.cc:2368] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12.
W20260709 13:19:25.333722 13348 leader_election.cc:336] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.586s (SENT)
I20260709 13:19:25.348964 13348 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:25.354075 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:19:25.355351 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 207655us
I20260709 13:19:25.358306 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:25.358647 12775 raft_consensus.cc:1240] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:25.359426 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:25.346267 16966 raft_consensus.cc:697] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:25.362632 16966 consensus_queue.cc:237] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:25.363349 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:25.363811 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.363901 12709 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:25.368361 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:25.370400 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 90460.9us
I20260709 13:19:25.333541 16901 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:25.361951 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:25.382865 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:25.384009 16969 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.384385 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:25.384672 16969 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:25.385488 16896 consensus_queue.cc:1059] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:25.385881 16899 consensus_queue.cc:1059] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:25.386898 16969 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:25.387266 16710 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:25.389536 12629 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:25.359002 16886 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:25.339761 16776 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:25.432624 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:25.435995 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 50581.8us
I20260709 13:19:25.440665 13385 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
W20260709 13:19:25.441670 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 121181us
I20260709 13:19:25.444314 12779 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:25.381237 16941 consensus_queue.cc:1059] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:19:25.445881 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:25.451068 12778 raft_consensus.cc:3055] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:25.451494 12778 raft_consensus.cc:740] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:19:25.452210 16964 log.cc:927] Time spent T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.250s	user 0.002s	sys 0.001s
I20260709 13:19:25.452409 12778 consensus_queue.cc:260] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:25.453688 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:25.457824 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 151763us
I20260709 13:19:25.461086 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:25.461899 13382 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:25.461892 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:25.463500 13376 raft_consensus.cc:2410] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 12 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:19:25.463414 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:25.462603 13384 raft_consensus.cc:1240] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:25.468560 12708 raft_consensus.cc:2393] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 8: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:25.453415 12778 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:25.472724 13383 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:19:25.473838 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:19:25.470510 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.442703 16781 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 13. Current term is 14.
I20260709 13:19:25.438304 16913 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:25.484961 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:25.491361 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 183232us
I20260709 13:19:25.494127 12704 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:25.496557 12701 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:25.492218 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:25.498596 12627 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:25.471385 16886 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:25.506546 12708 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:25.514562 16891 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:25.515635 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:25.486167 16693 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:19:25.517458 12632 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:19:25.519418 16886 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:25.526460 12672 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:25.529385 16886 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:19:25.530089 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 130129us
I20260709 13:19:25.531036 12598 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:25.504791 16634 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:25.537586 12747 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:25.539546 12747 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:25.499172 16913 raft_consensus.cc:697] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:25.529495 12672 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:25.541927 12597 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:25.558344 12709 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
W20260709 13:19:25.558378 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 88375.3us
I20260709 13:19:25.568609 12597 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:25.555469 16899 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:25.556740 16634 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:25.558744 16693 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:25.569272 16896 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:25.581385 12778 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:19:25.547260 16977 consensus_queue.cc:1059] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:19:25.586561 16891 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:25.588201 16839 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:25.587857 16913 consensus_queue.cc:237] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:25.580242 16776 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:25.599020 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 69652.3us
I20260709 13:19:25.601727 16976 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:25.574672 16899 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:25.623127 12704 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:25.624171 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:25.624032 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:25.625869 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:25.626549 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:25.627580 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:25.628181 13383 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:25.629149 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:25.627754 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:25.623821 16839 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:25.630204 13349 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:25.635993 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 76754.9us
I20260709 13:19:25.627573 16896 raft_consensus.cc:697] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:19:25.642571 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 103112us
I20260709 13:19:25.625689 16899 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:25.624858 16693 raft_consensus.cc:697] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:25.644835 12783 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:19:25.648921 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.651593 16980 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:25.652338 12627 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:25.635443 16976 raft_consensus.cc:697] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:25.651400 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:25.637776 16891 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.657666 16896 consensus_queue.cc:237] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:25.656591 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:25.666787 12672 leader_election.cc:400] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12.
I20260709 13:19:25.671322 12627 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:25.671726 12598 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:25.673890 16634 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:25.683015 16954 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.560s	user 0.000s	sys 0.036s
I20260709 13:19:25.676663 16886 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:19:25.687916 13383 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:25.663097 16891 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:25.697854 16839 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:25.698751 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:25.699728 12627 raft_consensus.cc:2393] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 11: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:19:25.702073 13348 leader_election.cc:336] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.905s (SENT)
I20260709 13:19:25.703507 13348 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:25.704933 12672 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:25.695459 16776 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:25.693343 16693 consensus_queue.cc:237] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:25.711275 12672 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:25.716292 16941 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:25.716753 16941 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:25.717080 16941 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:25.717234 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 60353.4us
I20260709 13:19:25.719116 12598 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:25.719682 16891 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:25.719986 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 71990.5us
I20260709 13:19:25.720877 12747 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:25.721179 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:25.693297 16976 consensus_queue.cc:237] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:25.725917 16710 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:25.725929 16912 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.727814 16912 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:25.730480 16912 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:25.730103 16941 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:25.731273 16921 raft_consensus.cc:3055] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:25.731856 16921 raft_consensus.cc:740] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:25.734524 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:25.735219 12627 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 10
W20260709 13:19:25.735569 16958 log.cc:927] Time spent T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.572s	user 0.011s	sys 0.000s
I20260709 13:19:25.739444 16969 raft_consensus.cc:3055] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:25.740442 16969 raft_consensus.cc:740] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:25.745431 12672 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:25.741480 16878 raft_consensus.cc:3055] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:25.749635 16878 raft_consensus.cc:740] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:25.755708 16941 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:25.756313 16878 consensus_queue.cc:260] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:25.747305 16776 raft_consensus.cc:697] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:25.757658 16878 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:25.758535 16776 consensus_queue.cc:237] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:25.762001 16973 log.cc:927] Time spent T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.378s	user 0.002s	sys 0.002s
I20260709 13:19:25.763721 16886 consensus_queue.cc:1059] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:25.765653 16891 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:19:25.766610 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 47596.8us
I20260709 13:19:25.768229 12598 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:25.769294 16982 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:25.769801 16982 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:25.770272 16982 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:25.752279 16969 consensus_queue.cc:260] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:25.771903 16969 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:25.754478 16896 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:25.775487 16751 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:25.776619 12709 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:25.776757 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:25.778206 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:25.780467 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:25.781461 16664 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:25.781996 16664 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:25.754385 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:25.765470 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 44717.8us
I20260709 13:19:25.733197 16921 consensus_queue.cc:260] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:25.777002 16751 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.755319 16713 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:19:25.754912 16967 log.cc:927] Time spent T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.392s	user 0.003s	sys 0.000s
I20260709 13:19:25.739444 12776 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
W20260709 13:19:25.795248 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 89392.3us
I20260709 13:19:25.781262 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:25.727806 16980 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:25.813297 12783 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:25.813879 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:25.814582 12781 raft_consensus.cc:3055] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:25.815069 12781 raft_consensus.cc:740] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:25.815845 12781 consensus_queue.cc:260] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:25.808154 16887 raft_consensus.cc:3055] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:25.816776 12781 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:25.816980 16887 raft_consensus.cc:740] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:25.817251 12597 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:25.807478 12705 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
I20260709 13:19:25.814670 16896 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:25.820957 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:25.821851 12704 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:19:25.810101 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:25.829522 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:25.829946 12703 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:19:25.835760 13384 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:25.832216 12597 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:25.838416 12597 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:25.824554 16648 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:25.845073 12627 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:25.805612 16941 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:25.849936 12777 raft_consensus.cc:1240] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8. Ops: []
W20260709 13:19:25.856805 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 73865.4us
I20260709 13:19:25.869274 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:25.819276 16839 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:19:25.873803 13382 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:19:25.836858 16913 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:25.809634 16751 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:25.870404 16976 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:25.873883 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:25.893932 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.849527 16977 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:25.898231 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:25.900687 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:25.901384 13376 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:25.911276 16648 raft_consensus.cc:697] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:25.912647 16648 consensus_queue.cc:237] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:25.899508 16921 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:25.927668 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:25.868111 16980 consensus_queue.cc:1059] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:19:25.930140 13383 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 10
W20260709 13:19:25.928717 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 90389.3us
I20260709 13:19:25.868834 16710 consensus_queue.cc:1059] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:19:25.868551 16899 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:25.874420 16941 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:25.942198 16984 log.cc:927] Time spent T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:19:25.944316 13383 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:25.948513 16977 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:25.950790 12597 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:25.955968 16977 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:25.956521 16977 raft_consensus.cc:697] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:25.957393 16977 consensus_queue.cc:237] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:26.005939 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.027935 12779 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:25.931737 16781 consensus_queue.cc:1059] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 13:19:26.028991 13377 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
W20260709 13:19:26.033879 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 173568us
I20260709 13:19:25.929512 16913 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:26.006042 16664 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:26.057718 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 124774us
W20260709 13:19:26.059623 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 204041us
I20260709 13:19:26.046756 16901 raft_consensus.cc:3055] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:19:26.087077 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.083205 16751 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:26.089610 16990 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.151s
I20260709 13:19:26.031545 16966 consensus_queue.cc:1059] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:19:26.091810 16941 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:26.098456 16899 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:26.103982 16913 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:26.107694 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 44589.7us
W20260709 13:19:26.107743 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 287420us
I20260709 13:19:26.074869 16901 raft_consensus.cc:740] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:26.083011 16933 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:26.118237 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.127969 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:26.128947 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:26.131664 12672 leader_election.cc:400] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12.
I20260709 13:19:26.138530 12629 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:26.142886 12627 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:19:26.144932 12672 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:26.097239 16982 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:26.150317 16995 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:19:26.172782 12598 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:26.151715 16664 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:26.182291 16899 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:26.152619 16848 raft_consensus.cc:3055] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:26.153358 16839 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:26.182901 16848 raft_consensus.cc:740] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:26.184141 16848 consensus_queue.cc:260] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:26.184954 16982 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:26.154156 16913 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:26.158474 16941 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:26.150496 17001 consensus_queue.cc:1059] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 13:19:26.159551 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:26.159829 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:26.162523 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:26.163178 16912 raft_consensus.cc:3055] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:26.163847 16990 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:26.164850 12672 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:26.166069 16878 raft_consensus.cc:3055] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:26.193338 12707 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:19:26.144405 16845 consensus_queue.cc:1059] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:19:26.181293 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:26.155416 13349 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:26.186924 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:26.158727 16989 log.cc:927] Time spent T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:19:26.193300 12703 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
W20260709 13:19:26.249014 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.130527 16933 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:26.194883 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 63687.5us
I20260709 13:19:26.195704 12672 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:26.262957 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 67338.5us
I20260709 13:19:26.193464 12781 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:19:26.270285 12781 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:26.160923 16887 consensus_queue.cc:260] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:26.277884 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:26.272020 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 221467us
I20260709 13:19:26.182919 16899 raft_consensus.cc:697] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:26.273977 16664 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:26.194929 16912 raft_consensus.cc:740] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:26.297528 13380 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:26.295998 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:26.290062 12779 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:26.254719 16848 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:26.199590 16878 raft_consensus.cc:740] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:26.313076 12780 raft_consensus.cc:1275] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:19:26.313479 12780 pending_rounds.cc:85] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 4
I20260709 13:19:26.313643 12782 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:19:26.307996 12672 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:26.315936 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:26.316746 12775 raft_consensus.cc:2393] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 12: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:26.256598 17002 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.312404 16664 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.312660 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:26.319978 12781 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:26.313719 12780 pending_rounds.cc:107] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:19:26.253364 16913 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:26.187299 16996 consensus_queue.cc:1059] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:19:26.320453 12780 raft_consensus.cc:2889] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:26.320927 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:26.322266 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:26.324380 12672 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:26.331490 12598 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:26.302371 16710 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:26.187426 16941 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:26.302496 16999 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.313766 16878 consensus_queue.cc:260] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:26.345670 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:26.352202 16996 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:26.323491 16751 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:26.324034 16913 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:26.303047 17003 consensus_queue.cc:1059] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.194s
I20260709 13:19:26.370350 16982 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:26.375115 16887 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:26.390334 16740 raft_consensus.cc:3055] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:19:26.400056 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 244704us
W20260709 13:19:26.391108 16986 log.cc:927] Time spent T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.454s	user 0.003s	sys 0.000s
I20260709 13:19:26.392807 16901 consensus_queue.cc:260] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:26.394351 12673 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:26.341221 16710 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:26.404739 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 73381.9us
W20260709 13:19:26.405350 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 69200.4us
I20260709 13:19:26.418773 12780 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:26.337608 16664 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:26.368450 16969 consensus_meta.cc:287] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.596s	user 0.006s	sys 0.006s
I20260709 13:19:26.369326 16990 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:26.333098 17002 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:26.398154 16913 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:26.436112 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:26.440527 17009 log.cc:927] Time spent T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:19:26.443454 16664 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:26.443984 12598 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:26.447108 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 52935us
I20260709 13:19:26.404029 16740 raft_consensus.cc:740] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:26.453393 12673 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:26.434697 16710 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:26.454651 16899 consensus_queue.cc:237] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:26.459749 16740 consensus_queue.cc:260] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:26.461215 16740 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:26.461700 12517 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 4c13fa2aff324f65ad0bf6191ea68fab with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 639ab2db265f4747bb7afb93e833f530 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:19:26.462347 12517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 144196us
I20260709 13:19:26.464757 12598 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:26.465931 16648 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:26.466171 12598 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:26.466609 16648 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:26.467365 16648 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:26.467497 12598 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:26.459796 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.455526 16969 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:19:26.485834 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:26.412384 16999 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:26.419624 16966 consensus_queue.cc:1059] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:19:26.485826 12781 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:26.454546 16878 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:26.412875 16912 consensus_queue.cc:260] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:26.489717 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:26.450830 16845 consensus_queue.cc:1059] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:19:26.503373 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:26.487404 16990 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:19:26.507083 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:26.508980 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.514366 12626 raft_consensus.cc:1240] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:26.455170 16982 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:26.520169 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:26.460321 17007 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.101s
I20260709 13:19:26.527086 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:26.532264 13385 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:19:26.535205 16941 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:26.536586 17002 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:26.538059 16968 log.cc:927] Time spent T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 1.126s	user 0.005s	sys 0.001s
I20260709 13:19:26.539286 16994 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:26.539748 16994 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.540122 16994 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:26.541805 16975 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:26.545979 16845 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:26.546962 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.538093 12631 raft_consensus.cc:1275] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:19:26.548166 16982 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:26.548300 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:26.546532 16845 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:26.550390 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:26.550822 16845 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:26.552646 12672 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:26.554109 17022 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:26.554721 17022 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:26.551048 12626 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:26.559770 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:26.560673 12774 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 12: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:26.516203 16933 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:26.589615 12598 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:26.589242 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:26.590795 12783 raft_consensus.cc:2410] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:19:26.524565 16912 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:19:26.592661 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 273517us
I20260709 13:19:26.526299 16990 raft_consensus.cc:697] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:26.598304 12672 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:19:26.595041 13348 leader_election.cc:336] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.929s (SENT)
I20260709 13:19:26.557317 17022 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:26.603590 13348 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:26.581663 16941 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.530707 16751 consensus_queue.cc:1059] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 13:19:26.609126 16891 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:26.578264 16969 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
W20260709 13:19:26.679008 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:26.689010 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.624993 16896 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:26.614300 16664 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:19:26.673924 16891 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.698472 13349 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:26.691226 17007 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:26.689353 17022 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:26.726006 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 45656.2us
W20260709 13:19:26.728487 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 145942us
I20260709 13:19:26.692260 16648 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:19:26.734068 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 62287.4us
I20260709 13:19:26.738935 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:26.673673 16941 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:26.744565 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 149561us
I20260709 13:19:26.720742 16891 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:26.764880 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.719890 16896 raft_consensus.cc:697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:26.769052 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:26.731937 17002 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:19:26.776985 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.792498 13384 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:26.796316 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:26.765956 16839 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:26.802590 13380 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:19:26.776329 17002 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:19:26.834115 16921 consensus_meta.cc:287] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.903s	user 0.011s	sys 0.002s
I20260709 13:19:26.827602 12626 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:19:26.835371 12626 raft_consensus.cc:606] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:26.835741 12626 raft_consensus.cc:614] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:26.837038 16896 consensus_queue.cc:237] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:26.805754 16999 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:26.847844 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 244813us
I20260709 13:19:26.852561 16921 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:19:26.860266 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.829241 17002 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:26.798487 17016 consensus_queue.cc:1048] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.106s
W20260709 13:19:26.855790 16987 log.cc:927] Time spent T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:19:26.880295 12673 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:26.876806 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:26.881848 17025 log.cc:927] Time spent T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:19:26.807300 16710 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:26.806346 16839 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:26.886267 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:26.888114 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:26.888870 12631 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:26.887076 12626 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:26.891657 16664 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.892544 16693 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:26.892257 16664 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:26.894189 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 409147us
W20260709 13:19:26.899653 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.902196 16941 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:26.907279 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 136585us
I20260709 13:19:26.908257 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:26.909734 16998 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:26.911010 16998 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:26.911540 16998 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:26.912031 16998 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:26.916255 16913 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:26.767108 16990 consensus_queue.cc:237] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:26.923620 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:26.929646 16822 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:26.930358 16822 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:26.932703 16901 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:26.933053 16822 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:26.937215 17028 log.cc:927] Time spent T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
W20260709 13:19:26.939954 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 59738.1us
I20260709 13:19:26.945165 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:26.947153 12777 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:26.950165 16994 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:26.951593 12673 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:26.952816 16994 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:26.954324 16878 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:26.956240 16941 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:26.956933 16941 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:26.959631 16941 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:26.960090 16994 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:26.961212 16878 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.961930 16878 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:26.964108 12673 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:26.964962 16878 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:26.966572 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:26.967725 12704 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:19:26.968662 17003 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.969321 17003 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:26.969952 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:26.973701 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:26.974412 12628 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:19:26.976084 17003 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:26.977085 13349 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:26.978925 16995 consensus_queue.cc:1048] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:26.980352 16781 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:26.980913 16781 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.981367 16995 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:26.981941 16995 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.981392 16781 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:26.982581 16995 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:26.977756 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 124800us
W20260709 13:19:26.978332 13348 leader_election.cc:336] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 2.223s (SENT)
I20260709 13:19:26.987960 13348 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:26.856575 16991 consensus_queue.cc:1059] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.069s
W20260709 13:19:26.989874 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 80428.8us
I20260709 13:19:26.992647 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:26.993870 12701 raft_consensus.cc:2393] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 10: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
W20260709 13:19:27.006790 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.943876 17016 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:19:27.054756 17025 log.cc:927] Time spent T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
W20260709 13:19:27.068172 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 76073.2us
I20260709 13:19:27.054095 17003 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:27.073932 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:27.096145 12707 raft_consensus.cc:2393] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 11: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:27.084025 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:27.097106 12701 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:27.089303 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 111020us
I20260709 13:19:27.077869 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:27.091584 12673 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:27.070525 12704 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:27.107970 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:27.108582 12708 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:27.097640 16848 consensus_meta.cc:287] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.779s	user 0.008s	sys 0.004s
I20260709 13:19:27.056381 16994 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:27.151597 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 43651.4us
I20260709 13:19:27.157574 12705 raft_consensus.cc:1240] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:27.157943 12673 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:19:27.094640 16887 consensus_meta.cc:287] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.679s	user 0.011s	sys 0.000s
W20260709 13:19:27.155438 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.152061 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:27.159545 12673 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:27.085901 17022 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.168479 12709 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:19:27.169314 17037 log.cc:927] Time spent T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:19:27.170445 12702 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:19:27.173336 12703 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:27.178022 12627 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:19:27.194466 12747 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:27.179848 12706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20db0f202ea546699789c2325d241b35"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:19:27.166853 12707 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:27.212659 12706 raft_consensus.cc:606] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:27.213167 12706 raft_consensus.cc:614] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:27.186607 16710 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:27.188246 12700 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:27.193524 17007 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:27.193990 12705 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:19:27.194628 17010 consensus_queue.cc:1059] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:19:27.196017 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.200023 16664 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:27.200627 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.193037 17002 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:27.209203 16839 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:27.243129 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.151320 16999 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:27.221077 12701 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:19:27.225103 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:27.227095 12598 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:27.166265 16822 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:27.206817 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:27.235601 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:27.245680 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:27.237051 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:27.247068 16913 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:27.262038 12709 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:19:27.219431 17016 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:27.270419 12775 raft_consensus.cc:2393] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 10: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:27.274413 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:27.274900 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 47935.8us
I20260709 13:19:27.277509 12598 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:27.285784 17023 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:27.289017 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 94668.9us
I20260709 13:19:27.290827 12598 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:27.215551 17022 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:27.301185 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.216660 16891 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:27.247953 16999 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:27.248904 17018 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:27.294623 12747 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:27.294132 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:27.252358 16822 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:27.307188 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:27.287240 16975 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:27.309259 12629 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:27.318380 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:27.321902 12634 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:19:27.322086 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:27.322651 12633 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:27.317135 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:27.310395 16999 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:27.312539 16822 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:27.312851 16975 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:27.337852 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:27.342224 17022 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:27.325690 17010 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:27.344719 12747 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:27.345659 17013 raft_consensus.cc:3055] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:27.346117 17013 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:19:27.346832 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.346912 17017 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:27.347078 17010 consensus_queue.cc:1059] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:19:27.349686 17010 raft_consensus.cc:3055] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:27.350205 17010 raft_consensus.cc:740] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:27.351933 12634 raft_consensus.cc:2393] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 11: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:27.346921 12747 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:27.353960 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:27.355010 12625 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 15: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:19:27.356734 17043 log.cc:927] Time spent T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:19:27.315425 16998 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:27.354298 17040 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 13:19:27.364835 12708 raft_consensus.cc:2410] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260709 13:19:27.340703 16912 consensus_meta.cc:287] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.743s	user 0.008s	sys 0.003s
I20260709 13:19:27.370394 12779 raft_consensus.cc:1240] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:27.340461 16975 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:27.375748 17013 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:27.377775 16994 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:27.341156 16995 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:27.380096 16995 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:27.329512 16648 consensus_queue.cc:1059] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:19:27.387204 16891 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:27.388540 17002 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:27.389241 17010 consensus_queue.cc:260] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:27.390219 12747 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:27.390578 17010 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:27.392532 12747 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:27.394851 16998 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:27.372925 17014 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:27.398078 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:27.398716 12634 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:27.405354 16781 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:27.407609 16781 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:27.410796 12673 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:27.411720 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.412295 16891 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:27.412379 12673 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:27.412820 16891 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:27.413194 16891 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:27.414667 16913 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:27.414517 17035 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:27.415181 16913 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:27.415560 16648 raft_consensus.cc:3055] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:27.415680 16913 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:27.415995 16648 raft_consensus.cc:740] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:27.416806 16648 consensus_queue.cc:260] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:27.377300 17013 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:27.422035 16648 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:27.432149 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.433683 17023 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:27.436002 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:27.436681 12629 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:27.442418 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:27.447453 17021 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:27.447647 16998 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:19:27.448801 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 56326.5us
I20260709 13:19:27.450151 16998 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:19:27.453051 17028 log.cc:927] Time spent T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.052s	user 0.003s	sys 0.000s
I20260709 13:19:27.454378 16776 raft_consensus.cc:3055] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:27.454978 16776 raft_consensus.cc:740] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:27.456262 16776 consensus_queue.cc:260] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:27.458300 12673 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:27.459101 12628 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:27.459756 16941 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:27.460759 16941 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:27.461251 16941 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:27.463096 16982 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:27.467015 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:27.467373 16997 raft_consensus.cc:3055] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:27.467881 16997 raft_consensus.cc:740] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:27.468885 16997 consensus_queue.cc:260] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:27.470235 12633 raft_consensus.cc:1240] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:27.470337 16997 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:27.474830 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:27.475572 12627 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:27.477741 16901 consensus_meta.cc:287] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.544s	user 0.005s	sys 0.007s
I20260709 13:19:27.478006 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:27.478907 13380 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:27.479192 12704 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:27.481048 12630 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:27.484293 12709 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:27.484835 17021 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.485354 17021 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:27.487673 17021 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:27.488936 16994 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.489672 13349 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:27.490374 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:27.491258 12709 raft_consensus.cc:2410] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:19:27.491873 16975 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:27.493027 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:27.493680 12625 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:27.493902 16776 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:27.494335 12706 raft_consensus.cc:1240] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:27.489691 16994 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:27.497663 12706 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 13. Current term is 14. Ops: []
W20260709 13:19:27.497864 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.498389 12629 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:27.498843 16998 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.499414 12782 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:19:27.499544 16998 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:27.502835 16998 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:27.493690 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:27.504242 16891 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:27.506064 12625 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
W20260709 13:19:27.507509 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 47136.5us
I20260709 13:19:27.507818 12783 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:27.508088 16999 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:27.514057 17022 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:27.514675 17022 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:27.515246 17022 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:27.508500 12673 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:27.510556 12783 raft_consensus.cc:1275] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:27.517256 12782 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:19:27.519305 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.522574 12773 tablet_service.cc:1558] Processing DeleteTablet for tablet 6609d5f93ab048558ea46e511ef3b2ec with delete_type TABLET_DATA_TOMBSTONED (TS 3d539f8d95d343d8b1696803f18b30d2 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:50664
W20260709 13:19:27.523157 12528 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54792 (request call id 227) took 15266 ms (client timeout 15000 ms). Trace:
I20260709 13:19:27.513455 16822 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:27.512228 13380 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:27.525221 16886 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:27.527127 16886 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:27.527822 16886 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:27.525817 12527 catalog_manager.cc:5697] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 1 to 7, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 7 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:27.532019 12673 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:27.535472 12633 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:19:27.537171 12673 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:27.507244 16913 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:27.555964 16839 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:27.556680 16839 raft_consensus.cc:697] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:27.536264 12633 raft_consensus.cc:606] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:27.560026 12625 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:27.560997 12708 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:27.563660 12631 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:27.563603 12627 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:19:27.592502 13349 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:19:27.601387 13348 leader_election.cc:336] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.544s (SENT)
I20260709 13:19:27.601999 13348 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:27.551823 16995 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:27.534904 16710 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.606869 13349 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:27.615131 12625 raft_consensus.cc:1240] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:27.614794 17021 consensus_queue.cc:1059] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:27.619726 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:27.621672 12704 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:19:27.619958 12527 catalog_manager.cc:5697] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:27.624747 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:27.616685 16781 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.621222 16848 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:19:27.642834 12747 connection.cc:256] RPC call timeout handler was delayed by -1.3461506590001591s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
W20260709 13:19:27.643062 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:27.643453 12747 connection.cc:256] RPC call timeout handler was delayed by -1.2777506590000485s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1622s for a response
W20260709 13:19:27.643970 12747 connection.cc:256] RPC call timeout handler was delayed by -1.2336506590002045s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16710000000000003s for a response
W20260709 13:19:27.644395 12747 connection.cc:256] RPC call timeout handler was delayed by -1.1571506590000809s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1756s for a response
W20260709 13:19:27.644838 12747 connection.cc:256] RPC call timeout handler was delayed by -1.0410506590001205s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1885s for a response
I20260709 13:19:27.605178 16966 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:19:27.649358 12628 raft_consensus.cc:1240] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:27.638034 12527 catalog_manager.cc:5697] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 7 to 8, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 8 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:27.629078 16912 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:19:27.658983 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 424848us
I20260709 13:19:27.653630 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:27.659567 12527 catalog_manager.cc:5697] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:27.629316 16740 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:27.634800 16995 raft_consensus.cc:697] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:19:27.661729 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.639323 17003 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.625449 16710 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:27.674212 12625 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:19:27.630818 17042 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:27.672794 12527 catalog_manager.cc:5697] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 4 to 5, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 5 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:19:27.679832 12517 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 4c13fa2aff324f65ad0bf6191ea68fab with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 639ab2db265f4747bb7afb93e833f530 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:19:27.730118 12517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 50317.3us
I20260709 13:19:27.681622 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:27.695230 16999 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:27.700230 12706 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:19:27.670010 17017 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:27.653751 16848 raft_consensus.cc:697] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:27.685662 17058 tablet_replica.cc:333] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:19:27.686208 16845 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:27.684267 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:27.690538 16822 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:27.699909 17003 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:27.704887 16994 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:27.698484 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:27.707254 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 73342.9us
I20260709 13:19:27.716084 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:27.699533 16913 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:27.523620 12528 rpcz_store.cc:269] 0709 13:19:12.256824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.266756 (+  9932us) service_pool.cc:224] Handling call
0709 13:19:12.430508 (+163752us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b540011eed4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31c090 00007f4c6d3baf3a 00007f4c731711cc 00007f4c7326f37e 00007f4c717302a4 00007f4c71730232 00007f4c71730161 00007f4c717300dc 00007f4c7172f392 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:12.705157 (+274649us) spinlock_profiling.cc:243] Waited 1.07 ms on lock 0x7b5400002210. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28e640 00007f4c73165b76 00007f4c7326fc3e 00007f4c717302a4 00007f4c71730232 00007f4c71730161 00007f4c717300dc 00007f4c7172f392 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:27.522911 (+14817754us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:24.122766 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:24.143743 (+ 20977us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:24.145696 (+  1953us) write_op.cc:435] Acquired schema lock
0709 13:19:24.148242 (+  2546us) tablet.cc:662] Decoding operations
0709 13:19:24.486154 (+337912us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:24.486230 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:24.486255 (+    25us) tablet.cc:685] Acquiring locks for 32 operations
0709 13:19:24.488393 (+  2138us) tablet.cc:701] Row locks acquired
0709 13:19:24.488410 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:19:24.488563 (+   153us) write_op.cc:270] Start()
0709 13:19:24.488679 (+   116us) write_op.cc:276] Timestamp: P: 1783603164488509 usec, L: 0
0709 13:19:24.488699 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:19:24.489376 (+   677us) log.cc:844] Serialized 12524 byte log entry
0709 13:19:24.891162 (+401786us) op_driver.cc:464] REPLICATION: finished
0709 13:19:25.084611 (+193449us) spinlock_profiling.cc:243] Waited 5.63 ms on lock 0x7b540011eed4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3af32a 00007f4c6d334f14 00007f4c6d334ec3 00007f4c6d334e38 00007f4c6d334df3 00007f4c6d3340dc 00007f4c6d35de16 00007f4c6d345cbb 00007f4c6d33039d 00007f4c6d333d75
0709 13:19:25.149927 (+ 65316us) write_op.cc:301] APPLY: starting
0709 13:19:25.160782 (+ 10855us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:25.168342 (+  7560us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:25.168394 (+    52us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:25.194824 (+ 26430us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:25.195727 (+   903us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=64
0709 13:19:25.195827 (+   100us) write_op.cc:312] APPLY: finished
0709 13:19:25.196715 (+   888us) log.cc:844] Serialized 212 byte log entry
0709 13:19:25.198192 (+  1477us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:25.198810 (+   618us) write_op.cc:454] Released schema lock
0709 13:19:25.198908 (+    98us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:25.199628 (+   720us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:25.199660 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4035907,"reported_tablets":51,"spinlock_wait_cycles":39211136,"child_traces":[["op",{"apply.queue_time_us":238649,"apply.run_cpu_time_us":11670,"apply.run_wall_time_us":55237,"num_ops":32,"prepare.queue_time_us":106026,"prepare.run_cpu_time_us":13451,"prepare.run_wall_time_us":441959,"raft.queue_time_us":54976,"raft.run_cpu_time_us":32133,"raft.run_wall_time_us":239262,"replication_time_us":402199,"spinlock_wait_cycles":12942592,"thread_start_us":276374,"threads_started":4,"wal-append.queue_time_us":75250,"wal-append.run_cpu_time_us":10717,"wal-append.run_wall_time_us":1633229}]]}
W20260709 13:19:27.708715 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.698077 12628 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 14. Ops: []
I20260709 13:19:27.642458 16781 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:27.732229 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 118271us
W20260709 13:19:27.735423 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 56419us
I20260709 13:19:27.741149 12527 catalog_manager.cc:5697] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:27.742657 17058 raft_consensus.cc:2243] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:27.744143 16886 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:27.761067 12672 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:27.751031 12629 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:27.762058 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.682672 16740 raft_consensus.cc:697] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:27.731916 16648 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.766559 16648 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:27.767179 16886 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:27.766960 12527 catalog_manager.cc:5697] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 5 to 10, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 10 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:27.768208 16966 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:27.768580 17058 raft_consensus.cc:2272] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:27.626390 16839 consensus_queue.cc:237] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:27.861609 17058 ts_tablet_manager.cc:1916] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:27.788349 17035 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:27.782299 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.864538 16710 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:27.786175 17017 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:27.788352 16921 consensus_queue.cc:1048] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:19:27.839011 16886 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:27.768815 12527 catalog_manager.cc:5697] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:27.855858 17054 log.cc:927] Time spent T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.168s	user 0.004s	sys 0.000s
I20260709 13:19:27.843075 16991 raft_consensus.cc:993] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e: : Instructing follower 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:19:27.737303 17021 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:19:27.850167 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:27.867777 16740 consensus_queue.cc:237] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:27.753494 16975 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:27.914055 16975 raft_consensus.cc:1081] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Signalling peer 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:19:27.749553 16941 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:27.760296 16995 consensus_queue.cc:237] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:27.772910 17062 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:27.846419 17010 raft_consensus.cc:3055] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:27.785840 12628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:19:27.925613 12628 raft_consensus.cc:606] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:27.764612 16822 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:27.772285 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 73852.9us
W20260709 13:19:27.852367 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:27.881807 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.838495 16648 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:27.879887 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:27.884474 12629 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:27.841683 13348 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:27.897364 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:27.874492 17003 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:27.932945 12778 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
W20260709 13:19:27.869474 17057 log.cc:927] Time spent T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
W20260709 13:19:27.871291 17052 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:19:27.900121 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:27.895943 17035 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:27.911731 12527 catalog_manager.cc:5697] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 8 to 9, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 9 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:27.913295 17021 raft_consensus.cc:3055] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:27.940903 17021 raft_consensus.cc:740] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:27.942004 12632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2806) took 1446 ms. Trace:
I20260709 13:19:27.942510 12632 rpcz_store.cc:276] 0709 13:19:26.495644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.502366 (+  6722us) service_pool.cc:224] Handling call
0709 13:19:26.524123 (+ 21757us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:26.919644 (+395521us) spinlock_profiling.cc:243] Waited 4.24 ms on lock 0x7b18001d6140. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c2455 00007f4c6d3aa060 00007f4c6d3aabfb 00007f4c6d3b564b 00007f4c6d3b1d8f
0709 13:19:27.225273 (+305629us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:19:27.225301 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:27.267236 (+ 41935us) log.cc:844] Serialized 26 byte log entry
0709 13:19:27.414384 (+147148us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b18001d6140. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c5522 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:19:27.630264 (+215880us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:27.630290 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:27.630336 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:27.870458 (+240122us) spinlock_profiling.cc:243] Waited 49.6 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:27.941741 (+ 71283us) raft_consensus.cc:1685] finished
0709 13:19:27.941773 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:27.941909 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":53325,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":9494,"spinlock_wait_cycles":164963328,"thread_start_us":612683,"threads_started":2,"wal-append.queue_time_us":388000}
I20260709 13:19:27.943819 12527 catalog_manager.cc:5697] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 7 to 8, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 8 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:27.944655 12630 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:27.908669 12625 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
I20260709 13:19:27.945823 13380 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:19:27.922640 12598 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:27.936564 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:27.946902 13385 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:27.938412 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:27.935153 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:27.949738 17003 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:27.950254 17003 raft_consensus.cc:697] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:27.951284 17003 consensus_queue.cc:237] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:27.951103 12527 catalog_manager.cc:5697] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 1 to 9, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 9 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:27.908110 16994 consensus_queue.cc:1059] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
W20260709 13:19:27.952103 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 110523us
I20260709 13:19:27.927380 16991 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:19:27.953862 16710 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:27.954260 13348 leader_election.cc:336] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.835s (SENT)
I20260709 13:19:27.954386 16710 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:27.954718 13348 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:27.955046 16891 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:27.955116 12673 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:27.955710 17002 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:27.956476 16710 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:27.957000 16941 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:27.957839 17050 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:27.957953 12631 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:19:27.958472 17050 raft_consensus.cc:697] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:27.925153 17010 raft_consensus.cc:740] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:27.959404 17050 consensus_queue.cc:237] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:27.962778 12598 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:27.966114 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:27.968389 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:27.977931 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:27.979370 13385 raft_consensus.cc:2410] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:19:27.955749 16891 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.981541 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:27.981623 16891 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:27.976709 16921 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:19:27.984026 16966 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:27.984405 16891 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:27.977468 12598 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:27.985987 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:27.986538 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:27.984340 12774 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:19:27.985852 13348 leader_election.cc:336] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.663s (SENT)
I20260709 13:19:27.988283 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:27.989514 12629 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 12: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:27.994321 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:27.977995 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:27.996505 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:27.998185 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:28.001865 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:28.003508 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:28.003508 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:28.005771 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:28.005301 12527 catalog_manager.cc:5697] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 7 to 8, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 8 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:28.009754 12777 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:19:27.933708 16982 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:19:27.973307 16991 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:27.949199 17022 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:28.006292 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
W20260709 13:19:28.012784 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.013170 12774 raft_consensus.cc:2393] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 12: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:28.027814 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:28.010411 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:28.029914 12778 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:28.030572 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:28.031324 12706 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:28.033090 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:28.033711 12703 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:19:27.985872 17062 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:28.036072 12627 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:27.977272 17060 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:28.040037 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:27.939018 16822 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:28.042164 12780 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 12: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:28.039968 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:28.040158 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:28.045437 12704 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:28.047015 12707 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:19:28.047354 12705 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:28.048422 12707 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:19:28.049134 12700 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:19:27.960067 17010 consensus_queue.cc:260] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:28.022804 16998 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:28.009425 16710 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:27.940146 16848 consensus_queue.cc:237] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:28.023619 16781 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:28.058346 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 80458.2us
I20260709 13:19:28.027654 16991 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:28.019019 17063 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:28.063009 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:28.044411 16997 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:28.071148 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:28.083026 12700 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:28.050355 17042 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.088065 13385 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 12. (term mismatch)
I20260709 13:19:28.087289 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:28.089227 13377 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:28.088423 13385 pending_rounds.cc:85] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd: Aborting all ops after (but not including) 11
I20260709 13:19:28.093611 13385 pending_rounds.cc:107] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd: Aborting uncommitted NO_OP operation due to leader change: 11.12
I20260709 13:19:28.056278 17062 raft_consensus.cc:697] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:28.093914 13385 raft_consensus.cc:2889] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:28.093108 16966 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:19:28.094696 17042 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:28.079046 17010 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:28.102017 17035 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:28.095824 16776 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 13:19:28.095770 16899 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:19:28.071563 16991 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:28.141827 12706 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:19:28.142530 12700 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:28.130779 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:28.114490 12778 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:28.143539 12774 raft_consensus.cc:2393] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 10: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:28.146544 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:28.147379 12774 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
W20260709 13:19:28.114462 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 108236us
I20260709 13:19:28.119766 12704 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
W20260709 13:19:28.174428 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.025799 17021 consensus_queue.cc:260] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:28.044963 17060 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:28.193994 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:28.194567 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.197314 13376 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:28.187840 13349 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:28.198261 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:28.199084 12709 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 9. Current term is 10. Ops: []
W20260709 13:19:28.206739 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.096503 16822 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:28.071699 16710 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:28.230546 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:19:28.318760 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 256366us
I20260709 13:19:28.186239 17011 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.197680 16781 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:28.094254 16648 consensus_queue.cc:1059] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.222s
I20260709 13:19:28.094381 16998 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.225s
I20260709 13:19:28.355983 12747 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:28.368245 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.340301 17002 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:28.384907 12747 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:28.353135 16648 consensus_queue.cc:1243] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer 639ab2db265f4747bb7afb93e833f530 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:19:28.414286 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.377705 17002 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.380990 17013 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:19:28.480989 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 96182.2us
I20260709 13:19:28.482219 12747 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:28.335265 17011 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:28.487913 17071 log.cc:927] Time spent T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:19:28.488485 17060 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:28.424086 17002 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:28.500416 13348 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.623s (SENT) [suppressed 15 similar messages]
I20260709 13:19:28.418533 17073 raft_consensus.cc:2764] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 13:19:28.506538 17065 log.cc:927] Time spent T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
W20260709 13:19:28.501639 17066 log.cc:927] Time spent T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
W20260709 13:19:28.517338 12632 consensus_meta.cc:287] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.520s	user 0.004s	sys 0.008s
I20260709 13:19:28.484440 17022 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:28.543336 12632 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:28.555433 12597 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:19:28.576197 17074 log.cc:927] Time spent T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:19:28.516433 17018 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:28.544365 17013 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:28.379047 17062 consensus_queue.cc:237] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:28.546523 16899 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:28.607512 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 107118us
I20260709 13:19:28.584676 16648 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:28.584203 17042 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:28.615298 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 133186us
W20260709 13:19:28.609020 13348 leader_election.cc:336] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 2.033s (SENT)
W20260709 13:19:28.641722 12527 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.012s	sys 0.018s
W20260709 13:19:28.653141 12527 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.012s	sys 0.019s
W20260709 13:19:28.617017 16845 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.006s	sys 0.011s
I20260709 13:19:28.613390 16648 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:28.656612 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 438894us
I20260709 13:19:28.599359 17018 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:28.611512 16899 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:28.622026 13348 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:19:28.663168 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 82304.1us
I20260709 13:19:28.652858 16710 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:28.660096 16648 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:28.669322 12747 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:28.670215 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:28.660804 16886 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:28.658955 16845 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.007s	sys 0.011s
I20260709 13:19:28.696674 16710 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:19:28.706018 17078 log.cc:927] Time spent T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:19:28.679451 17089 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:19:28.706950 17072 log.cc:927] Time spent T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.283s	user 0.001s	sys 0.000s
I20260709 13:19:28.698102 17086 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:28.725320 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:28.695071 16886 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:28.746644 13349 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:28.754153 17013 consensus_queue.cc:1059] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:28.736487 17038 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:19:28.755115 17013 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:28.762234 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:28.763127 12632 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:19:28.718878 17011 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:28.781947 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 172969us
W20260709 13:19:28.795035 13348 leader_election.cc:336] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 3.055s (SENT)
I20260709 13:19:28.783787 17058 ts_tablet_manager.cc:1929] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.4
I20260709 13:19:28.803710 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:28.804632 13382 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:28.737746 17095 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.806192 17017 raft_consensus.cc:3055] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:19:28.818256 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 107933us
I20260709 13:19:28.824393 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:28.829900 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:28.830212 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:28.830817 12774 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:19:28.799734 17022 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:19:28.831591 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.839800 12632 raft_consensus.cc:1275] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:19:28.833046 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:28.806738 17011 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:28.836513 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:28.844555 12779 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:28.841615 12782 raft_consensus.cc:2393] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 8: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:28.841949 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:28.842062 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:28.845757 12781 raft_consensus.cc:2376] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12. Re-sending same reply.
I20260709 13:19:28.846408 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:28.846617 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:28.847244 12781 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 5.
I20260709 13:19:28.847643 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:28.848022 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:28.848654 12781 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:19:28.842712 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:28.850203 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:28.816977 17018 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:28.851354 12778 raft_consensus.cc:2376] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12. Re-sending same reply.
I20260709 13:19:28.852265 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:28.820734 17017 raft_consensus.cc:740] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:28.831588 17058 log.cc:1199] T 6609d5f93ab048558ea46e511ef3b2ec P 3d539f8d95d343d8b1696803f18b30d2: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-2-root/wals/6609d5f93ab048558ea46e511ef3b2ec
I20260709 13:19:28.837381 17022 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.866671 16899 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:28.867305 17058 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50664 (request call id 5) took 1345 ms. Trace:
W20260709 13:19:28.866671 17092 log.cc:927] Time spent T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:19:28.876418 12631 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:19:28.822652 17095 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:28.882346 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:28.867786 16886 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:28.892017 16991 consensus_meta.cc:287] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.692s	user 0.010s	sys 0.000s
I20260709 13:19:28.892519 16991 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:28.893986 12777 raft_consensus.cc:2393] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 8: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:28.859375 16966 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:28.875300 17022 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:28.881212 17010 consensus_meta.cc:287] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.776s	user 0.011s	sys 0.001s
I20260709 13:19:28.925495 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:28.926935 17095 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:28.902839 17093 log.cc:927] Time spent T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
W20260709 13:19:28.903548 17091 log.cc:927] Time spent T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:19:28.904026 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:28.905224 16966 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:28.909752 17084 log.cc:927] Time spent T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:19:28.929803 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:28.867790 17058 rpcz_store.cc:276] 0709 13:19:27.521985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.522232 (+   247us) service_pool.cc:224] Handling call
0709 13:19:27.685020 (+162788us) ts_tablet_manager.cc:1151] Deleting tablet 6609d5f93ab048558ea46e511ef3b2ec
0709 13:19:27.685125 (+   105us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:19:28.756576 (+1071451us) tablet_metadata.cc:665] Metadata flushed
0709 13:19:28.783618 (+ 27042us) tablet_metadata.cc:665] Metadata flushed
0709 13:19:28.867165 (+ 83547us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1043,"tablet-delete.queue_time_us":161551,"thread_start_us":127116,"threads_started":1}
I20260709 13:19:28.930446 16966 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:28.930511 12776 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:19:28.909832 17017 consensus_queue.cc:260] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:28.932426 17017 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:28.922259 12628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:33786 (request call id 29) took 1154 ms. Trace:
I20260709 13:19:28.933717 12628 rpcz_store.cc:276] 0709 13:19:27.767764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.785625 (+ 17861us) service_pool.cc:224] Handling call
0709 13:19:28.921895 (+1136270us) spinlock_profiling.cc:243] Waited 34.5 ms on lock 0x7b54002722a0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d32f520 00007f4c6d3ac652 00007f4c7237a60a 00007f4c6b9007d4 00007f4c6b900762 00007f4c6b900691 00007f4c6b90060c 00007f4c6b8ff8c2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:28.922178 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79265536}
I20260709 13:19:28.934242 16781 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:28.934710 16781 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:28.935885 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:28.936611 12781 raft_consensus.cc:2393] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 8: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:28.940604 16648 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:28.854072 17011 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:28.942776 17003 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:28.922719 12672 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:28.959896 12779 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:19:28.921180 17021 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:28.942792 16969 raft_consensus.cc:3055] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:28.964406 17042 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:28.964672 17010 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.964203 17002 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:28.970233 12625 raft_consensus.cc:1275] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:19:28.970894 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:29.002091 12597 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:29.003268 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:29.004134 12628 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:29.004245 12747 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:29.014611 13348 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:28.987859 16969 raft_consensus.cc:740] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:29.009951 17095 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:29.037375 13348 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:29.011560 17086 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:28.995679 17003 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:29.001695 17010 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:29.079703 12628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:19:29.001902 17042 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:29.048473 17095 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:29.023165 17101 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:29.102222 12774 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:19:29.023785 17079 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:19:29.102772 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 65482.5us
I20260709 13:19:29.012974 17018 consensus_queue.cc:1059] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
W20260709 13:19:29.104249 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 181629us
W20260709 13:19:29.107324 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 140585us
I20260709 13:19:29.156116 12673 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:29.019729 17013 consensus_queue.cc:1059] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:19:29.123994 13349 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:29.161580 12673 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:29.162972 12673 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:29.124805 12628 raft_consensus.cc:606] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:29.164034 12628 raft_consensus.cc:614] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:29.155126 12672 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:29.038564 17103 consensus_queue.cc:1059] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.101s
I20260709 13:19:29.102249 17050 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:29.166738 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 155987us
W20260709 13:19:29.174952 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:29.177874 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.179117 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:29.122874 16845 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:29.158032 17095 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:29.168944 17111 log.cc:927] Time spent T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.124s	user 0.007s	sys 0.000s
I20260709 13:19:29.124116 17113 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:29.151077 16899 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:29.219823 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 64783.5us
I20260709 13:19:29.238418 12672 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:29.169596 17060 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:29.151916 16740 consensus_queue.cc:1048] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 13:19:29.233639 16878 raft_consensus.cc:3055] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:19:29.271215 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 108112us
W20260709 13:19:29.272068 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 110829us
I20260709 13:19:29.236086 17113 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:29.240538 17098 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.222774 12516 catalog_manager.cc:5028] TS 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): tablet 6609d5f93ab048558ea46e511ef3b2ec (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:19:29.123301 17086 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:29.234295 16991 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:29.248704 17018 raft_consensus.cc:3055] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:29.170049 16969 consensus_queue.cc:260] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:29.187654 17050 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:29.313424 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:29.265584 17013 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:29.250103 16899 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:29.327250 12631 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:29.285106 17113 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:29.292608 16845 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:29.318686 17018 raft_consensus.cc:740] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:29.265435 17060 raft_consensus.cc:697] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:29.282672 16781 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:29.314064 12516 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 119717us
I20260709 13:19:29.341339 16969 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:29.381464 17118 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.129s
W20260709 13:19:29.384189 17115 log.cc:927] Time spent T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:19:29.386154 17010 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:29.386409 17112 log.cc:927] Time spent T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:19:29.388144 17010 raft_consensus.cc:3055] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:29.388626 17010 raft_consensus.cc:740] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:29.391192 16899 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:29.341512 17013 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:29.395824 17003 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:29.347828 17011 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:29.344981 17101 raft_consensus.cc:697] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:29.428613 12631 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:19:29.430478 13349 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:29.311231 17098 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:29.275197 16878 raft_consensus.cc:740] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:19:29.426482 17022 consensus_meta.cc:287] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.510s	user 0.012s	sys 0.000s
I20260709 13:19:29.453294 17120 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:29.453621 16781 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:29.454325 17127 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:29.455063 17127 raft_consensus.cc:697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:29.420869 17013 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:29.457993 17042 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:29.458498 17117 log.cc:927] Time spent T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 13:19:29.457240 17119 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:19:29.443588 17017 consensus_meta.cc:287] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.511s	user 0.016s	sys 0.000s
W20260709 13:19:29.465479 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 227144us
I20260709 13:19:29.445984 17118 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:29.474002 12707 raft_consensus.cc:1240] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 9. Ops: []
I20260709 13:19:29.470302 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:29.456337 17127 consensus_queue.cc:237] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:29.481483 12708 raft_consensus.cc:1240] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:29.468407 16648 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:29.422572 17010 consensus_queue.cc:260] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:29.498404 12703 raft_consensus.cc:1240] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
W20260709 13:19:29.500128 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 321102us
I20260709 13:19:29.500878 12672 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:29.478358 17042 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:29.481482 17128 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.488045 17017 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.493623 16781 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:29.553516 12703 raft_consensus.cc:1240] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:29.471688 17022 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:29.560709 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 59787.1us
I20260709 13:19:29.534613 17010 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:29.565464 12705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 2122) took 1507 ms. Trace:
W20260709 13:19:29.554450 17121 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
W20260709 13:19:29.586637 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 276926us
I20260709 13:19:29.627499 12703 raft_consensus.cc:1240] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 8. Current term is 9. Ops: [7.6-8.7]
I20260709 13:19:29.477617 17018 consensus_queue.cc:260] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:29.571545 12705 rpcz_store.cc:276] 0709 13:19:28.053243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.053499 (+   256us) service_pool.cc:224] Handling call
0709 13:19:28.070071 (+ 16572us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:28.085099 (+ 15028us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:28.085139 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:28.939603 (+854464us) spinlock_profiling.cc:243] Waited 2.01 ms on lock 0x7b2c00011ed8. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6bcaba11 00007f4c6bcbb372 00007f4c6d3eda01 00007f4c6d3b243a 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2
0709 13:19:28.964305 (+ 24702us) log.cc:844] Serialized 26 byte log entry
0709 13:19:29.116918 (+152613us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:29.166085 (+ 49167us) log.cc:844] Serialized 14 byte log entry
0709 13:19:29.271892 (+105807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:29.271966 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:29.555424 (+283458us) raft_consensus.cc:1685] finished
0709 13:19:29.560221 (+  4797us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:29.560397 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":51312,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":66,"raft.queue_time_us":246289,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":7090,"spinlock_wait_cycles":11281536,"thread_start_us":257705,"threads_started":2,"wal-append.queue_time_us":358801}
I20260709 13:19:29.561002 17017 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:29.536665 17118 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:29.553751 17122 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:29.504053 17119 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:29.659720 12673 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:29.663565 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 162797us
I20260709 13:19:29.560066 16781 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:29.661167 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:29.664965 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:29.503727 16878 consensus_queue.cc:260] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:29.555294 17042 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:29.678860 16781 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:29.679157 17017 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:29.556782 17128 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:29.662225 17126 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:29.482084 17060 consensus_queue.cc:237] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:29.681317 17128 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:29.661350 17137 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:29.663987 17125 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:29.662607 17013 consensus_queue.cc:1059] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
W20260709 13:19:29.716084 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.698414 17050 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:29.707571 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 48007.4us
W20260709 13:19:29.717218 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.692503 17086 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:29.507128 17101 consensus_queue.cc:237] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:29.726616 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:29.726547 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:29.726980 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:29.727820 12633 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 13: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:29.727973 12632 raft_consensus.cc:2368] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7.
I20260709 13:19:29.729171 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:29.729430 12632 raft_consensus.cc:1240] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:19:29.729791 12625 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:19:29.730175 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:29.667709 17119 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:29.731338 17133 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:29.732086 17135 log.cc:927] Time spent T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 13:19:29.732015 17133 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:29.733222 17119 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:29.736066 17123 log.cc:927] Time spent T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
W20260709 13:19:29.738202 17134 log.cc:927] Time spent T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:19:29.740240 17022 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:29.739915 17113 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:29.731387 17136 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:29.742271 17113 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:29.743542 17042 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:29.714499 16648 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.744411 16648 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:29.745710 17095 consensus_meta.cc:287] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.507s	user 0.012s	sys 0.000s
I20260709 13:19:29.746140 17095 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:29.747957 17128 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:29.667904 17118 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:29.748632 17128 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:29.749123 17128 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:29.749847 17141 consensus_queue.cc:1048] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:19:29.753706 17085 consensus_queue.cc:1059] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:19:29.723229 16878 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:29.724257 17126 raft_consensus.cc:697] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:29.755367 17126 consensus_queue.cc:237] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:29.756063 17021 consensus_meta.cc:287] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.762s	user 0.010s	sys 0.000s
I20260709 13:19:29.756645 12775 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:29.758123 17136 raft_consensus.cc:3055] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:29.719636 17097 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.742322 17098 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:29.758548 17136 raft_consensus.cc:740] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:29.758998 12631 raft_consensus.cc:1240] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:19:29.759552 17136 consensus_queue.cc:260] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:29.761759 17077 log.cc:927] Time spent T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 1.199s	user 0.001s	sys 0.005s
I20260709 13:19:29.699182 17018 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:29.764045 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 2123) took 1708 ms. Trace:
W20260709 13:19:29.770859 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.748010 17139 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:29.750130 17118 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:29.765441 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 47604.8us
I20260709 13:19:29.746467 16648 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:29.720595 17125 raft_consensus.cc:697] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:29.758893 17097 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:29.776420 12775 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:29.699529 17017 consensus_queue.cc:1059] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 13:19:29.771027 12702 rpcz_store.cc:276] 0709 13:19:28.056543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.056817 (+   274us) service_pool.cc:224] Handling call
0709 13:19:28.057132 (+   315us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:28.062855 (+  5723us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:28.062887 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:28.081933 (+ 19046us) log.cc:844] Serialized 52 byte log entry
0709 13:19:28.326188 (+244255us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:28.326231 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.326298 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:29.761027 (+1434729us) spinlock_profiling.cc:243] Waited 3.11 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:29.763678 (+  2651us) raft_consensus.cc:1685] finished
0709 13:19:29.763707 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:29.763970 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6089,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":15375,"spinlock_wait_cycles":7158144,"thread_start_us":218791,"threads_started":1,"wal-append.queue_time_us":315917}
I20260709 13:19:29.793445 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:29.794036 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:29.748147 17095 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:29.777819 17139 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:29.775107 17122 raft_consensus.cc:697] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:19:29.799451 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 115943us
I20260709 13:19:29.803743 12774 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:19:29.700631 17137 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:29.804873 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:29.804921 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 140019us
I20260709 13:19:29.807850 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:29.808463 12628 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:19:29.813282 12747 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:29.815436 12631 raft_consensus.cc:1275] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:19:29.817342 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:29.803295 17139 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:29.823630 12747 leader_election.cc:400] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7.
I20260709 13:19:29.826596 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:29.828133 12704 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:19:29.832496 12747 leader_election.cc:403] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:29.833455 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:29.828581 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:29.833292 17122 consensus_queue.cc:237] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:29.834211 17097 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:29.835311 17081 log.cc:927] Time spent T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 1.134s	user 0.002s	sys 0.000s
I20260709 13:19:29.835202 17133 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:29.849973 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:29.850344 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:29.850618 13384 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:29.850427 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:29.820097 17013 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:29.842166 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:29.851904 13380 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:19:29.837040 12704 raft_consensus.cc:2393] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 13: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:29.845118 17125 consensus_queue.cc:237] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:29.853044 12626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2853) took 1886 ms. Trace:
I20260709 13:19:29.853525 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:29.854653 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:29.837383 12700 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:29.844197 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:29.858186 12747 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:29.853418 12626 rpcz_store.cc:276] 0709 13:19:27.966349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.975550 (+  9201us) service_pool.cc:224] Handling call
0709 13:19:27.975971 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:28.041153 (+ 65182us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:19:28.041182 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:28.049605 (+  8423us) log.cc:844] Serialized 26 byte log entry
0709 13:19:28.553863 (+504258us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:28.553933 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.557840 (+  3907us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:29.834707 (+1276867us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:29.852746 (+ 18039us) raft_consensus.cc:1685] finished
0709 13:19:29.852768 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:29.852985 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74092,"raft.queue_time_us":65225,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":6663,"spinlock_wait_cycles":6664576,"thread_start_us":497441,"threads_started":1,"wal-append.queue_time_us":611212}
I20260709 13:19:29.840291 12598 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:29.814677 17086 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:29.862212 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:29.836436 17149 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:29.865005 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:29.847371 17101 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:19:29.869199 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:29.869282 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:29.867769 17013 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:29.864938 17146 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:29.894351 17146 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:29.896968 17146 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:29.898646 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:29.900384 12704 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:29.869658 17095 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:29.885876 17039 consensus_queue.cc:1048] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:19:29.889492 17150 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:29.902099 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:29.868634 17086 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:29.890231 17152 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:29.909737 17095 raft_consensus.cc:697] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:29.912097 17095 consensus_queue.cc:237] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:29.909823 17152 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:29.916983 17152 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:29.918840 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:29.908214 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:29.963485 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 50178.4us
W20260709 13:19:29.964057 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 45400.7us
I20260709 13:19:29.965361 12672 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:29.900415 17013 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:29.980275 17013 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:29.982208 12704 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:19:29.982573 12704 pending_rounds.cc:85] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 10
I20260709 13:19:29.982767 12704 pending_rounds.cc:107] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:19:29.982971 12704 raft_consensus.cc:2889] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:29.983983 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:29.971918 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:29.973021 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:29.974788 17148 log.cc:927] Time spent T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 13:19:29.962566 17140 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:29.979591 17086 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:29.984589 12704 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
I20260709 13:19:29.905952 17073 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:29.995098 17073 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:29.995502 17073 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:29.992951 17140 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:29.994118 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:29.994460 17137 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:29.900452 17128 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:29.985172 17152 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:30.011049 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:29.871775 17149 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.023030 12598 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:29.989108 17118 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:30.029703 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:30.034412 13349 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:30.043562 17069 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
W20260709 13:19:30.047256 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.047601 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:30.048403 12704 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:30.048568 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:30.049284 12708 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:19:30.053840 12707 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:30.025899 17152 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:30.033407 16648 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:30.056774 12700 raft_consensus.cc:2376] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12. Re-sending same reply.
I20260709 13:19:30.051913 17060 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:19:30.064083 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 205955us
I20260709 13:19:30.036072 16921 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:19:30.067912 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:30.057202 17152 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:30.074086 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:30.058954 16648 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:30.064915 17060 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:19:30.082583 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 47265.9us
I20260709 13:19:30.077374 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:30.085867 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:30.086429 12598 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:30.088618 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:30.091292 12625 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:19:30.077345 17086 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:30.092741 17086 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.094189 17097 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:30.094676 17097 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:30.095093 17097 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:30.095710 17086 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:30.077435 17010 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:30.098519 17010 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.070379 17137 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:30.102751 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:30.103144 17137 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.104936 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:30.105893 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:30.107637 17101 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 6. Current term is 7.
I20260709 13:19:30.074448 17042 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.108647 17042 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.110571 12629 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:30.076622 16648 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:30.117693 17060 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:30.079499 17149 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:30.131062 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:30.132241 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:30.132895 12706 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:30.124056 17013 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.134617 12672 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:30.135567 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:30.136373 12630 raft_consensus.cc:2393] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 13: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:30.136484 12672 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:30.136830 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:30.137593 13377 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:30.138876 12629 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:19:30.101375 17010 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:30.102805 17140 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:30.140937 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:30.143440 12672 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:30.150185 12706 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:19:30.137037 13349 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:30.158195 17137 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:30.069789 17139 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.162705 12708 raft_consensus.cc:1275] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:19:30.174947 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:30.178103 12709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1756) took 2602 ms. Trace:
I20260709 13:19:30.141122 17149 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:30.143146 17127 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:30.161060 17073 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.155122 16878 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:30.178602 12709 rpcz_store.cc:276] 0709 13:19:27.562634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.562808 (+   174us) service_pool.cc:224] Handling call
0709 13:19:27.563133 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:28.220916 (+657783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.165059 (+1944143us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b5400172ad4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:30.165214 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1462045568}
I20260709 13:19:30.139844 17125 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:19:30.175567 17070 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:30.183657 17149 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:30.200129 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:30.202805 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:30.206074 17067 log.cc:927] Time spent T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 1.883s	user 0.007s	sys 0.004s
I20260709 13:19:30.206545 17073 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:30.200512 17103 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.935s	user 0.029s	sys 0.008s
I20260709 13:19:30.196846 17070 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:30.211784 17010 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:30.197867 17069 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:30.198371 17149 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:30.210944 17011 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:30.212457 16781 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:30.239382 17108 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:19:30.217056 17103 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.966s	user 0.029s	sys 0.008s
I20260709 13:19:30.240587 17108 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.248340 17171 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:30.250031 17139 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:30.226204 17086 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:30.226622 17073 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:30.220477 17101 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:30.220765 17128 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:30.227025 17010 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.221375 17070 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:30.226905 17150 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:30.254321 17162 log.cc:927] Time spent T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
W20260709 13:19:30.231976 12516 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 981fb2f435e24d06a81fc02cf4591cc5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 522fd2ba0e9f402691b5ef29af8579cd is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:19:30.298569 12516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 77395.8us
I20260709 13:19:30.272692 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:30.299863 12774 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:30.300274 17042 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:30.301219 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:30.302757 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:30.306083 12775 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36478 (request call id 2547) took 2323 ms (client timeout 1639 ms). Trace:
W20260709 13:19:30.306520 12775 rpcz_store.cc:269] 0709 13:19:27.982493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:27.985668 (+  3175us) service_pool.cc:224] Handling call
0709 13:19:30.305874 (+2320206us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b5400089654. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:30.305995 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4070569856}
I20260709 13:19:30.307885 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:30.308490 12775 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:30.309212 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:30.314281 16998 raft_consensus.cc:3055] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:30.314723 16998 raft_consensus.cc:740] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:30.255299 17103 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:30.277170 12779 raft_consensus.cc:1240] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:30.278403 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:30.279989 17075 log.cc:927] Time spent T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 1.725s	user 0.001s	sys 0.003s
I20260709 13:19:30.280458 17010 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:30.280730 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:30.283203 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:30.335134 12782 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:30.283994 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:30.286959 17118 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:30.264976 17073 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:30.290079 17166 log.cc:927] Time spent T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:19:30.266754 17086 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.252084 17120 consensus_queue.cc:1059] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:19:30.350313 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:30.290962 17156 log.cc:927] Time spent T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.387s	user 0.000s	sys 0.003s
I20260709 13:19:30.291229 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:30.266676 17101 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.352849 12780 raft_consensus.cc:2410] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:19:30.294559 17150 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:30.295589 17018 consensus_meta.cc:287] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.521s	user 0.009s	sys 0.004s
I20260709 13:19:30.221776 16648 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.353545 17018 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.354064 17018 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.354846 17150 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:30.355232 16648 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:30.320168 12777 raft_consensus.cc:1275] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:19:30.260066 17137 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:30.356031 17018 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:30.356662 12781 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:30.356355 17137 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.349337 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:30.357883 12783 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:30.347524 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:30.355351 17150 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:30.329780 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 53581.1us
I20260709 13:19:30.358564 12782 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:30.351644 12779 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:30.333993 16848 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:30.361399 16848 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.349507 17118 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.363473 16848 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:30.361001 12778 raft_consensus.cc:2410] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:19:30.364750 17118 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:30.366426 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:30.367142 13382 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:30.376302 12779 raft_consensus.cc:1275] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:30.377650 12630 raft_consensus.cc:1275] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:30.379756 17086 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:30.379463 17152 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.383081 17097 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.385187 17097 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:30.386579 17108 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:30.388401 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:30.394248 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:30.394999 12707 raft_consensus.cc:2368] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 13. Current term is 14.
I20260709 13:19:30.389377 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:30.389914 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:30.390133 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:30.390619 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:30.391230 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:30.391279 17060 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.266146 17171 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.398494 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:30.396193 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:30.400691 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 115945us
I20260709 13:19:30.365727 17120 consensus_queue.cc:1243] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 639ab2db265f4747bb7afb93e833f530 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:19:30.403702 12779 raft_consensus.cc:1275] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:19:30.404033 12779 pending_rounds.cc:85] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 6
I20260709 13:19:30.341508 17103 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.406719 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:30.405095 13382 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:30.404855 12779 pending_rounds.cc:107] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:19:30.407994 12779 raft_consensus.cc:2889] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:30.408289 12672 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:30.375222 17174 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:30.409768 12783 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:30.409997 17129 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:30.407336 17173 consensus_queue.cc:1048] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:19:30.412310 17129 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:30.412842 17129 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:30.419373 17073 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:30.420218 17021 raft_consensus.cc:3055] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:30.420331 17174 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.420986 17021 raft_consensus.cc:740] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:30.412364 12782 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:30.422173 17021 consensus_queue.cc:260] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.422847 17174 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:30.423666 17021 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:30.426170 12774 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:30.371472 17150 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.432708 13349 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:30.435292 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:30.406733 17086 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:30.441428 17150 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:30.437409 17108 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:30.442458 17108 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.444854 17108 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:30.445294 17103 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:30.446040 12672 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:30.446319 17179 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:30.446777 17179 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:30.446902 17171 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:30.447176 17179 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:30.447533 17173 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:30.447350 12672 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:30.448096 17173 raft_consensus.cc:697] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:30.448740 12672 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:30.449523 16664 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:30.449730 16941 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:30.450237 16664 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:30.450620 16664 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:30.450690 17007 consensus_queue.cc:1048] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:30.451056 16941 consensus_queue.cc:1048] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:30.451032 16664 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:30.452411 12672 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:30.452551 17060 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:30.453159 17140 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:30.453671 17140 raft_consensus.cc:697] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:30.455024 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:30.459671 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:30.454559 17140 consensus_queue.cc:237] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.463968 17007 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:30.464504 17007 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.467623 17007 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:30.468569 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.438565 16648 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.410827 17139 consensus_queue.cc:1059] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:19:30.408891 17149 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.474010 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:30.484949 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:30.485667 13382 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:30.468343 17152 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:30.500085 16664 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.505474 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:30.506314 12783 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:30.516355 12783 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:30.523720 12672 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:30.526777 17179 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:30.530260 17176 log.cc:927] Time spent T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:19:30.530871 17179 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:30.504206 16664 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:30.461838 17086 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.475989 17108 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:30.545426 13382 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:30.484818 16648 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.449108 17173 consensus_queue.cc:237] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.520701 17178 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:30.531977 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:30.552062 13377 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:30.488060 17097 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:30.539155 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:30.539794 17139 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:30.546921 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:30.519740 12629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:19:30.553469 12776 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:30.560541 17086 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:30.562592 12779 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:30.564883 12629 raft_consensus.cc:606] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:19:30.566960 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.580897 12783 raft_consensus.cc:1240] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:30.563519 17101 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:30.566598 17097 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.596247 17097 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:30.601440 12782 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:19:30.601760 16648 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:30.577157 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:19:30.586486 17002 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:30.602800 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:30.573832 17139 raft_consensus.cc:697] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:30.604305 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:30.609179 13377 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:30.623716 12701 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:30.568783 17178 raft_consensus.cc:697] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:30.550194 17108 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.625679 17059 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:30.608220 12783 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:36488
I20260709 13:19:30.573218 17137 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:30.630797 12783 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:30.631171 12783 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:30.633750 12672 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:30.642001 12701 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:19:30.647459 17149 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:30.648116 17140 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:30.648572 17140 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:30.648981 17140 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:30.657727 17108 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:30.661226 17114 log.cc:927] Time spent T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb: Append to log took a long time: real 1.405s	user 0.005s	sys 0.010s
I20260709 13:19:30.642762 12783 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.665700 17178 consensus_queue.cc:237] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.670156 13383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 702) took 2473 ms. Trace:
I20260709 13:19:30.629719 17179 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:30.675828 12776 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:30.678251 13383 rpcz_store.cc:276] 0709 13:19:28.196235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.196375 (+   140us) service_pool.cc:224] Handling call
0709 13:19:28.196698 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:28.335897 (+139199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:28.335932 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:28.335979 (+    47us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:28.336012 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.669849 (+2333837us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54002652d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:30.669915 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:30.670074 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":319196928}
I20260709 13:19:30.685276 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:30.686124 12776 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:19:30.689765 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 64758.8us
I20260709 13:19:30.628278 17070 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.692374 12783 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:30.635432 17059 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.691255 17179 raft_consensus.cc:697] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:19:30.711027 17184 log.cc:927] Time spent T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:19:30.715060 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:30.715900 12783 raft_consensus.cc:2393] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 9: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:30.716456 12776 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:19:30.716710 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:30.803972 12783 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:30.805186 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:30.806146 12776 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:30.805050 12672 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:30.797895 17190 log.cc:927] Time spent T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:19:30.786497 16899 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.816265 17118 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:30.786602 17086 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.797712 17125 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:30.709553 17139 consensus_queue.cc:237] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.827235 17118 raft_consensus.cc:697] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:30.836009 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:30.827180 17086 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.823495 16899 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.835541 17129 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.852535 12774 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:30.858623 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:19:30.861024 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.863307 17136 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:30.828842 17125 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.869474 12774 raft_consensus.cc:1240] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:30.874221 12706 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:30.875838 17132 raft_consensus.cc:3055] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:30.876394 17132 raft_consensus.cc:740] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:30.873659 12776 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:30.878634 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:30.866320 17070 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:30.832962 17179 consensus_queue.cc:237] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.877419 17132 consensus_queue.cc:260] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.889353 17132 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:30.891494 16899 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:30.874493 17140 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.897221 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:30.897504 17129 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:30.898005 12777 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:30.898948 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:30.899102 17101 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.899960 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:30.899864 17101 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.871407 16878 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:19:30.899317 17140 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:30.904562 17086 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:30.905784 17059 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:30.907727 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:30.908859 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:30.910708 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.912021 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:30.913147 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:30.909427 12780 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:30.914820 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:30.915522 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:30.915930 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:30.913703 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:30.916548 12781 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:19:30.916662 12778 raft_consensus.cc:2393] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 10: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:30.911504 16998 consensus_queue.cc:260] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.918232 17118 consensus_queue.cc:237] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:30.919241 16998 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:30.922950 17125 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:30.924103 16941 consensus_queue.cc:1048] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:30.924358 12672 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:30.925549 12672 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:30.926071 16941 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:30.926484 16941 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:30.926862 16941 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:30.928046 17101 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:30.916077 12780 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:30.930299 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:30.930938 12780 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:30.926801 17027 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:30.931802 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:30.932394 12780 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:30.917402 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:30.934036 12778 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:19:30.944242 17013 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:30.948621 12777 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:19:30.952384 12782 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:30.970548 17007 raft_consensus.cc:3055] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:30.970995 17007 raft_consensus.cc:740] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:30.971910 17007 consensus_queue.cc:260] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.973744 12777 pending_rounds.cc:85] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 8
I20260709 13:19:30.977743 12777 pending_rounds.cc:107] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:19:30.978112 12777 raft_consensus.cc:2889] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:30.980926 12781 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:30.973306 17007 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:30.954348 12775 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:30.955250 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:30.986658 12703 raft_consensus.cc:2376] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13. Re-sending same reply.
I20260709 13:19:30.956746 12672 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:30.959101 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:30.961817 17173 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:30.963219 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:30.963413 16941 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:30.964448 12778 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:31.002696 13383 raft_consensus.cc:2393] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 16: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:31.003851 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:31.004518 12778 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:31.004204 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:30.965467 12708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 561) took 1054 ms. Trace:
I20260709 13:19:31.005553 12709 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:19:31.005716 12708 rpcz_store.cc:276] 0709 13:19:29.910904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:30.049927 (+139023us) service_pool.cc:224] Handling call
0709 13:19:30.162408 (+112481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:30.175950 (+ 13542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.965265 (+789315us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b540011c1d0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:30.965407 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":238187776}
I20260709 13:19:31.007817 12703 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:30.991218 12782 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:19:30.994593 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:30.999068 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:31.000748 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:30.975425 12520 maintenance_manager.cc:643] P 480284ff372740a09dfd1f972d55d7eb: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 7.529s	user 0.193s	sys 0.081s Metrics: {"bytes_written":30686,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":227937,"dirs.run_cpu_time_us":665,"dirs.run_wall_time_us":19233,"drs_written":1,"lbm_read_time_us":54907,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":3,"lbm_write_time_us":411407,"lbm_writes_1-10_ms":2,"lbm_writes_10-100_ms":6,"lbm_writes_gt_100_ms":1,"lbm_writes_lt_1ms":24,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":92342,"threads_started":1,"update_count":32}
I20260709 13:19:30.965412 13349 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:31.015309 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 49995.9us
W20260709 13:19:31.016062 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 1.86404e+06us
I20260709 13:19:31.017885 13348 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:31.001802 12672 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:31.021325 12778 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:19:31.021960 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:31.022256 17138 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:31.022549 17079 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:31.023087 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:31.026268 17128 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.032829 13348 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:31.035971 17132 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:31.049060 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:31.050774 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:31.053833 13380 raft_consensus.cc:1240] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:31.036646 17125 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:31.037092 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:31.040493 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:31.041860 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:31.042706 13382 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:19:31.046871 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:31.007592 17122 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:31.057219 12777 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:31.059870 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:31.060789 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:31.061542 13384 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:31.063333 16941 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:31.064530 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:31.066159 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:31.002084 17173 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:31.068814 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:31.069247 12747 leader_election.cc:400] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 13. Current term is 14.
I20260709 13:19:31.041954 17138 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:31.073160 12747 leader_election.cc:403] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:31.074252 13384 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:31.076241 12705 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:31.051177 17089 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:31.079066 12777 raft_consensus.cc:2393] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 9: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:31.086699 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:31.054201 17079 raft_consensus.cc:697] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 11 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:31.078384 17138 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:31.077886 17136 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:31.078713 17122 raft_consensus.cc:697] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:31.099241 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:19:31.100075 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.103299 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:31.103523 12706 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:31.045431 17070 consensus_queue.cc:1059] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:19:31.124141 13385 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:31.124331 16982 consensus_queue.cc:1059] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:31.125847 12521 maintenance_manager.cc:419] P 480284ff372740a09dfd1f972d55d7eb: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8085 bytes on disk
I20260709 13:19:31.127470 17147 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:31.121636 17132 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:19:31.128082 17147 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:31.128453 12520 maintenance_manager.cc:643] P 480284ff372740a09dfd1f972d55d7eb: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.002s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":340,"lbm_reads_lt_1ms":4}
I20260709 13:19:31.131377 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:31.131923 12747 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:31.105720 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:31.134096 12702 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:19:31.072640 16941 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.136055 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:31.137337 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:31.137809 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:31.138403 12633 raft_consensus.cc:2368] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 13. Current term is 14.
I20260709 13:19:31.140018 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:31.051530 17128 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:31.097509 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:31.140977 12627 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:31.143350 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:31.151084 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:31.152591 13378 raft_consensus.cc:2393] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 13: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:31.148813 12597 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:31.153282 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:31.154439 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:31.151994 12781 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:31.156226 12781 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:31.156530 12782 raft_consensus.cc:1275] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:19:31.154460 12597 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:31.160001 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:31.163448 12747 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:31.123317 17089 raft_consensus.cc:697] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:31.161204 12597 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:31.164959 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:31.126143 16982 raft_consensus.cc:3055] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:31.178292 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:31.179966 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:31.144618 17147 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:31.184191 12597 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:31.164839 16899 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:31.183609 17101 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:31.146004 17085 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:31.160732 17059 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.146925 17173 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:31.178814 17108 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:31.161217 16921 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:31.206745 12781 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:19:31.226326 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:31.226691 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:31.190286 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:31.227829 12627 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:31.210172 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:31.216231 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:31.160403 17095 consensus_queue.cc:1059] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:19:31.143663 16998 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:19:31.181833 16982 raft_consensus.cc:740] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:31.231623 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:31.164877 17080 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:19:31.201905 17085 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.179044 17136 consensus_queue.cc:1059] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:19:31.248196 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:31.246925 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:31.194087 17101 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:31.257285 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 217565us
I20260709 13:19:31.294093 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:31.259435 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:31.294296 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:31.261124 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:31.295653 12778 raft_consensus.cc:2393] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 16: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:31.265051 12705 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:31.267022 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:31.280452 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:31.220257 17108 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.299196 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:31.233479 16899 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:31.274127 16982 consensus_queue.cc:260] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:31.275517 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:31.305858 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:31.188872 17089 consensus_queue.cc:237] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:31.124186 17079 consensus_queue.cc:237] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:31.301874 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:31.302112 12780 raft_consensus.cc:2393] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 13: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:19:31.305851 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.307446 13385 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:31.281997 12781 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:19:31.179652 16941 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:31.280067 17085 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:31.315052 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:31.316000 12625 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
W20260709 13:19:31.316119 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.316995 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:31.220453 17059 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:31.317585 12625 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:19:31.300851 17101 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:31.320699 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:31.301321 17108 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:31.309067 17128 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:31.325999 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 141906us
I20260709 13:19:31.217861 17122 consensus_queue.cc:237] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:31.333539 12783 raft_consensus.cc:1240] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:31.336362 12597 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:31.337399 12633 raft_consensus.cc:1240] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 7. Current term is 9. Ops: []
I20260709 13:19:31.339049 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:31.341795 12597 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:31.344870 12597 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:31.324538 17007 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:31.309962 17011 consensus_queue.cc:1048] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:19:31.332963 16899 raft_consensus.cc:3055] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:31.327036 17039 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:31.334075 16982 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:31.334160 17132 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 13. Current term is 14.
I20260709 13:19:31.335352 17124 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:31.354979 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:31.366698 12631 raft_consensus.cc:1240] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:19:31.367748 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:31.359628 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:31.356709 16899 raft_consensus.cc:740] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:31.356313 17007 raft_consensus.cc:697] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:31.385790 12780 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:19:31.398830 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:31.403941 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:31.406293 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:31.406900 12631 raft_consensus.cc:3055] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:31.407209 12631 raft_consensus.cc:740] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:31.407868 12631 consensus_queue.cc:260] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:31.408764 12631 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:31.410171 12672 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:31.411861 17126 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:31.412250 17146 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:31.412632 17126 raft_consensus.cc:697] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:31.413772 17146 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:31.413973 17126 consensus_queue.cc:237] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:31.415653 17146 consensus_queue.cc:1048] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:19:31.423753 17201 log.cc:927] Time spent T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:19:31.429450 17013 raft_consensus.cc:3055] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:31.430023 17013 raft_consensus.cc:740] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:31.431126 17013 consensus_queue.cc:260] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:31.432919 17013 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:31.363255 17039 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:31.435683 17007 consensus_queue.cc:237] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:31.450402 12781 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:19:31.454454 17070 raft_consensus.cc:3055] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:31.454895 17070 raft_consensus.cc:740] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:31.455713 17070 consensus_queue.cc:260] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:31.456714 17070 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:31.457938 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:31.458048 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:31.393843 17197 log.cc:927] Time spent T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:19:31.460386 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:31.462070 12627 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:31.466410 12780 raft_consensus.cc:2410] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:19:31.469244 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:31.469660 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:31.470252 12779 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:19:31.470463 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:31.470772 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:31.480084 12781 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:19:31.481779 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:31.482398 12708 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:31.483093 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:31.483798 12704 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 14: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
W20260709 13:19:31.496461 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.498368 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:31.501343 12781 raft_consensus.cc:3055] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:31.502389 12781 raft_consensus.cc:740] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:31.503377 12781 consensus_queue.cc:260] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:31.375942 17124 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:31.512833 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:31.523998 12631 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
I20260709 13:19:31.525071 12708 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:31.526177 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:31.518244 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:31.528553 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:31.464399 17137 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:31.542021 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: true dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:31.542768 13379 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:31.548808 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:31.551013 12746 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:31.552670 12746 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:31.555555 13348 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:31.563715 13348 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:31.561407 17173 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:31.530292 17132 consensus_queue.cc:1059] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:19:31.541520 17086 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:31.558987 17128 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.565155 17164 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:31.561690 16998 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:31.539139 16886 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:19:31.569849 13348 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:31.565300 17059 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:31.566313 17022 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:31.527592 16899 consensus_queue.cc:260] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:31.595161 13348 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:31.565065 17208 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:31.602283 16899 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:31.598475 17059 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:31.605754 17059 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:31.606180 17059 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:31.590409 17128 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:31.594710 17173 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:31.613148 17128 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:31.561105 17011 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:19:31.616235 17124 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:31.616506 17208 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:31.617300 17039 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:31.617293 17208 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:19:31.642781 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 337594us
I20260709 13:19:31.627121 17208 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:31.653720 17208 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:31.661698 12633 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:19:31.621945 17011 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:31.621146 17173 consensus_queue.cc:1059] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:19:31.626305 17022 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:19:31.642176 16921 raft_consensus.cc:697] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:31.692201 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:31.708662 12630 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:19:31.713238 13379 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 8.
W20260709 13:19:31.738209 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:31.694115 17138 consensus_meta.cc:287] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.547s	user 0.010s	sys 0.001s
I20260709 13:19:31.748343 17138 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:31.752971 17138 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:31.753860 12703 raft_consensus.cc:2393] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 13: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:31.756134 16982 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:31.756731 16982 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:31.760509 16982 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:31.761912 16982 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.763067 16982 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:31.713124 17039 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.803208 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:31.822171 17039 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:31.740818 17210 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.115s
W20260709 13:19:31.845124 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:31.834055 17202 log.cc:927] Time spent T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.243s	user 0.000s	sys 0.000s
I20260709 13:19:31.833376 16982 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:31.874576 12598 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:31.834419 16941 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:31.886241 12627 raft_consensus.cc:1275] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:19:31.877763 17139 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.022s	sys 0.044s
W20260709 13:19:31.876623 17203 log.cc:927] Time spent T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.394s	user 0.002s	sys 0.003s
I20260709 13:19:31.902367 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:31.903683 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.902866 12629 raft_consensus.cc:1240] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: []
W20260709 13:19:31.876199 17085 consensus_meta.cc:287] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.553s	user 0.011s	sys 0.001s
W20260709 13:19:31.903908 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 344376us
W20260709 13:19:31.892623 17108 consensus_meta.cc:287] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.548s	user 0.006s	sys 0.006s
W20260709 13:19:31.901832 17139 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.022s	sys 0.045s
I20260709 13:19:31.906311 12598 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:31.922891 12746 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:31.929071 12747 leader_election.cc:400] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 13. Current term is 14.
I20260709 13:19:31.935688 12598 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:31.843242 16921 consensus_queue.cc:237] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:31.918972 17209 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:31.937049 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:31.938475 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:31.940969 12630 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:31.941838 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.942170 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:31.918503 17086 raft_consensus.cc:697] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:31.947921 12747 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:31.948161 12598 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:31.950464 12598 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:31.931574 17136 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:31.963477 12747 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:31.975617 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:31.914502 17085 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:31.918807 17059 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:31.985591 12598 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:31.992367 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:31.945016 17132 consensus_queue.cc:1243] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Peer 639ab2db265f4747bb7afb93e833f530 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:19:31.928886 17108 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:31.952620 16998 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:32.007517 12774 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2609) took 1011 ms. Trace:
I20260709 13:19:31.945470 17209 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:32.009056 12774 rpcz_store.cc:276] 0709 13:19:30.996510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.047998 (+ 51488us) service_pool.cc:224] Handling call
0709 13:19:31.048409 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:31.058947 (+ 10538us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:31.058994 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:31.069849 (+ 10855us) log.cc:844] Serialized 26 byte log entry
0709 13:19:31.296426 (+226577us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:31.296468 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:31.296534 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:31.833343 (+536809us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:31.998268 (+164925us) raft_consensus.cc:1685] finished
0709 13:19:31.998301 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:32.007424 (+  9123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8412,"raft.queue_time_us":3297,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":6661,"spinlock_wait_cycles":42480896,"thread_start_us":196434,"threads_started":1,"wal-append.queue_time_us":382326}
I20260709 13:19:32.012122 16998 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.012217 13349 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:31.976339 17126 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:19:32.012814 16998 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:32.014281 17089 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:32.017231 17070 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:32.018602 12627 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:19:32.019697 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:32.020505 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:32.020807 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:32.021910 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:32.022413 17022 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:19:32.022946 17013 consensus_meta.cc:287] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.542s	user 0.007s	sys 0.002s
I20260709 13:19:32.022883 17022 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:32.023427 17209 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:32.023634 17013 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:31.959501 17208 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:32.024668 17013 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:32.023543 17022 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:32.025918 12628 raft_consensus.cc:1240] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:32.026535 17208 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:32.027812 17132 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:32.028573 16998 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:31.959740 17098 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:32.029443 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:32.029670 17138 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:32.030369 17085 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:32.030272 17138 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:32.031483 12517 catalog_manager.cc:5244] ChangeConfig:REMOVE_PEER RPC for tablet 6609d5f93ab048558ea46e511ef3b2ec with cas_config_opid_index 4: aborting the task: latest config opid_index 5; task opid_index 4
I20260709 13:19:32.032740 17138 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:32.006999 17130 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:32.033895 17130 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:32.036187 17130 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:31.957871 17164 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:32.038988 13377 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:19:32.039801 13383 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:19:32.025982 17039 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:32.042763 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:19:32.054322 12597 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.024s	sys 0.039s
W20260709 13:19:32.054857 12597 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.025s	sys 0.040s
I20260709 13:19:31.959089 17213 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:32.055276 12701 raft_consensus.cc:1240] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:31.953413 17211 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:19:32.055465 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 701306us
I20260709 13:19:32.056646 12631 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:32.002911 17218 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:32.065093 12627 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:32.058759 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:32.071180 12708 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:32.071593 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:32.068609 12776 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:19:32.073623 12703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1785) took 1016 ms. Trace:
I20260709 13:19:32.066506 12597 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:32.073975 12628 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:32.074488 12709 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:32.047613 17125 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.042160 17164 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:32.078315 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:32.076275 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:32.074024 12703 rpcz_store.cc:276] 0709 13:19:31.057378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.057491 (+   113us) service_pool.cc:224] Handling call
0709 13:19:32.073423 (+1015932us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b5400404f54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:32.073550 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1594296192}
I20260709 13:19:32.080792 12597 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:32.076496 13377 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:19:32.080940 12782 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:32.086365 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:32.059281 17098 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:32.006422 17086 consensus_queue.cc:237] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:32.075301 17125 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:32.085810 17211 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:32.079362 17039 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:19:32.086606 12597 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:32.096589 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:32.097133 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:32.099764 12626 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:32.098941 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:32.102605 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:32.108580 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:32.109035 12708 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:19:32.109762 12634 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:32.112571 12633 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:32.112565 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:32.110447 12708 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:19:32.116791 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:32.104600 16899 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:32.118785 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:32.120597 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:32.088774 17164 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:32.126549 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:32.127437 12783 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:32.133970 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:32.164536 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:32.175694 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 68182us
I20260709 13:19:32.113838 17208 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:32.138137 12704 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:32.120544 16899 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:32.205255 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:32.205864 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:32.206660 13377 raft_consensus.cc:2393] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 13: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:32.206965 12630 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:32.207938 12627 raft_consensus.cc:1275] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:19:32.220016 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:32.132594 17013 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:32.279119 17224 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:19:32.279907 17226 log.cc:927] Time spent T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:19:32.280193 17217 log.cc:927] Time spent T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:19:32.280998 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:32.280998 17108 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:32.282042 17059 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:32.284574 13383 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:19:32.281994 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 57545.4us
W20260709 13:19:32.287878 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:32.298381 17125 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:32.355962 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:32.368093 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:32.207710 16899 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:32.132066 17022 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:32.391513 12673 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.004s	sys 0.027s
W20260709 13:19:32.393251 12673 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.005s	sys 0.027s
W20260709 13:19:32.393538 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 530461us
W20260709 13:19:32.392450 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 416879us
I20260709 13:19:32.397045 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:32.400121 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:32.400869 12747 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:32.338192 17224 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:32.409659 12781 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:32.416429 12672 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:32.419052 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:32.420222 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 434733us
I20260709 13:19:32.421448 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:32.409346 12747 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:32.418130 12672 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:32.425069 12747 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:32.418803 17224 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:32.445032 12598 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:32.446357 12747 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:32.374293 17013 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:32.401743 17229 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:32.407581 17228 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:32.466589 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:32.435073 17085 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:32.434805 17128 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:19:32.471841 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 365213us
I20260709 13:19:32.476431 12781 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:19:32.471755 12747 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:32.435343 17227 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.419490 17124 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.050s
W20260709 13:19:32.486874 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 68820.4us
I20260709 13:19:32.511288 12700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 2150) took 1043 ms. Trace:
I20260709 13:19:32.479354 17085 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:32.564175 17085 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:32.569424 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:32.491333 16886 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:32.570389 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:32.573550 12632 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:32.562340 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:32.561933 12700 rpcz_store.cc:276] 0709 13:19:31.468134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.468249 (+   115us) service_pool.cc:224] Handling call
0709 13:19:31.468507 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:31.538548 (+ 70041us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:19:31.906837 (+368289us) log.cc:844] Serialized 14 byte log entry
0709 13:19:32.307645 (+400808us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:32.307700 (+    55us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:32.307748 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.511026 (+203278us) spinlock_profiling.cc:243] Waited 69.8 ms on lock 0x7b5400249ad4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:32.511080 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:32.511215 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63994,"prepare.queue_time_us":464470,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":42,"raft.queue_time_us":61363,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":11298,"spinlock_wait_cycles":160513920,"thread_start_us":687763,"threads_started":2,"wal-append.queue_time_us":552679}
W20260709 13:19:32.551852 17101 consensus_meta.cc:287] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 1.222s	user 0.010s	sys 0.004s
I20260709 13:19:32.572888 12781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2632) took 1084 ms. Trace:
I20260709 13:19:32.468111 17108 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:19:32.572371 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:32.472522 17132 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:19:32.580888 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 109344us
I20260709 13:19:32.585361 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:32.545405 17007 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.579443 12781 rpcz_store.cc:276] 0709 13:19:31.477575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.480914 (+  3339us) service_pool.cc:224] Handling call
0709 13:19:31.481145 (+   231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:31.830539 (+349394us) spinlock_profiling.cc:243] Waited 7.09 ms on lock 0x7b58007accd0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d30e0aa 00007f4c6d390447 00007f4c6d3a8fe7 00007f4c6d3aa7de 00007f4c6d3b4b49 00007f4c6d3b5416 00007f4c6d3b1d8f 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4
0709 13:19:32.409110 (+578571us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54001f34a0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d327a1d 00007f4c6d30e1a4 00007f4c6d390447 00007f4c6d3a8fe7 00007f4c6d3aa7de 00007f4c6d3b4b49 00007f4c6d3b5416 00007f4c6d3b1d8f 00007f4c6d3b183e 00007f4c72376419
0709 13:19:32.561331 (+152221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.561543 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":413535360}
I20260709 13:19:32.478708 17128 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:32.468369 17136 raft_consensus.cc:697] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:32.576820 16886 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:32.520429 17098 raft_consensus.cc:697] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:32.583256 17108 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:32.597944 17128 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:32.618763 16886 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:32.522647 17245 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:32.620738 12597 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:32.590487 17132 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:32.615537 17007 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:32.622658 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 61749.8us
I20260709 13:19:32.638725 12632 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:32.619764 17248 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:32.645215 17242 log.cc:927] Time spent T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:19:32.576597 17227 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:32.630990 17229 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.646313 16982 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:32.579978 17101 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:32.677175 17101 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:32.678465 13383 raft_consensus.cc:1240] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:19:32.640575 12597 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:32.634532 17108 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:32.641160 17245 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:32.685704 17245 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:32.686183 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 45792.5us
I20260709 13:19:32.686981 12597 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:32.598382 17247 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:32.690112 17247 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.690735 17247 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:32.651854 16921 consensus_queue.cc:1059] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:19:32.693403 17247 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:32.694103 17251 log.cc:927] Time spent T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:19:32.694598 17243 log.cc:927] Time spent T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
W20260709 13:19:32.659864 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 116214us
I20260709 13:19:32.619516 17228 consensus_queue.cc:1059] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:19:32.696074 17228 raft_consensus.cc:3055] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:32.696496 17228 raft_consensus.cc:740] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:32.697526 17228 consensus_queue.cc:260] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:32.699100 17228 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:32.654217 17022 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:32.700866 17250 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:32.661396 17248 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:32.701593 17250 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:32.702952 17248 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:32.705224 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:32.702220 17247 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.706863 17247 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:32.707238 17098 consensus_queue.cc:237] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:32.707988 17209 consensus_meta.cc:287] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.684s	user 0.006s	sys 0.003s
I20260709 13:19:32.708354 17209 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:32.710879 16899 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:32.713052 16899 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:32.618115 17244 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:32.714037 17085 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:32.714681 12628 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
W20260709 13:19:32.667747 17164 consensus_meta.cc:287] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.528s	user 0.005s	sys 0.005s
I20260709 13:19:32.667960 16710 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:32.715715 17164 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:32.668203 17095 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:32.647392 17132 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:32.718247 17247 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:32.669883 17229 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:32.722072 17245 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:32.677608 17227 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:32.673529 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:32.727044 17132 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:32.673874 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:32.674158 17013 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:32.642694 17136 consensus_queue.cc:237] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:32.730087 17213 raft_consensus.cc:993] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: : Instructing follower 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:19:32.730614 17213 raft_consensus.cc:1081] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Signalling peer 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:19:32.733361 12626 raft_consensus.cc:1240] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:32.735276 12625 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:19:32.736419 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:32.737128 12632 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:19:32.738456 12747 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:32.749943 12775 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:19:32.752583 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:32.757479 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:32.759160 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:32.644198 17007 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:32.739308 16710 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.786108 16710 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:32.789168 16710 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:32.789901 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:32.799494 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:32.802749 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:32.775655 17264 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:32.815186 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 126714us
I20260709 13:19:32.784374 17234 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:19:32.839345 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:19:32.833602 17211 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.008s	sys 0.016s
W20260709 13:19:32.844381 17211 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.008s	sys 0.017s
W20260709 13:19:32.866870 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 60213.6us
I20260709 13:19:32.885509 17233 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:32.886128 17233 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:32.886906 12775 raft_consensus.cc:1240] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:32.888345 17233 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:32.861756 17085 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:32.892241 12782 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:19:32.896593 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:32.897213 12783 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:19:32.902992 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:32.903424 12709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 2155) took 1433 ms. Trace:
I20260709 13:19:32.903931 12709 rpcz_store.cc:276] 0709 13:19:31.469882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.471236 (+  1354us) service_pool.cc:224] Handling call
0709 13:19:32.074314 (+603078us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:32.074874 (+   560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.903182 (+828308us) spinlock_profiling.cc:243] Waited 603 ms on lock 0x7b5400404f50. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:32.903349 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1386387328}
I20260709 13:19:32.905030 12781 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:19:32.862668 17211 raft_consensus.cc:697] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:32.908293 12775 raft_consensus.cc:1240] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:19:32.886832 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:32.911695 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:32.913725 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:32.914273 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:32.914881 12780 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:32.919986 12782 raft_consensus.cc:1240] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:32.916852 17209 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:32.907413 17164 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:32.912185 17125 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.930688 17268 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.938832 17274 consensus_queue.cc:1059] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:32.948571 17108 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:32.949698 12632 raft_consensus.cc:1240] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:32.951678 17108 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:32.957729 17211 consensus_queue.cc:237] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:32.928673 17279 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:32.928921 17234 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:19:32.914539 16886 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:32.967289 16886 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:32.961404 17279 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:32.970170 17279 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:19:32.944705 17125 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:32.968348 17248 raft_consensus.cc:3055] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:19:32.972272 17216 log.cc:927] Time spent T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.784s	user 0.006s	sys 0.000s
I20260709 13:19:32.972236 17248 raft_consensus.cc:740] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:32.974200 17248 consensus_queue.cc:260] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:32.944399 17268 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:32.975231 17132 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:32.975522 17248 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:32.984560 17229 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:32.986793 17245 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:32.987655 17140 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:32.990021 17011 raft_consensus.cc:3055] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:32.990429 17011 raft_consensus.cc:740] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:32.989791 17128 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:32.991252 17011 consensus_queue.cc:260] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:32.973383 17125 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:33.000317 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 298308us
I20260709 13:19:33.001480 17139 consensus_queue.cc:1048] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:33.002101 17139 consensus_queue.cc:1243] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:19:33.006067 17139 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:33.009689 17140 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:32.957269 17224 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:33.010216 17140 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:33.012285 17140 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:33.012738 17140 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.018854 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:33.019474 12700 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:33.021867 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:33.022419 17128 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:33.022485 12705 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:19:33.023720 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:33.024255 12706 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:33.024298 12631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2894) took 1009 ms. Trace:
I20260709 13:19:33.024775 12631 rpcz_store.cc:276] 0709 13:19:32.014937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.022983 (+  8046us) service_pool.cc:224] Handling call
0709 13:19:32.023273 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:32.067065 (+ 43792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.024090 (+957025us) spinlock_profiling.cc:243] Waited 33.2 ms on lock 0x7b540002c954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:33.024244 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76251520}
I20260709 13:19:32.994660 17268 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:33.025910 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:33.026654 12709 raft_consensus.cc:2393] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 10: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:33.027890 17224 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:33.028810 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:33.028985 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:33.029987 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:33.030536 12707 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:33.034997 12706 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:33.002686 12673 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:33.040565 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:33.042243 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:33.042385 12702 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:33.043771 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:33.044788 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:33.046449 12701 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:33.046489 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:33.047806 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:33.049597 17229 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:33.050160 17229 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:33.052282 17229 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:33.052932 17132 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:33.053133 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:33.056348 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:33.056977 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:33.058288 17085 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.058586 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:33.058954 17085 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:33.059744 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:33.062528 17245 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:33.063118 17245 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:33.063670 17245 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:33.066716 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:33.067190 12775 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:19:33.067700 12775 pending_rounds.cc:85] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 6
I20260709 13:19:33.068035 12775 pending_rounds.cc:107] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:19:33.068075 12701 raft_consensus.cc:1240] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:33.068594 12775 raft_consensus.cc:2889] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:33.070858 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:33.071929 12700 raft_consensus.cc:2410] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:19:33.072685 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:33.074306 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:33.072578 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:33.074829 12701 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:19:33.075628 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:33.077701 12705 raft_consensus.cc:1240] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:33.074321 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:33.079473 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:33.080560 12706 raft_consensus.cc:2368] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10.
I20260709 13:19:33.085664 12780 raft_consensus.cc:1240] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:19:33.090304 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:33.090448 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:33.091146 12775 raft_consensus.cc:2393] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 13: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:33.091516 12780 raft_consensus.cc:2393] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 9: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:33.093580 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:33.097044 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 94418.4us
W20260709 13:19:33.101011 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 54638.6us
I20260709 13:19:33.102854 12747 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:33.104106 12747 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:33.013159 17140 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:33.080399 12702 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:19:33.107419 12700 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:19:33.107969 12780 raft_consensus.cc:1275] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:19:33.110041 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:33.116848 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:33.117389 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:33.029515 17264 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:33.122606 12597 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:33.123178 12781 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:33.124308 17248 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:33.124666 17264 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:33.122606 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:33.125113 17264 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:33.126529 12597 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:33.129379 12779 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:36488
I20260709 13:19:33.142004 12779 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:33.143911 12779 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:33.144747 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.147367 12747 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:33.148430 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:33.150063 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 50426.3us
W20260709 13:19:33.149223 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 49487.6us
I20260709 13:19:33.151441 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:33.053494 17132 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:33.126600 17086 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:33.158798 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:33.142589 17210 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:33.138768 17208 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
W20260709 13:19:33.197180 17231 log.cc:927] Time spent T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.699s	user 0.000s	sys 0.001s
I20260709 13:19:33.126581 17164 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:33.118271 17229 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:33.118459 17085 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:33.142378 17268 consensus_queue.cc:1059] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:19:33.218873 17268 raft_consensus.cc:3055] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:33.219300 17268 raft_consensus.cc:740] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:33.146900 17264 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:33.181077 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 264097us
I20260709 13:19:33.153661 12747 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:33.233299 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 79720.7us
I20260709 13:19:33.117389 17128 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.138856 17125 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:33.242841 17125 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:33.244457 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:33.245067 12776 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
I20260709 13:19:33.149858 17013 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:33.193835 17211 consensus_queue.cc:1048] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:33.152585 17227 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:33.246237 17227 raft_consensus.cc:697] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:19:33.245589 17284 log.cc:927] Time spent T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:19:33.246588 12776 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:19:33.247056 17236 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.247148 17227 consensus_queue.cc:237] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:33.247674 17236 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:33.247911 17268 consensus_queue.cc:260] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:33.249236 17268 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:33.128973 17269 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:33.149930 17223 consensus_queue.cc:1059] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:19:33.173482 17138 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:33.250281 17288 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.236907 17086 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:33.250903 17086 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:33.250788 17288 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:33.252853 17086 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:33.237404 17229 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:33.238123 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:33.254256 17236 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:33.255000 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:33.256543 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:33.257128 13383 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:19:33.253376 17288 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:33.208803 17260 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:19:33.260658 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:33.262297 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:33.263315 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:33.263568 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:33.199668 12673 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:33.185456 17059 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:33.224210 12777 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:33.263623 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:33.242769 17128 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:33.276718 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:33.279289 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:33.280581 12631 raft_consensus.cc:2410] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:19:33.266181 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:33.266805 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:33.282876 13377 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:33.267828 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:33.245301 17125 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:33.261433 17253 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:33.296535 17234 consensus_queue.cc:1059] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:33.285492 17059 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:33.297127 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:33.282747 12779 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:33.312412 13385 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:33.317945 13348 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:33.318136 12779 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:33.319339 13348 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:33.321909 17128 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:33.326795 17132 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:33.330340 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.337553 12746 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:33.338838 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:33.342833 12777 raft_consensus.cc:2410] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:19:33.343551 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:33.346060 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:33.341171 12746 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:33.299319 17245 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:33.346514 13348 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:33.338548 17212 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:33.340231 17234 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:33.341938 17132 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:33.353924 13348 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:33.339684 17286 raft_consensus.cc:3055] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:33.355010 12783 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:33.358255 17253 consensus_queue.cc:1059] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:33.359683 17253 raft_consensus.cc:3055] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:33.360072 17253 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:33.338693 17080 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:19:33.362486 12778 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:33.366663 17059 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:33.368288 17282 log.cc:927] Time spent T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:19:33.373045 17245 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:33.349783 17013 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:19:33.375200 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.382865 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:33.383558 12783 raft_consensus.cc:2393] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 9: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:33.350903 17212 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:33.356137 17236 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:33.356539 17140 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:33.356875 17286 raft_consensus.cc:740] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:33.407814 17286 consensus_queue.cc:260] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:33.408735 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:33.409139 17286 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:33.412117 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:33.412039 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:33.413074 12629 raft_consensus.cc:2393] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 11: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:33.413527 12626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 760) took 1104 ms. Trace:
I20260709 13:19:33.414135 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:33.413954 12626 rpcz_store.cc:276] 0709 13:19:32.309595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.309889 (+   294us) service_pool.cc:224] Handling call
0709 13:19:32.310269 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:32.733788 (+423519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.413307 (+679519us) spinlock_profiling.cc:243] Waited 423 ms on lock 0x7b54002ed454. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:33.413479 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":972585344}
I20260709 13:19:33.353708 17164 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:33.361029 17253 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:33.435128 12631 raft_consensus.cc:1275] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:19:33.436604 17164 consensus_queue.cc:1048] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:33.384070 17013 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:33.390539 17212 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:33.385771 17089 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:33.443353 12780 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:19:33.452813 12776 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:33.438553 17234 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:33.453986 13379 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:19:33.439661 17013 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:33.448333 17253 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:33.470750 17264 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:33.473393 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:33.479938 17011 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:33.480273 17022 consensus_queue.cc:1059] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:33.481475 17022 raft_consensus.cc:3055] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:33.481845 17022 raft_consensus.cc:740] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:33.482771 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:33.456218 17211 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:33.468573 17234 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:33.455782 17249 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:19:33.496712 17271 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:33.515936 12634 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:19:33.479732 17264 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:33.525254 17300 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:33.530159 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 64926.4us
W20260709 13:19:33.532262 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 347205us
I20260709 13:19:33.554088 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:33.554787 12706 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:33.532609 17130 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:33.533365 17007 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.536141 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:33.536639 17223 raft_consensus.cc:3055] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:33.536880 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:33.537451 17039 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:19:33.537509 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 337891us
I20260709 13:19:33.536208 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:33.538409 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:33.539528 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:33.491407 17164 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:33.559979 17011 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:33.560640 17089 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:33.561039 17300 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:33.513129 17132 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:33.569742 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:33.486186 17211 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:33.566289 13385 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:33.541754 17268 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:33.576298 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:33.576473 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:33.576500 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:33.577378 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:33.587778 12702 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:33.573838 12706 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:19:33.591835 12747 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:33.592711 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.593019 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:33.596196 12747 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:33.598474 12673 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:33.571542 17223 raft_consensus.cc:740] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:33.599361 17245 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:33.522143 17022 consensus_queue.cc:260] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:33.609791 12673 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:33.611259 17178 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:19:33.622231 17137 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:33.623047 17137 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:33.623493 17212 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:33.582775 17089 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:33.590236 16710 consensus_queue.cc:1059] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:19:33.596568 17268 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:33.576611 17007 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:33.584430 17164 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:33.594931 17086 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:33.656013 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:33.626608 17085 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:33.587978 17011 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:33.602511 17227 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:33.589419 17269 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:19:33.662711 12673 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:19:33.670837 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 104178us
I20260709 13:19:33.588819 17132 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:33.712718 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 50122.4us
I20260709 13:19:33.713950 12673 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:33.647117 16994 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:33.650017 17262 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:33.715389 12673 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:33.724097 12673 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:33.716267 12708 raft_consensus.cc:1275] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:19:33.611140 17140 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 13:19:33.674252 12633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 2917) took 1281 ms. Trace:
I20260709 13:19:33.669044 17245 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:33.662307 17137 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:33.730367 12633 rpcz_store.cc:276] 0709 13:19:32.392988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.393197 (+   209us) service_pool.cc:224] Handling call
0709 13:19:32.719387 (+326190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:32.719680 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:32.719708 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:32.719747 (+    39us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:32.719776 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:32.719830 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.674083 (+954253us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b5400159ad0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:33.674197 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":394,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"spinlock_wait_cycles":749446528}
I20260709 13:19:33.675560 17022 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:33.746075 12779 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:33.716150 17234 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:33.649863 17178 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 13:19:33.719151 17147 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:33.729434 17179 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:33.673687 17130 consensus_queue.cc:1059] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:19:33.721915 16941 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:19:33.753607 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 157509us
I20260709 13:19:33.770370 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:33.674857 17085 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:33.722704 17227 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:33.670543 17138 consensus_queue.cc:1059] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 13:19:33.775763 17137 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:33.747445 12783 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:33.790774 17288 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.822856 12783 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:33.791914 12702 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:19:33.775030 16994 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:33.828554 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:33.810038 17208 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:33.810564 17250 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.811568 17089 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:33.772593 17234 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.836170 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:33.784891 17306 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.850448 12780 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:19:33.815630 17249 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:33.780131 17227 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:33.815215 17108 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:33.910557 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:33.894352 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:33.832146 17209 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.833232 17229 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:33.904963 17297 log.cc:927] Time spent T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.277s	user 0.000s	sys 0.001s
I20260709 13:19:33.975165 12780 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
W20260709 13:19:33.995497 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 93771.4us
I20260709 13:19:33.822180 17013 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.015326 12672 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:33.838150 17288 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.035512 12780 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:33.971163 17128 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:33.918754 17307 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.852118 17208 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:33.972244 17300 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:34.042232 13383 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:19:34.043404 12775 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:19:33.892238 17250 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.047122 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:34.003527 17268 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:34.022271 17253 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:34.010623 17212 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:34.010489 17007 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:33.907526 17234 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.017889 17164 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:34.032366 17011 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:34.050846 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:34.067809 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.056282 13379 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:19:34.004705 17209 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.021894 17248 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:33.925355 17249 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.043951 16710 consensus_queue.cc:1243] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer 3d539f8d95d343d8b1696803f18b30d2 log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:19:34.004029 17108 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.029728 17264 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:34.069034 12598 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:34.069285 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
W20260709 13:19:34.066588 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 342567us
I20260709 13:19:34.050194 17137 raft_consensus.cc:697] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:34.079169 17013 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:34.023823 17138 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:19:34.065001 17085 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:34.065516 17253 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.004565 17229 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.014375 17261 consensus_queue.cc:1059] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:19:33.912256 17306 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.050112 17311 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:34.061084 17132 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:34.098995 17248 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.094594 17178 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:33.995136 17223 consensus_queue.cc:260] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.048171 17128 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.110951 12673 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:34.112890 17308 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:34.125293 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 56087.9us
I20260709 13:19:34.136157 13349 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:34.131536 12673 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:34.051388 17307 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.125254 17313 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.129705 17178 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:34.143955 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:34.109643 17264 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.135390 17233 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:34.147678 17305 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:34.158178 17178 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:34.148806 17227 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:34.172168 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:34.167451 17309 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:19:34.172320 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.173382 17138 raft_consensus.cc:3055] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:34.162531 17233 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:34.125752 17311 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.179275 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:34.182824 12634 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 6.
W20260709 13:19:34.183516 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 162815us
I20260709 13:19:34.157960 17313 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.141011 17308 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.175860 16710 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:34.148044 17137 consensus_queue.cc:237] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.199321 16998 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:34.177717 17325 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:34.178357 17288 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:34.183979 17233 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:34.205767 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:34.206355 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:34.207466 12774 raft_consensus.cc:1240] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:34.192742 17250 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:34.212694 17288 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:34.179111 17138 raft_consensus.cc:740] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:34.213107 17224 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:34.213621 17138 consensus_queue.cc:260] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.176581 17321 consensus_queue.cc:1059] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:19:34.196717 16710 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:34.235375 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 74314.4us
I20260709 13:19:34.222772 12747 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:34.178084 17312 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:19:34.204535 17234 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:34.202414 17209 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:34.209882 17329 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:34.240543 12747 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:34.240792 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 134536us
I20260709 13:19:34.192113 17323 consensus_queue.cc:1059] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:19:34.213104 17288 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:34.363837 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:34.364701 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 124339us
W20260709 13:19:34.314618 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.374130 12634 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:19:34.252898 12598 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:34.378432 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.316633 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:34.372773 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 184441us
I20260709 13:19:34.335422 12778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 2322) took 1279 ms. Trace:
I20260709 13:19:34.387118 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.376886 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
W20260709 13:19:34.291152 17320 log.cc:927] Time spent T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:19:34.388672 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:34.390661 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:34.392144 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 139437us
I20260709 13:19:34.393461 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:34.232409 16710 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:34.238860 17138 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:34.231036 17325 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:34.402484 12706 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:34.358677 17311 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:34.406459 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.360733 17245 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:34.387529 12778 rpcz_store.cc:276] 0709 13:19:33.055650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.069154 (+ 13504us) service_pool.cc:224] Handling call
0709 13:19:33.069405 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.364748 (+295343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.335112 (+970364us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b540040e554. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:34.335343 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":673576832}
I20260709 13:19:34.267010 17209 raft_consensus.cc:3055] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:34.234583 17328 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:34.233918 17326 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:34.325424 17307 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:34.360055 17211 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:34.266438 17248 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:34.266810 17136 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.388757 17227 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:34.416024 17085 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:34.460289 17085 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.232465 17324 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:34.462396 17085 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:34.462769 17324 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.357939 17313 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:34.445793 17245 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:34.466105 17245 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.468086 17245 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:34.404894 12598 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:34.469381 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 64640.4us
I20260709 13:19:34.267197 17322 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:34.237869 17253 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:34.275550 17306 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:34.257078 17229 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:34.275369 17128 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:34.472467 12598 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:34.473997 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 236544us
I20260709 13:19:34.399214 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.253237 17329 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:34.476662 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:34.378504 17288 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:34.409960 17336 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:19:34.424052 17318 log.cc:927] Time spent T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 13:19:34.257342 17261 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.159s
W20260709 13:19:34.426106 17302 log.cc:927] Time spent T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.671s	user 0.005s	sys 0.000s
I20260709 13:19:34.267630 17140 consensus_queue.cc:1048] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.147s
I20260709 13:19:34.389930 17333 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.298130 17331 consensus_queue.cc:1059] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.131s
I20260709 13:19:34.427667 12774 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 2320) took 1372 ms. Trace:
I20260709 13:19:34.232692 17224 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.394547 17164 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:34.499743 17164 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.252199 17108 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:34.415724 17013 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:34.502238 17013 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.504482 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:34.504748 17013 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:34.413578 17132 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:34.505146 12629 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:19:34.505426 17132 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.507216 17249 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:34.439389 17340 consensus_queue.cc:1059] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:19:34.456099 17178 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.415313 17325 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.457870 17227 raft_consensus.cc:697] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:34.431437 17209 raft_consensus.cc:740] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:19:34.458698 17022 consensus_meta.cc:287] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.704s	user 0.007s	sys 0.005s
I20260709 13:19:34.458348 17136 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:34.459841 17179 consensus_meta.cc:287] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.691s	user 0.012s	sys 0.000s
I20260709 13:19:34.299599 17330 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:34.479351 12598 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:34.524132 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 44862.6us
W20260709 13:19:34.479898 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 95544.2us
I20260709 13:19:34.470515 17248 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:34.484122 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:34.525514 12634 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:34.461122 17233 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.487213 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.465445 17324 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:34.487946 12633 raft_consensus.cc:1217] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Deduplicated request from leader. Original: 8.5->[9.6-12.8]   Dedup: 9.6->[10.7-12.8]
I20260709 13:19:34.528883 12633 pending_rounds.cc:85] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 6
I20260709 13:19:34.529119 12633 pending_rounds.cc:107] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:19:34.529353 12633 raft_consensus.cc:2889] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:34.490061 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:34.493011 12774 rpcz_store.cc:276] 0709 13:19:33.054947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.065668 (+ 10721us) service_pool.cc:224] Handling call
0709 13:19:33.066024 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:34.212569 (+1146545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.427396 (+214827us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400127854. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:34.427587 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2624583424}
I20260709 13:19:34.480631 17271 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:34.496781 12625 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:19:34.497798 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:34.498986 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:34.499207 17224 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:34.478442 17329 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:34.511958 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:34.512020 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:34.475057 17253 consensus_queue.cc:1059] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:19:34.536023 12629 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 13: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:19:34.523016 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.526742 12598 leader_election.cc:400] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10.
I20260709 13:19:34.537707 12598 leader_election.cc:403] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:19:34.514756 17164 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:34.541541 12672 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:34.542603 13379 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:34.527216 17336 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:34.539484 17132 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:34.547551 17259 consensus_queue.cc:1059] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:34.548628 17259 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:34.549098 17259 raft_consensus.cc:697] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:34.544252 12598 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:34.533533 17330 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.549957 17259 consensus_queue.cc:237] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.552718 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:34.553013 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:34.552107 17330 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:34.553727 12517 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 4c13fa2aff324f65ad0bf6191ea68fab with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 639ab2db265f4747bb7afb93e833f530 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:34.554126 17271 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:34.554312 17136 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:34.555615 17329 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.556108 17233 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:34.556696 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:34.545411 12632 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:34.526335 17179 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:34.558001 17329 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:34.549319 16710 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.563275 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:34.564064 12628 raft_consensus.cc:2393] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 15: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:34.566918 17179 consensus_queue.cc:1059] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:34.567404 12672 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:34.569072 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:34.570067 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:34.575711 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:34.577268 17347 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:19:34.579228 17261 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:34.579650 17261 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:34.580004 17261 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:34.560160 16710 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:34.581771 17349 raft_consensus.cc:3055] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:34.582217 17349 raft_consensus.cc:740] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:34.583029 17349 consensus_queue.cc:260] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:34.585299 17342 log.cc:927] Time spent T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:19:34.585664 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:34.586501 12632 raft_consensus.cc:2410] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:19:34.587070 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:34.587615 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:34.588325 13384 raft_consensus.cc:2393] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 15: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:34.583125 12634 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:34.588857 17224 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10.
I20260709 13:19:34.495244 17333 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.587647 13383 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:19:34.563047 17340 raft_consensus.cc:3055] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:34.593524 12597 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:34.573987 17329 raft_consensus.cc:3055] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:34.607963 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:34.609192 17253 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:34.538422 17209 consensus_queue.cc:260] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.589775 17274 raft_consensus.cc:3055] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:34.620165 17336 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:34.623232 17336 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:34.623593 17336 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:34.598165 17340 raft_consensus.cc:740] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:34.609871 17132 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:34.619325 17253 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:34.630386 17253 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:34.629922 17340 consensus_queue.cc:260] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.640196 17340 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:34.645051 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:34.645903 12774 raft_consensus.cc:2393] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 11: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:34.646721 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:34.648342 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:34.649457 12774 raft_consensus.cc:2410] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:19:34.647506 12775 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:19:34.652848 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:34.655464 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:34.610399 17329 raft_consensus.cc:740] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:34.662832 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:34.657415 13349 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:34.664314 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.665892 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:34.666936 12625 raft_consensus.cc:2393] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 11: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:34.667068 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.667490 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.668398 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.669004 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:34.669634 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.671066 13348 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:34.670009 12627 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:34.642328 17274 raft_consensus.cc:740] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:34.681842 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:34.682137 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.682879 12701 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:34.685307 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:34.685773 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:34.686522 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:34.686674 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:34.687245 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:34.687609 12625 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 11: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:34.686158 13349 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:34.687095 12634 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:19:34.693400 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:34.694205 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:34.695672 12774 raft_consensus.cc:2410] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:19:34.696462 17013 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:34.696730 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:34.698194 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:34.629870 17132 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.594373 17227 consensus_queue.cc:237] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:34.705288 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 140939us
I20260709 13:19:34.719074 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:34.725189 12634 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:34.721803 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 153199us
I20260709 13:19:34.679946 17306 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:34.680217 17307 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:34.732841 12701 raft_consensus.cc:1240] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:34.690598 17234 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:34.698351 17164 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:34.690557 17330 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:34.738292 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.703621 17305 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.744632 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.745970 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:34.747931 12597 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.647s (SENT) [suppressed 8 similar messages]
W20260709 13:19:34.753865 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:34.754557 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.732456 17013 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:34.757252 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.734169 17307 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:34.734606 17306 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:34.762825 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.765445 12706 raft_consensus.cc:2410] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 13:19:34.756263 17345 log.cc:927] Time spent T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 13:19:34.739092 17354 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:34.767632 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.769695 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.772059 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.746182 17178 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:34.763803 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:34.776893 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.780964 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.783646 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.784749 12747 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:34.764991 17307 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:34.768153 17306 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:34.761464 17013 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.769243 17164 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.790362 12673 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:34.798079 12709 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:34.791527 17330 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:34.772596 17325 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:34.735095 17329 consensus_queue.cc:260] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.800993 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.760704 17305 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.791936 17136 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:34.756323 17274 consensus_queue.cc:260] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.794994 17315 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.819388 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.798535 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:34.812582 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.813982 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.822928 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.767303 17361 consensus_queue.cc:1059] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:19:34.816454 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.803858 12673 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:34.823454 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:34.801542 12634 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:19:34.806540 17118 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:34.805884 17330 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:34.828969 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:34.830032 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:34.794935 17262 consensus_queue.cc:1059] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:19:34.818987 17136 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:34.837548 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:34.825471 16998 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:34.842553 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.846549 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:34.846632 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:34.844025 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.802487 17322 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:19:34.853715 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:34.804620 17164 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.834448 17330 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:34.831956 17274 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:34.831564 17311 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:34.856624 17137 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:34.839201 17234 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:34.870007 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:34.848371 17358 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:34.871503 17108 raft_consensus.cc:3055] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:34.841878 17128 raft_consensus.cc:3055] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:34.868065 17326 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:34.848243 17313 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:34.850178 17136 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:34.905228 12774 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:19:34.829000 17315 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.849841 17333 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:34.863911 17085 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:19:34.914897 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 62205.4us
W20260709 13:19:34.918249 17367 log.cc:927] Time spent T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:19:34.911247 17128 raft_consensus.cc:740] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:34.943053 17128 consensus_queue.cc:260] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.907702 17108 raft_consensus.cc:740] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:34.946611 17108 consensus_queue.cc:260] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.948385 17108 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:34.925776 17353 log.cc:927] Time spent T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
W20260709 13:19:34.926573 17359 log.cc:927] Time spent T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:19:34.944913 17128 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:34.927809 17305 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:34.929232 17363 log.cc:927] Time spent T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 13:19:34.916872 17234 raft_consensus.cc:697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:19:34.930708 17351 log.cc:927] Time spent T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:19:34.930371 17253 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.831720 17229 consensus_queue.cc:1048] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:19:34.968588 17164 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:34.867268 17118 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:19:34.919147 17326 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.934794 17245 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:34.965245 17138 consensus_meta.cc:287] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.550s	user 0.011s	sys 0.001s
I20260709 13:19:34.970292 17245 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.972250 17326 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:34.952176 17261 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.960021 17336 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.939337 17132 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:34.974577 17261 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:34.951995 17352 log.cc:927] Time spent T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.405s	user 0.002s	sys 0.000s
W20260709 13:19:34.966527 17130 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.190s	user 0.005s	sys 0.005s
I20260709 13:19:34.901130 17262 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:34.967960 17234 consensus_queue.cc:237] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.975620 17262 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.977710 17262 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:34.972318 17253 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:34.974982 17349 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:34.979249 17348 raft_consensus.cc:3055] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:34.979655 17348 raft_consensus.cc:740] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:34.980057 17288 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.980436 17348 consensus_queue.cc:260] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.939493 17137 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:34.981835 17137 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.982116 17315 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:34.982178 17147 raft_consensus.cc:3055] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:34.972313 17245 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:34.982686 17147 raft_consensus.cc:740] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:34.974812 17336 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:34.985904 17313 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:34.986745 17314 raft_consensus.cc:3055] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:34.987115 17314 raft_consensus.cc:740] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:34.988018 17288 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:34.987859 17314 consensus_queue.cc:260] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:34.988220 17307 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:34.989070 17314 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:34.989861 17354 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:34.997368 17307 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:35.048077 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:35.049307 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:35.051877 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:35.062350 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 147587us
I20260709 13:19:35.146402 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:35.146867 17271 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.147676 17271 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:35.064558 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:35.150086 12700 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:35.066076 12701 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:34.916132 12704 raft_consensus.cc:1240] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:34.922798 12705 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 11. Current term is 12. Ops: [11.12-11.12]
I20260709 13:19:35.067678 12706 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:35.069689 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:35.077224 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:34.998834 17308 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:34.997684 17330 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:35.109725 17322 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:35.112797 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.166451 12780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2676) took 1333 ms. Trace:
I20260709 13:19:35.166162 17322 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:35.166965 12780 rpcz_store.cc:276] 0709 13:19:33.832844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.006900 (+174056us) service_pool.cc:224] Handling call
0709 13:19:34.035271 (+ 28371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:34.036024 (+   753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:35.165944 (+1129920us) spinlock_profiling.cc:243] Waited 28 ms on lock 0x7b54001fd950. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:35.166238 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64428288}
I20260709 13:19:35.118547 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:35.170109 12779 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:19:35.119920 17313 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.171478 17313 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:35.174825 17313 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:34.983940 17130 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.209s	user 0.005s	sys 0.005s
W20260709 13:19:35.176091 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.158843 12627 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:19:35.120597 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:35.170445 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:35.169227 17322 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:35.190073 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:35.191148 12628 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
W20260709 13:19:35.196029 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.204342 17208 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:35.205293 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:35.206236 12631 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:35.210206 17137 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:35.212576 17264 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:35.216398 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:35.217779 17321 raft_consensus.cc:3055] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:35.218343 17321 raft_consensus.cc:740] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:35.220105 17321 consensus_queue.cc:260] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:35.220810 17348 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:35.227761 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:19:35.230911 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.251470 17269 raft_consensus.cc:3055] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:35.252120 17269 raft_consensus.cc:740] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:35.254173 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:35.260181 17330 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:35.253260 17269 consensus_queue.cc:260] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:35.293541 17341 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.006s	sys 0.016s
W20260709 13:19:35.294081 17341 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.007s	sys 0.016s
I20260709 13:19:35.294200 17321 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:35.242609 17137 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:35.297801 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.259756 17013 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:35.298099 17137 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:35.299975 13378 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 13:19:35.300473 13378 pending_rounds.cc:85] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd: Aborting all ops after (but not including) 11
I20260709 13:19:35.300755 13378 pending_rounds.cc:107] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:19:35.301136 13378 raft_consensus.cc:2889] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:35.301730 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 86338.4us
I20260709 13:19:35.293391 17269 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:35.307535 13382 raft_consensus.cc:1240] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:35.308750 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:19:35.309026 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.310879 17325 consensus_queue.cc:1059] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:35.312047 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:35.324939 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:35.327008 13349 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:33.313580 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:35.327497 17288 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:35.288822 17136 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:35.301303 17137 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:35.332116 17288 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.342216 17288 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:35.343442 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:35.343734 17209 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:35.344616 13378 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 11: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:35.346092 13382 raft_consensus.cc:1240] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:35.346903 13376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51762 (request call id 771) took 1290 ms. Trace:
I20260709 13:19:35.355346 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:35.356691 13377 raft_consensus.cc:2410] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:19:35.357775 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:35.359370 17380 log.cc:927] Time spent T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.164s	user 0.002s	sys 0.002s
I20260709 13:19:35.347393 13376 rpcz_store.cc:276] 0709 13:19:34.056724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.056875 (+   151us) service_pool.cc:224] Handling call
0709 13:19:34.057208 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:34.161754 (+104546us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:34.161784 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:34.164992 (+  3208us) log.cc:844] Serialized 26 byte log entry
0709 13:19:34.499429 (+334437us) spinlock_profiling.cc:243] Waited 2.85 ms on lock 0x7b1800089b00. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf520d1 00007f4c6d36da0f
0709 13:19:34.544831 (+ 45402us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:34.544882 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.951336 (+406454us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:35.346259 (+394923us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b540026cad4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:35.346336 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:35.346421 (+    85us) raft_consensus.cc:1685] finished
0709 13:19:35.346442 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:35.346710 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3654,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":1337,"spinlock_wait_cycles":276453760,"thread_start_us":45237,"threads_started":1,"wal-append.queue_time_us":381739}
I20260709 13:19:35.345882 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:35.364351 17306 consensus_meta.cc:287] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.558s	user 0.010s	sys 0.008s
I20260709 13:19:35.365022 17306 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:35.367619 17306 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:35.368882 17249 raft_consensus.cc:3055] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:35.369474 17249 raft_consensus.cc:740] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:35.370805 17249 consensus_queue.cc:260] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:35.372056 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:35.372403 17249 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:35.369225 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:35.375043 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:35.375953 12782 raft_consensus.cc:2393] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 12: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:35.378126 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:35.378377 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:35.378552 12632 raft_consensus.cc:1240] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:35.381332 12626 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:35.390112 12781 raft_consensus.cc:2368] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14.
I20260709 13:19:35.373546 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:35.392159 12780 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:35.371299 17329 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:35.322517 17013 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:35.368793 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:35.393006 12781 raft_consensus.cc:1240] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:35.328399 17264 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:35.414315 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 87378.3us
I20260709 13:19:35.415863 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:35.416988 13379 raft_consensus.cc:2393] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 11: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:35.417845 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:35.423851 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:35.424000 13385 raft_consensus.cc:1240] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:35.425935 12774 raft_consensus.cc:2393] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 14: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:35.431731 12778 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:35.433297 12777 raft_consensus.cc:2393] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 13: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:35.436707 12700 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:19:35.410794 17271 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:35.431591 17307 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:35.451393 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:35.456492 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:35.442925 17013 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:35.468250 17013 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:35.474539 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:35.474740 17348 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:35.475625 12774 raft_consensus.cc:2410] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:19:35.476694 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:35.477958 12774 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:35.481755 12778 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:35.482528 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:35.454288 17264 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:35.486824 17264 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:35.488147 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:35.489894 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:35.490909 12783 raft_consensus.cc:2393] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 9: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:35.456720 17223 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:35.519454 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:35.520242 12777 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:35.452909 17307 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.528997 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:35.524101 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:35.524531 17013 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:35.531179 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.530937 17013 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:35.532586 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.535787 12631 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:19:35.536586 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:35.536859 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:35.537276 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.537518 12701 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:19:35.537531 12706 raft_consensus.cc:2393] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:35.524598 17307 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:35.550994 17288 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:35.551596 17128 consensus_meta.cc:287] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.588s	user 0.008s	sys 0.008s
I20260709 13:19:35.547905 17013 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:35.569957 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:35.571168 13376 raft_consensus.cc:2393] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 9: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:35.570233 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:35.566023 12673 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:35.565689 17136 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:35.583397 16710 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:35.582999 17147 consensus_queue.cc:260] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:35.587569 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:35.588476 12701 raft_consensus.cc:2393] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 14: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
W20260709 13:19:35.596040 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 50158.6us
I20260709 13:19:35.601060 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:35.584118 16710 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:35.606966 12631 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:35.611734 16994 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:35.612416 16994 raft_consensus.cc:697] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:35.613552 16994 consensus_queue.cc:237] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:35.617697 17147 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:35.620273 17288 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:35.622217 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:35.627895 12598 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:35.631340 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 166992us
W20260709 13:19:35.632182 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.631686 17249 consensus_queue.cc:1059] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:19:35.648955 17136 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.654475 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:35.654301 17136 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:35.640287 17128 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.656415 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:35.662297 17321 consensus_queue.cc:1059] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:35.663722 17325 raft_consensus.cc:3055] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:35.664145 17325 raft_consensus.cc:740] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:35.665844 17013 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:35.653682 17300 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:35.664115 12746 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:35.669322 12746 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:35.670316 17271 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.672681 17136 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:35.670403 17013 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:35.675076 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:35.672202 17271 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:35.665756 17325 consensus_queue.cc:260] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:35.681618 17108 consensus_meta.cc:287] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.683s	user 0.011s	sys 0.014s
I20260709 13:19:35.683126 17086 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:35.685516 17271 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:35.688490 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 60732us
I20260709 13:19:35.687526 12746 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:35.691084 17086 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:35.693046 12746 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:35.697350 12627 raft_consensus.cc:1240] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:19:35.698303 12628 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:19:35.698460 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:35.698745 17128 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:35.693362 17386 log.cc:927] Time spent T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:19:35.699494 17086 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:35.700462 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.702709 16710 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:19:35.703302 13348 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:35.708369 13385 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:35.719713 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:35.722420 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:35.723145 12626 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:19:35.724972 12707 raft_consensus.cc:2393] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 14: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:35.709748 17147 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.722580 12625 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:35.746273 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:35.752202 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 49086.1us
I20260709 13:19:35.758582 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:35.761365 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:35.744804 17136 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:35.764878 17147 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:35.747538 16710 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:35.761116 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:35.767566 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:35.715848 17086 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:35.772982 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.761808 12597 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:35.770620 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:35.777834 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:35.782922 12779 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:35.788599 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:35.794162 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 215386us
I20260709 13:19:35.797103 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:35.793936 12707 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:19:35.797118 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:35.800323 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:35.800060 12630 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:35.802099 12701 raft_consensus.cc:1275] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:19:35.787967 17300 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:35.810338 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:35.811241 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:35.812032 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:35.812875 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:35.813632 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:35.817178 12630 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:35.795837 17249 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:19:35.819782 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 58113.8us
W20260709 13:19:35.822521 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 129642us
I20260709 13:19:35.824890 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:35.826238 12633 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:35.826632 13349 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:35.829531 17316 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:35.830801 12747 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:35.831815 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:35.832563 17136 consensus_queue.cc:1059] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:35.832762 17264 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:35.833680 17136 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:35.834101 17136 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.832664 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:35.834545 17341 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.834884 12630 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:19:35.838725 13349 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:35.835266 17341 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:35.836308 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.840010 17316 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:35.839148 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:35.841112 13349 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:35.842041 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:35.842859 17140 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:35.842981 17344 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.843302 13349 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:35.843535 12632 raft_consensus.cc:3055] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:35.843413 17140 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:35.843959 12632 raft_consensus.cc:740] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:35.843814 17338 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:35.844647 17234 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:35.845527 17234 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:35.845971 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:35.834623 17322 raft_consensus.cc:3055] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:35.834466 17136 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:35.843557 17344 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:35.846818 12634 raft_consensus.cc:2393] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 13: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:35.840061 12703 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:19:35.842051 12672 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:35.834479 17264 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:35.824442 12746 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:35.833117 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:35.852968 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:35.844924 12632 consensus_queue.cc:260] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:35.854295 12633 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:35.856946 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:35.857828 12701 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:19:35.857846 12633 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:35.860005 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:35.860273 12704 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:19:35.846597 17140 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:35.847328 17322 raft_consensus.cc:740] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:35.866745 12597 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:35.851704 17264 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:35.870746 12707 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:35.872143 12779 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:19:35.876022 12633 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:19:35.877923 12632 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:35.833360 17307 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:35.874697 17264 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:35.887579 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:35.894747 13379 raft_consensus.cc:2393] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 14: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:35.879336 17330 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:35.879482 16710 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.876791 17249 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:35.895694 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:35.894572 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:35.898340 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:35.894170 17322 consensus_queue.cc:260] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:35.884460 17348 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:19:35.907965 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 51514.4us
I20260709 13:19:35.908030 12705 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:35.908308 17288 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:19:35.909170 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 84801.3us
W20260709 13:19:35.911346 17329 consensus_meta.cc:287] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.001s
I20260709 13:19:35.911727 12707 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:19:35.912153 12630 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 13:19:35.914026 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 72090.2us
I20260709 13:19:35.889468 17137 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:35.919380 12746 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:35.917436 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:35.901289 17013 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:19:35.933931 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 100882us
I20260709 13:19:35.920095 17137 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:35.921927 17288 raft_consensus.cc:697] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 13 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:35.952732 12746 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:35.955722 12708 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:35.943464 17013 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:35.942728 17086 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:35.949908 17137 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:35.922287 16710 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:35.974261 12597 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:35.975379 17253 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:35.976354 12597 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:35.977166 17249 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:35.977610 17249 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:35.978021 17249 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:35.978869 17274 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:19:35.934705 17330 consensus_queue.cc:1059] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:19:35.981360 17288 consensus_queue.cc:237] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:35.986307 17403 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:35.989943 17344 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:35.999336 17321 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:35.999677 12703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1824) took 1331 ms. Trace:
I20260709 13:19:36.000092 17321 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.000305 12703 rpcz_store.cc:276] 0709 13:19:34.668075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.668190 (+   115us) service_pool.cc:224] Handling call
0709 13:19:35.999443 (+1331253us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54001d9054. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:35.999602 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2692739840}
I20260709 13:19:36.002739 17321 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:36.003187 16710 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:36.003494 17325 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:36.012013 12597 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:36.023229 12627 raft_consensus.cc:1240] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:36.024600 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:36.026319 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 58010.3us
I20260709 13:19:36.049124 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:36.028659 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:36.050166 12625 raft_consensus.cc:2410] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:19:36.050256 12628 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:36.030738 12708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 2184) took 1289 ms. Trace:
I20260709 13:19:36.031150 17271 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:36.052834 12708 rpcz_store.cc:276] 0709 13:19:34.741278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.741487 (+   209us) service_pool.cc:224] Handling call
0709 13:19:36.030563 (+1289076us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54002486d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:36.030680 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2695963904}
I20260709 13:19:36.053078 17271 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.055460 17321 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:36.055745 17271 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:36.032151 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:36.056042 17321 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.056465 12626 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:36.009220 17147 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:36.058105 17321 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:36.059859 12673 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:36.065748 17321 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:36.073869 17341 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:36.082181 13349 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:36.085021 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:36.085259 17128 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:36.085604 17234 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:36.085757 12708 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:36.086108 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:36.086898 12633 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
W20260709 13:19:36.065223 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 53302.4us
I20260709 13:19:36.034935 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:36.088291 12627 raft_consensus.cc:2368] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13.
I20260709 13:19:36.088498 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:36.042384 17307 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:36.046626 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:36.090718 12631 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:36.098489 12631 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:19:36.103123 17137 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.037619 17322 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:36.107163 12528 catalog_manager.cc:5697] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 9 to 12, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 12 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:36.110571 13349 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:19:36.026947 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 74256.5us
W20260709 13:19:36.111173 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 61622us
I20260709 13:19:36.124971 17136 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.127524 17136 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:36.128069 17329 consensus_queue.cc:1059] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:36.131457 12747 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:36.132864 12672 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:36.112943 12707 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
W20260709 13:19:36.145871 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 86107.5us
I20260709 13:19:36.138469 12779 raft_consensus.cc:1240] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:36.140723 12528 catalog_manager.cc:5697] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 5 to 12, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 12 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:36.137950 12775 raft_consensus.cc:1275] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:36.133895 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:36.164717 12632 raft_consensus.cc:1275] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:19:36.137972 12598 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:36.165308 12632 pending_rounds.cc:85] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 7
W20260709 13:19:36.127543 12527 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54800 (request call id 230) took 41982 ms (client timeout 15000 ms). Trace:
I20260709 13:19:36.165916 12632 pending_rounds.cc:107] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:19:36.166533 12632 raft_consensus.cc:2889] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:36.162963 12747 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:36.168020 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:36.168593 12597 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:36.135800 17140 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:36.178566 12673 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:36.134271 17013 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:36.154088 17253 raft_consensus.cc:2764] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:19:36.136260 17307 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:19:36.144601 12517 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet af96e52b004d4c8c942cc0ec33c95c99 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:19:36.215176 17137 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:36.215618 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:36.139475 17264 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.137740 17348 consensus_queue.cc:1059] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 13:19:36.218556 17321 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:36.219683 16994 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:19:36.220649 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:36.222426 17136 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:36.170002 17403 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:36.219336 12528 catalog_manager.cc:5697] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:36.171085 17223 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:19:36.227831 12517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 83429.3us
I20260709 13:19:36.171742 17330 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:19:36.178853 17209 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
W20260709 13:19:36.240455 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 102626us
I20260709 13:19:36.240299 17130 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.242269 17403 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:36.240779 17136 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:36.250418 17264 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:36.239137 17321 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:36.170128 17086 consensus_queue.cc:1048] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:19:36.225528 17147 consensus_queue.cc:1059] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:19:36.258443 17348 raft_consensus.cc:3055] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:36.259028 12528 catalog_manager.cc:5697] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 11 to 14, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 14 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:36.247881 17223 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:36.269075 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:36.270058 17137 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.270598 17322 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.271072 17249 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.296497 17348 raft_consensus.cc:740] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:36.301879 17137 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.302716 17348 consensus_queue.cc:260] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:36.303459 17249 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:36.238730 16994 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:36.304085 16994 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:36.246768 17013 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.304561 17137 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:36.242463 17271 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:36.296535 17316 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.306318 17271 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:36.306769 17271 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:36.306442 17316 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:36.308591 17316 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:36.299464 17264 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:36.241122 17307 raft_consensus.cc:697] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:36.309194 17264 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.310539 17307 consensus_queue.cc:237] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:36.311431 17264 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:36.299623 12528 catalog_manager.cc:5697] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 8 to 11, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 11 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:36.312713 12528 catalog_manager.cc:5697] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 10 to 15, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 15 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:36.314144 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:36.301927 17322 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:36.274966 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 120475us
I20260709 13:19:36.304709 17013 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.325337 12673 leader_election.cc:400] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13.
I20260709 13:19:36.325927 12673 leader_election.cc:403] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:19:36.305984 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:36.304198 12630 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:36.284276 17330 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:36.289888 17321 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:36.166308 12527 rpcz_store.cc:269] 0709 13:18:54.145854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.146206 (+   352us) service_pool.cc:224] Handling call
0709 13:19:36.127327 (+41981121us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:28.717299 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:28.768971 (+ 51672us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:28.769040 (+    69us) write_op.cc:435] Acquired schema lock
0709 13:19:28.769067 (+    27us) tablet.cc:662] Decoding operations
0709 13:19:28.770597 (+  1530us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:28.770716 (+   119us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:28.770764 (+    48us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:19:28.868636 (+ 97872us) tablet.cc:701] Row locks acquired
0709 13:19:28.868697 (+    61us) write_op.cc:260] PREPARE: finished
0709 13:19:28.879378 (+ 10681us) write_op.cc:270] Start()
0709 13:19:28.901065 (+ 21687us) write_op.cc:276] Timestamp: P: 1783603168868804 usec, L: 0
0709 13:19:28.914065 (+ 13000us) op_driver.cc:348] REPLICATION: starting
0709 13:19:28.914799 (+   734us) log.cc:844] Serialized 4759 byte log entry
0709 13:19:30.659948 (+1745149us) spinlock_profiling.cc:243] Waited 6.32 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:30.793942 (+133994us) op_driver.cc:464] REPLICATION: finished
0709 13:19:30.987080 (+193138us) spinlock_profiling.cc:243] Waited 5.91 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6a2a80ac 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c70de8ef0
0709 13:19:31.228458 (+241378us) write_op.cc:301] APPLY: starting
0709 13:19:31.228556 (+    98us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:32.095944 (+867388us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:32.095971 (+    27us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:32.573084 (+477113us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:32.639229 (+ 66145us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=24,mrs_lookups=0
0709 13:19:32.652022 (+ 12793us) write_op.cc:312] APPLY: finished
0709 13:19:32.732349 (+ 80327us) log.cc:844] Serialized 113 byte log entry
0709 13:19:32.894721 (+162372us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:32.894934 (+   213us) write_op.cc:454] Released schema lock
0709 13:19:32.894994 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:32.918670 (+ 23676us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:32.921818 (+  3148us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":86103,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":434145,"apply.run_cpu_time_us":53408,"apply.run_wall_time_us":1686847,"cfile_cache_hit":46,"cfile_cache_hit_bytes":48116,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6219,"cfile_init":2,"lbm_read_time_us":119223,"lbm_reads_1-10_ms":5,"lbm_reads_10-100_ms":5,"lbm_reads_lt_1ms":2,"num_ops":12,"prepare.queue_time_us":569844,"prepare.run_cpu_time_us":22044,"prepare.run_wall_time_us":364294,"raft.queue_time_us":80801,"raft.run_cpu_time_us":34409,"raft.run_wall_time_us":355813,"replication_time_us":1879684,"spinlock_wait_cycles":28143872,"thread_start_us":1206784,"threads_started":5,"wal-append.queue_time_us":471820,"wal-append.run_cpu_time_us":38658,"wal-append.run_wall_time_us":3634075}]]}
I20260709 13:19:36.314438 12528 catalog_manager.cc:5697] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:36.329903 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:36.290328 17136 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:36.294126 17130 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.295102 17253 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.289738 17403 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:36.314993 16710 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:36.326776 17140 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:36.301638 17223 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:36.338609 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 46719us
I20260709 13:19:36.337953 12528 catalog_manager.cc:5697] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 8 to 9, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 9 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:19:36.356315 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 188409us
I20260709 13:19:36.331925 17330 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:36.360323 12747 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:36.363309 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 188544us
I20260709 13:19:36.336220 17249 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.372411 12528 catalog_manager.cc:5697] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 12 to 14, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 14 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:36.351946 17344 raft_consensus.cc:3055] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:36.381742 12528 catalog_manager.cc:5697] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:36.365161 17138 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:19:36.387754 17130 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:36.389349 17322 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:36.392091 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:36.377707 12597 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:36.356880 17140 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.368614 17137 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:36.375151 17325 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.371382 17300 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:36.415025 12747 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:36.415462 17399 log.cc:927] Time spent T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:19:36.427361 12528 catalog_manager.cc:5697] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:36.394141 17344 raft_consensus.cc:740] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:36.347056 17253 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.423729 17137 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:36.436899 17137 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:36.394731 17249 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.400295 17264 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:19:36.439430 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:36.437052 17408 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:36.422698 17086 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:36.444442 12528 catalog_manager.cc:5697] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 7 to 10, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 10 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:36.446911 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:36.447822 12775 raft_consensus.cc:2410] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:19:36.444455 13383 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:19:36.447899 12528 catalog_manager.cc:5697] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:19:36.456233 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 106679us
I20260709 13:19:36.427742 17306 consensus_queue.cc:1059] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
W20260709 13:19:36.457312 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 79758us
I20260709 13:19:36.459749 12628 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:19:36.460314 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:36.429215 17325 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.461755 12746 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:36.448591 17179 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:36.467921 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:36.468657 12775 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:36.465006 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:36.469801 12628 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:36.466840 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:36.449069 17086 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:36.437541 17274 consensus_queue.cc:1059] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:19:36.480926 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:36.476948 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:36.473027 12632 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:36.481745 12700 raft_consensus.cc:2393] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 14: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:36.482597 13348 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:36.484133 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:36.485160 12704 raft_consensus.cc:2393] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:36.489888 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:36.490594 12707 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:36.494274 17405 log.cc:927] Time spent T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:19:36.495635 17413 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.476742 12528 catalog_manager.cc:5697] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:36.497803 17013 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:36.498414 17406 log.cc:927] Time spent T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:19:36.498497 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:36.499698 17249 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:36.500126 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:36.501153 12634 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:36.477284 17086 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:36.509088 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:36.484251 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:36.510123 12709 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:19:36.498447 17013 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:36.512142 17013 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:36.508127 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:36.514067 12626 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:19:36.514632 17344 consensus_queue.cc:260] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:36.516031 17344 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:36.527499 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:36.514001 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:36.565044 12597 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:36.539469 12528 catalog_manager.cc:5697] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 6 to 8, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 8 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:36.552192 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:36.528896 12626 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:19:36.542367 12634 raft_consensus.cc:1275] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:36.549199 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:36.568382 12706 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:36.549835 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:36.562541 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:36.541154 12633 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:36.531296 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:36.572587 12701 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:19:36.575266 12708 raft_consensus.cc:2393] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 14: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:36.575603 12707 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:36.577528 12632 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:19:36.582594 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:36.583779 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 59274.1us
I20260709 13:19:36.522571 17348 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:36.590732 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:36.594492 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:36.595679 12705 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:36.532763 17306 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:36.548554 17140 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:36.599280 12625 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:36.562803 17341 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:36.560981 16994 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.606598 17324 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:36.608299 17269 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.608899 17269 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:36.608510 12528 catalog_manager.cc:5697] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:36.610685 12528 catalog_manager.cc:5697] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 6 to 13. New cstate: current_term: 13 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:36.611692 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:36.612380 12709 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:36.614708 17325 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:36.615278 17264 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:36.615758 17264 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:36.615409 17271 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.616200 17264 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:36.578058 17253 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:36.545679 17179 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:36.618151 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:36.619457 17179 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:36.619979 17179 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:36.620905 17138 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13.
I20260709 13:19:36.621847 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:36.622769 12627 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:36.623136 12597 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:36.583916 17209 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:36.623988 12709 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 14. Ops: []
I20260709 13:19:36.626343 12775 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:19:36.583483 17130 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:36.621122 17147 consensus_queue.cc:1059] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:19:36.629019 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:36.629534 16994 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:36.629683 17409 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:36.496193 17413 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.630982 12528 catalog_manager.cc:5697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 12 to 13, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 13 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:36.547430 17136 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.637594 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:36.645923 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:36.643309 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:36.646646 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:36.646103 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:36.641574 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:36.644500 12672 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:36.655202 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:36.655556 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:36.656531 12780 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:36.645128 12597 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:36.658926 12528 catalog_manager.cc:5697] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:36.659634 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:36.583680 17403 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.661921 12777 raft_consensus.cc:2393] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 14: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:36.664928 12528 catalog_manager.cc:5697] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
W20260709 13:19:36.667651 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:36.652387 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:36.717800 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:36.649190 17409 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:36.805925 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:36.869036 12782 raft_consensus.cc:2368] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 13. Current term is 14.
W20260709 13:19:36.869536 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:36.870407 12628 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:36.854382 12747 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:36.856051 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:36.621280 17179 consensus_queue.cc:1048] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:19:36.857987 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:36.876027 12625 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:19:36.869277 12702 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:19:36.706924 17271 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:36.876595 13381 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:36.858574 17413 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:36.627352 17341 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.639170 17325 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:36.632501 17321 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.639122 17416 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:36.639483 17130 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:36.858263 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:36.798159 17013 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:36.896041 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:36.872422 12528 catalog_manager.cc:5697] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 6 to 8, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 8 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:19:36.895115 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 250083us
I20260709 13:19:36.875663 16994 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:19:36.891261 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 271512us
I20260709 13:19:36.907683 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:36.908351 12628 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:36.883550 17209 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:19:36.905493 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 47284.2us
I20260709 13:19:36.914637 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:36.883678 17325 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:36.894433 17413 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:36.917906 12780 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:19:36.932355 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:36.933012 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:36.945015 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:36.950276 12626 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:19:36.931006 12528 catalog_manager.cc:5697] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:36.957098 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:36.958096 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 320326us
I20260709 13:19:36.919258 17325 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:36.959156 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 104882us
I20260709 13:19:36.961468 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:36.969508 12528 catalog_manager.cc:5697] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:36.978468 12628 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:19:36.978734 12528 catalog_manager.cc:5697] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 7 to 12, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 12 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:36.954689 17322 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:36.986038 12528 catalog_manager.cc:5697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 10 to 12, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 12 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:36.988838 17421 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:36.989418 17421 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:36.989846 17421 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:36.990070 17130 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:36.989113 12528 catalog_manager.cc:5697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 9 to 12, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 12 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:36.991855 12528 catalog_manager.cc:5697] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 9 to 13, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 13 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:36.994616 17086 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.995119 17426 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.995666 17426 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.994849 12528 catalog_manager.cc:5697] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:36.996909 17086 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:36.997573 17322 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:36.986229 12672 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:36.997401 12528 catalog_manager.cc:5697] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 10 to 14, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 14 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:36.971522 17300 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.909605 17416 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.000159 12528 catalog_manager.cc:5697] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:37.013306 12528 catalog_manager.cc:5697] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 8 to 11, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 11 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:19:36.948258 17413 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:36.987965 17403 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:37.000507 17418 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.014724 17013 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:37.012410 17269 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:37.033711 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 72323.9us
W20260709 13:19:37.024260 17410 log.cc:927] Time spent T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.525s	user 0.012s	sys 0.000s
I20260709 13:19:37.035818 12747 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:37.031528 17223 consensus_meta.cc:287] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.660s	user 0.002s	sys 0.010s
I20260709 13:19:37.037725 17427 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:37.020135 17426 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:37.051812 17426 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:37.027612 17130 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:19:37.032007 17137 consensus_meta.cc:287] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.593s	user 0.009s	sys 0.002s
I20260709 13:19:37.031913 17013 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:37.053871 17013 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:37.056916 17344 consensus_meta.cc:287] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.540s	user 0.002s	sys 0.008s
I20260709 13:19:37.045849 17403 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:37.053215 17137 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.057933 17403 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:37.060130 17403 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:37.060271 17137 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:37.032567 17418 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.061627 12781 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:37.062870 17418 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:37.061398 17427 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.039018 17413 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:37.040277 17425 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:37.021035 17306 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:37.023342 17136 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:37.043533 17321 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:37.027529 17330 consensus_meta.cc:287] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.652s	user 0.009s	sys 0.009s
I20260709 13:19:37.048430 12672 leader_election.cc:400] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14.
I20260709 13:19:37.074242 12672 leader_election.cc:403] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:37.046571 17223 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.071026 12707 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:19:37.075366 17409 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:37.075539 12707 raft_consensus.cc:606] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:37.048594 17420 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:37.076086 12707 raft_consensus.cc:614] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:37.049659 17416 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:37.015450 17300 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:37.050096 12747 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:37.050339 17322 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:37.027235 17086 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:37.063023 17419 log.cc:927] Time spent T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:19:37.078157 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:37.084239 17321 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.084607 12747 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:37.084816 17321 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.085644 17086 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.077538 17330 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.089628 17086 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:37.089934 17136 consensus_queue.cc:1048] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:37.090262 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:37.090418 17330 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:37.091112 12747 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:37.091269 17086 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:37.092141 17322 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:37.092243 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:19:37.092877 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 58329.1us
I20260709 13:19:37.092865 17409 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:37.108148 17409 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:37.108644 17409 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:37.093112 17416 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:37.093086 17341 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:37.095407 17412 raft_consensus.cc:3055] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:37.081348 17420 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:37.075985 17306 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:37.081926 17425 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:37.096822 17314 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14.
I20260709 13:19:37.098016 17300 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:37.098929 17321 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:37.110620 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:37.147006 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:37.148007 12782 raft_consensus.cc:2368] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13.
I20260709 13:19:37.109750 17140 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:37.098049 17223 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:37.106535 17322 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:37.135082 17137 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:19:37.139089 17264 consensus_meta.cc:287] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.522s	user 0.012s	sys 0.000s
I20260709 13:19:37.113816 12597 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:37.155025 12597 leader_election.cc:400] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 13. Current term is 14.
I20260709 13:19:37.155282 12597 leader_election.cc:403] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:37.145287 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:37.092294 17427 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:37.148826 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:37.100639 17086 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:37.148133 12747 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:37.140378 17412 raft_consensus.cc:740] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:37.142800 17013 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.160812 12597 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:37.166476 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:37.168090 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:37.156204 12775 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:37.153000 17322 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:37.164422 12672 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:37.174027 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:37.177116 12630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:19:37.177718 12630 raft_consensus.cc:606] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Received request to transfer leadership to TS 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:37.178032 12630 raft_consensus.cc:614] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:37.178213 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:37.158567 17321 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:37.190994 17321 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:37.191643 17321 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:37.155874 17436 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:37.195462 17412 consensus_queue.cc:260] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.196130 17306 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:37.197114 17412 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:37.160230 17086 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:37.158720 17137 raft_consensus.cc:697] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:37.182574 17341 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:37.182734 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:37.160496 17316 raft_consensus.cc:3055] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:37.186810 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:37.225430 12625 raft_consensus.cc:2393] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 13: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:37.149847 17421 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:37.188871 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:37.227725 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:37.189751 17414 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:37.152864 17140 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:37.190238 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:37.233004 12631 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:19:37.200001 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:37.200738 17300 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:19:37.205080 17431 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:37.205595 17436 raft_consensus.cc:697] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:37.158586 17264 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:37.180272 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:37.181070 17427 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:37.207746 12703 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:19:37.180634 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:37.180331 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:37.215049 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:37.245029 12632 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:37.218717 17418 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:37.220746 17130 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.187908 17325 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.228744 12625 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
W20260709 13:19:37.229790 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 65487.1us
I20260709 13:19:37.226306 17316 raft_consensus.cc:740] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:37.235980 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:37.245353 12706 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:37.256799 17130 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.257530 17421 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:37.271521 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:37.272267 12706 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:37.232739 17414 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:37.264855 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:37.258967 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:37.263347 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:37.281047 12704 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:37.274418 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:37.266053 17137 consensus_queue.cc:237] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.289093 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:37.291327 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:37.266199 17130 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:37.292209 12707 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:19:37.262636 17316 consensus_queue.cc:260] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:37.293762 17316 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:37.293812 17269 consensus_queue.cc:1059] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:37.296931 17128 raft_consensus.cc:3055] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:37.297458 17128 raft_consensus.cc:740] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:37.298048 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:37.298663 17128 consensus_queue.cc:260] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.300989 17128 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:37.297003 12672 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:37.303067 17264 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:37.303087 12672 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:37.306413 17329 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:37.306892 17329 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:37.307288 17329 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:37.268860 17322 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.309973 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:37.310330 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:37.311283 17322 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:37.301440 12630 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:37.291699 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:37.320901 12631 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:19:37.287410 12673 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:37.326597 12704 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:37.328373 12706 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:19:37.254907 17431 raft_consensus.cc:697] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:37.257634 17140 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:37.286772 17414 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:37.370775 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 83718us
I20260709 13:19:37.409775 12673 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:37.371343 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:37.376727 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:37.380429 12700 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:19:37.415742 12632 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
W20260709 13:19:37.378854 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 458477us
I20260709 13:19:37.241303 17422 consensus_queue.cc:1059] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 13:19:37.329137 17348 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:37.420794 12597 leader_election.cc:400] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13.
I20260709 13:19:37.421155 12597 leader_election.cc:403] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:19:37.411739 12708 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:19:37.421840 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:37.422765 12704 raft_consensus.cc:2410] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:19:37.257727 17427 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.426995 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:37.429517 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:37.430403 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:37.374763 17288 raft_consensus.cc:3055] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:37.431226 12632 raft_consensus.cc:2410] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:19:37.431458 12709 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7.
I20260709 13:19:37.324476 17421 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:37.286126 17147 consensus_queue.cc:1048] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.126s
I20260709 13:19:37.433877 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:37.436801 12597 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:37.442394 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:37.442937 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:37.411399 17321 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.378674 17130 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:37.453079 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:37.453509 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:37.447263 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:37.453177 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:37.275081 17418 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:37.453509 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:37.451092 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.437502 17288 raft_consensus.cc:740] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:37.474464 12709 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:19:37.453475 12775 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:19:37.474241 12704 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:19:37.474241 12703 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:19:37.492928 12781 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:37.494086 12782 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
W20260709 13:19:37.503001 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.503202 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:37.503618 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:37.504698 12709 raft_consensus.cc:2393] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:37.504599 12776 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:37.505123 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:37.445600 17209 raft_consensus.cc:697] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:37.506494 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:37.506990 12702 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:37.507053 12626 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:37.425416 17314 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:37.446184 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:37.453231 17271 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13.
I20260709 13:19:37.516562 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:37.515008 12629 raft_consensus.cc:2393] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 16: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:37.472879 17130 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:37.515003 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:37.524349 12777 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:37.480284 17234 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.532049 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:37.532896 17234 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:37.530678 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:37.533949 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:37.530969 17344 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.534915 17344 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.535974 17234 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:37.537675 17314 raft_consensus.cc:697] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:37.531625 17130 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:37.538657 17314 consensus_queue.cc:237] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.532176 12779 raft_consensus.cc:2393] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 12: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:37.531661 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:37.483052 17307 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:37.528635 17430 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:37.459218 17421 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:37.457645 17306 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:19:37.556442 12747 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:37.537297 17344 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:37.537312 17418 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:37.595641 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:37.602632 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:37.597707 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 160958us
I20260709 13:19:37.553825 17430 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:37.583221 17408 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.098s	user 0.005s	sys 0.032s
I20260709 13:19:37.614029 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:37.562072 17413 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:37.587253 17330 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:19:37.623567 13349 leader_election.cc:336] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.581s (SENT)
I20260709 13:19:37.625487 13349 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:37.612535 17408 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.132s	user 0.005s	sys 0.033s
I20260709 13:19:37.483515 17436 consensus_queue.cc:237] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.559250 17307 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:37.629725 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:37.600109 17264 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:37.643157 17264 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:37.645335 17264 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:37.612669 17430 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:37.637532 12783 raft_consensus.cc:1240] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:37.631508 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:37.619060 17322 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:37.562220 17086 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.630777 17408 raft_consensus.cc:697] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:37.626583 17418 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:37.625391 17330 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:37.659154 12709 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:19:37.604300 17136 consensus_queue.cc:1059] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:19:37.641752 17137 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:37.668704 17324 consensus_queue.cc:1059] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:37.670081 17324 raft_consensus.cc:3055] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:37.670502 17324 raft_consensus.cc:740] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:37.671501 17324 consensus_queue.cc:260] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.673575 17324 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:37.659530 17322 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:37.628803 17413 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:37.697512 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 73964.7us
I20260709 13:19:37.667677 17330 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:37.706765 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.600329 17431 consensus_queue.cc:237] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.620855 17288 consensus_queue.cc:260] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:37.717213 17329 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:37.720669 17408 consensus_queue.cc:237] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:37.721886 17409 consensus_meta.cc:287] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.613s	user 0.009s	sys 0.004s
I20260709 13:19:37.683128 17418 raft_consensus.cc:697] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:19:37.725399 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 111495us
I20260709 13:19:37.691390 17322 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:37.717685 17306 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:37.647828 17209 consensus_queue.cc:237] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:37.764060 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:37.688246 17137 raft_consensus.cc:697] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:37.769035 17329 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:37.769280 17013 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:37.769688 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:37.770061 12706 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 12: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:37.705641 17417 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:37.756079 17427 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:37.785619 17223 raft_consensus.cc:3055] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:37.781983 12632 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
W20260709 13:19:37.788885 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:37.810420 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 46461.1us
I20260709 13:19:37.762707 17321 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:37.818498 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 120236us
I20260709 13:19:37.823091 12747 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:37.781672 17288 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:37.831863 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:37.834544 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:37.794445 17179 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.765535 17409 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.843421 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:37.803802 17223 raft_consensus.cc:740] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:37.823745 17013 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:19:37.856930 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 254911us
I20260709 13:19:37.855597 12775 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:19:37.855664 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:37.854149 12747 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:37.858968 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:37.863004 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:37.864045 12707 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:37.868043 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.832978 17421 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:37.859210 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:37.876659 12783 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:19:37.870203 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:37.877521 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.830489 17427 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:37.871724 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:37.878371 12708 raft_consensus.cc:2410] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:19:37.841110 17416 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:37.888593 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:37.840003 17425 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:37.803431 17417 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:37.848706 17329 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.884919 12705 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:19:37.881278 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:37.892270 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:37.896394 12705 raft_consensus.cc:606] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:37.893553 13383 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:37.867534 17013 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:37.894323 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:37.886833 12706 consensus_queue.cc:237] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:37.853237 12672 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:37.897822 12781 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:19:37.901167 17417 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:37.902206 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:37.875593 17455 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:37.885438 17307 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:37.904830 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:37.888396 17300 consensus_meta.cc:287] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.637s	user 0.010s	sys 0.002s
W20260709 13:19:37.896207 17447 log.cc:927] Time spent T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.297s	user 0.002s	sys 0.000s
I20260709 13:19:37.840864 17414 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.909901 13376 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:37.911085 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:37.905017 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:37.916062 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 62979.4us
I20260709 13:19:37.851689 17179 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:37.915146 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 54292.9us
I20260709 13:19:37.931751 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:37.936911 12706 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:37.936501 13348 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:37.906961 17013 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:37.942026 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:37.939247 12632 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:19:37.945822 12707 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:37.945505 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:37.946519 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:37.953219 13382 raft_consensus.cc:2393] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:37.953159 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:37.951126 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:37.955466 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:37.956084 13384 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:37.941322 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:37.955960 12705 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:37.917977 17300 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 13. Current term is 14.
I20260709 13:19:37.914271 17086 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:37.959362 12747 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:37.960858 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:37.959707 12628 raft_consensus.cc:2393] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:37.961508 12626 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:19:37.981050 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:37.983532 17456 log.cc:927] Time spent T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:19:37.909456 17329 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:37.984058 17461 log.cc:927] Time spent T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.136s	user 0.002s	sys 0.000s
I20260709 13:19:37.985062 17457 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:37.985466 17457 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:37.985847 17457 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:37.986114 17329 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:37.908167 17223 consensus_queue.cc:260] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.986670 17322 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:37.987238 17223 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:37.989760 17416 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:37.990268 17416 raft_consensus.cc:697] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:37.991503 12746 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:37.962944 12702 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
W20260709 13:19:37.992110 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.992889 12702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94e3333498524f14a17661cdb38ef144"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:19:37.932596 17464 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:37.935824 17321 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.999804 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:37.936290 17455 raft_consensus.cc:697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:38.000422 13382 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:37.995496 12747 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:38.001585 12673 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:38.011988 17440 log.cc:927] Time spent T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 13:19:38.012843 17449 log.cc:927] Time spent T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.180s	user 0.001s	sys 0.001s
W20260709 13:19:38.016418 17451 log.cc:927] Time spent T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 13:19:38.020968 13384 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:38.013070 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:38.024143 12633 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:37.905721 17418 consensus_queue.cc:237] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:38.027061 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:38.024631 13377 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:37.989423 17322 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:37.955340 17417 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:38.034116 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:37.947362 17307 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 13:19:37.964179 17413 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:38.042443 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 47642.9us
I20260709 13:19:38.045990 17436 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:37.914116 17137 consensus_queue.cc:237] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:38.048651 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 107413us
I20260709 13:19:38.048558 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:38.028002 17329 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:19:38.034163 17128 consensus_meta.cc:287] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.733s	user 0.011s	sys 0.005s
I20260709 13:19:38.055037 17325 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:38.036610 17086 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.058357 12672 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:38.019805 17420 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:38.059957 12673 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:38.062296 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:38.040854 17264 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:38.040407 17417 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:37.999001 17234 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 13:19:37.956612 17454 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.092s
I20260709 13:19:38.021812 17321 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:38.004447 17416 consensus_queue.cc:237] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:38.069492 17329 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:38.081710 12672 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:38.067534 17436 raft_consensus.cc:697] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:38.076371 17086 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:38.093492 17414 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:38.096167 17432 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:38.081843 17179 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:38.097733 17460 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.098402 17460 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:38.100421 17316 consensus_meta.cc:287] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.806s	user 0.011s	sys 0.000s
W20260709 13:19:38.100593 17439 log.cc:927] Time spent T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.652s	user 0.003s	sys 0.007s
I20260709 13:19:38.101732 17325 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:38.102654 17325 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:38.092506 17329 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:38.079105 17417 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:38.121944 17412 consensus_meta.cc:287] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.906s	user 0.013s	sys 0.000s
I20260709 13:19:38.134884 12702 raft_consensus.cc:606] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:38.136871 12702 raft_consensus.cc:614] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:38.135736 12672 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:19:38.135979 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 144554us
I20260709 13:19:38.138199 12708 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:38.084821 17344 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:38.148497 17344 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:38.148880 17344 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:38.128854 17464 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:19:38.127599 17463 log.cc:927] Time spent T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.122s	user 0.003s	sys 0.000s
I20260709 13:19:38.159379 12746 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:38.160581 13382 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:38.163014 12746 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:38.165326 12704 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 11. Current term is 12. Ops: [11.12-11.12]
I20260709 13:19:38.167642 12704 raft_consensus.cc:1240] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:38.167994 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:38.168867 12705 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:38.105522 17325 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:38.155089 17130 consensus_meta.cc:287] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.616s	user 0.009s	sys 0.004s
I20260709 13:19:38.149550 17086 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:38.159879 17464 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:38.169298 17136 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:38.199889 12672 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:38.201711 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:38.189960 17420 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:38.155628 17469 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:38.198859 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:38.202718 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:38.217592 12672 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:38.170749 17473 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:38.221135 17348 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.017s	sys 0.051s
I20260709 13:19:38.222167 12707 raft_consensus.cc:2376] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 7. Re-sending same reply.
I20260709 13:19:38.223026 12703 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:19:38.219861 17412 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:19:38.227319 17348 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.017s	sys 0.051s
W20260709 13:19:38.274487 17324 consensus_meta.cc:287] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.600s	user 0.012s	sys 0.000s
I20260709 13:19:38.277781 17321 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:38.223026 17413 consensus_queue.cc:1059] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:19:38.253408 17473 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:38.267088 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.155854 17436 consensus_queue.cc:237] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:38.154865 17455 consensus_queue.cc:237] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:38.287014 17430 consensus_meta.cc:287] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.624s	user 0.012s	sys 0.000s
I20260709 13:19:38.287820 17430 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:38.260346 17341 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:38.290247 17473 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:38.292855 17013 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:38.294858 17344 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:38.296712 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:38.297592 17409 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:38.302316 17329 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:38.304145 13378 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:38.304796 17457 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:38.356215 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.197786 17130 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:38.308501 12746 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:38.315001 12629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56838 (request call id 1766) took 1147 ms. Trace:
I20260709 13:19:38.315124 17264 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:38.315987 12705 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:19:38.313391 12709 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
W20260709 13:19:38.327490 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 278664us
I20260709 13:19:38.316062 17460 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:38.306625 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.361529 12629 rpcz_store.cc:276] 0709 13:19:37.167215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.167335 (+   120us) service_pool.cc:224] Handling call
0709 13:19:38.314750 (+1147415us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b54002eb8d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:38.314890 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":661591936}
W20260709 13:19:38.366938 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 58582.7us
I20260709 13:19:38.368639 12746 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:38.368995 17476 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:38.371430 17474 log.cc:927] Time spent T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb: Append to log took a long time: real 0.053s	user 0.000s	sys 0.003s
I20260709 13:19:38.373080 17457 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:38.358403 17330 consensus_meta.cc:287] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.618s	user 0.012s	sys 0.002s
I20260709 13:19:38.375054 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:38.392910 17344 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:38.397097 17130 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:38.385982 17322 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:38.399237 17330 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:38.402634 12779 raft_consensus.cc:1240] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:38.405035 17330 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:38.405649 12747 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:38.406430 17264 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:19:38.406975 17478 log.cc:927] Time spent T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:19:38.406996 17322 raft_consensus.cc:697] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:38.410796 13349 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:19:38.412353 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 194860us
I20260709 13:19:38.417972 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:38.422053 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.422902 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:38.425793 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:38.426620 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.407815 17420 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:38.428458 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:38.431109 12702 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:38.436693 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:38.437995 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:38.441799 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:38.442092 13349 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:38.442431 12703 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:38.441484 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:38.445376 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:38.446198 12700 raft_consensus.cc:2393] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 16: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:38.420295 17288 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:38.447489 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:38.449139 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:38.451052 12704 raft_consensus.cc:2393] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 16: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:38.430152 17436 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:38.389500 17476 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:38.452560 17489 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:38.452503 17322 consensus_queue.cc:237] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:38.456619 17466 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:19:38.450096 17445 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:38.458091 17445 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:38.454794 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:38.462594 12705 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:38.441382 17307 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:38.478446 12702 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:38.497690 17487 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:38.497931 12707 raft_consensus.cc:1275] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:19:38.498405 17445 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:38.497748 17476 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:38.500993 17427 consensus_queue.cc:1048] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:38.500916 12705 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
W20260709 13:19:38.507269 17477 log.cc:927] Time spent T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:19:38.507982 12747 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:38.510579 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:38.509442 17329 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:38.512739 17475 log.cc:927] Time spent T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.152s	user 0.001s	sys 0.001s
I20260709 13:19:38.513192 17444 raft_consensus.cc:3055] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:38.513748 17444 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:38.515159 17444 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:38.516573 12633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56838 (request call id 1763) took 1350 ms. Trace:
I20260709 13:19:38.516891 17444 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:38.517002 12633 rpcz_store.cc:276] 0709 13:19:37.166383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.173786 (+  7403us) service_pool.cc:224] Handling call
0709 13:19:38.516385 (+1342599us) spinlock_profiling.cc:243] Waited 839 ms on lock 0x7b54000b7f54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:38.516513 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1928700160}
I20260709 13:19:38.509409 17436 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:38.518091 17436 raft_consensus.cc:697] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:38.518981 17436 consensus_queue.cc:237] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:38.522310 17013 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:38.467084 17430 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:38.524307 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:38.525243 12702 raft_consensus.cc:2393] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:38.526521 17417 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:38.534802 17417 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:38.541486 17314 raft_consensus.cc:3055] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:38.542168 17314 raft_consensus.cc:740] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:38.543104 17314 consensus_queue.cc:260] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:38.544770 17314 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:38.556393 12701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1856) took 1325 ms. Trace:
I20260709 13:19:38.565021 12704 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:38.567651 12700 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:38.572254 17444 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:38.573936 17425 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:38.574661 17425 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:38.580951 12780 raft_consensus.cc:1240] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:19:38.581664 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:38.582437 12778 raft_consensus.cc:2393] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 10: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:38.582461 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:38.583354 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:38.584025 12781 raft_consensus.cc:2393] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 12: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:38.584797 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:38.585875 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:38.556864 12701 rpcz_store.cc:276] 0709 13:19:37.230688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.230983 (+   295us) service_pool.cc:224] Handling call
0709 13:19:37.241837 (+ 10854us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:37.253351 (+ 11514us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:19:37.253646 (+   295us) log.cc:844] Serialized 14 byte log entry
0709 13:19:37.276415 (+ 22769us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:37.276492 (+    77us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:37.276555 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.388144 (+1111589us) spinlock_profiling.cc:243] Waited 8.61 ms on lock 0x7b18002ae9e0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2932a6 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419
0709 13:19:38.555990 (+167846us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b54003b33d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:38.556043 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:38.556304 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22442,"prepare.queue_time_us":407,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":22586,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":26099,"spinlock_wait_cycles":45705344}
I20260709 13:19:38.586405 12775 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:38.589332 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:38.590163 12776 raft_consensus.cc:2393] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 11: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:38.590304 12777 raft_consensus.cc:1240] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:38.593353 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:38.594321 12633 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:38.594520 12631 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:38.596522 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:38.597247 12627 raft_consensus.cc:1240] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:38.597775 12628 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:38.577365 17425 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:38.607936 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:38.612896 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:38.613708 12783 raft_consensus.cc:2393] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 16: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:38.612198 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:38.615877 12781 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:38.619453 17491 log.cc:927] Time spent T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.160s	user 0.003s	sys 0.001s
I20260709 13:19:38.622651 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:38.629079 17488 log.cc:927] Time spent T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 13:19:38.629946 17445 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:38.631171 17445 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:38.632259 12626 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d46747dede234934bea0f0c324106c28"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:19:38.633087 12626 raft_consensus.cc:606] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:38.635697 17445 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:38.639001 12779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 2393) took 1034 ms. Trace:
I20260709 13:19:38.639638 12779 rpcz_store.cc:276] 0709 13:19:37.604701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.604833 (+   132us) service_pool.cc:224] Handling call
0709 13:19:37.612827 (+  7994us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:38.403326 (+790499us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.638735 (+235409us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b54002cbd54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:38.638937 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1815944064}
I20260709 13:19:38.641955 12597 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:38.643093 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:38.643795 12626 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:38.645269 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:38.646024 12627 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:38.647254 12626 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:38.646157 17430 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:38.648427 17430 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:38.648769 17430 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:38.635883 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:38.658644 17330 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:38.659183 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:38.659426 17330 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:38.660321 12630 raft_consensus.cc:2368] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13.
I20260709 13:19:38.660844 17455 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:38.660159 17330 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:38.661537 17455 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:38.663972 17455 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:38.665277 17348 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.665858 17348 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:38.668277 17348 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:38.670699 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:19:38.656672 12627 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:38.672896 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:38.673942 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:38.658496 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:38.677487 12634 raft_consensus.cc:2393] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 16: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:38.678791 17271 consensus_queue.cc:1059] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:38.679996 17321 raft_consensus.cc:3055] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:38.680455 17321 raft_consensus.cc:740] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:38.680902 12673 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:38.681277 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:38.682138 12704 raft_consensus.cc:2393] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 15: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:38.681550 17321 consensus_queue.cc:260] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:38.683174 17147 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:38.683611 17147 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:38.684028 12673 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:38.683540 17321 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:38.685678 17140 consensus_queue.cc:1059] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:38.676539 17330 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:38.689110 17269 raft_consensus.cc:3055] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:38.689581 17269 raft_consensus.cc:740] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:38.691129 17269 consensus_queue.cc:260] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:38.692430 17269 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:38.684075 17147 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:38.696257 12781 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:38.689288 17330 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:38.699018 12775 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
W20260709 13:19:38.700352 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:38.700876 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 258853us
I20260709 13:19:38.702142 12628 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:38.702559 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:38.707937 12597 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:38.699205 17330 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:38.706461 12672 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:38.757850 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 249069us
I20260709 13:19:38.758103 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:38.761878 12746 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:38.779522 12673 leader_election.cc:400] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13.
I20260709 13:19:38.779172 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:38.780277 12673 leader_election.cc:403] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:19:38.784009 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:38.793236 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:38.794309 12775 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:38.797689 12673 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:38.764482 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 58258.6us
I20260709 13:19:38.800944 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:38.803757 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:38.811720 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:38.744690 17454 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:38.753621 17324 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.756358 17445 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:38.809427 17140 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:38.742978 17307 raft_consensus.cc:697] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:38.776360 17128 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:38.842319 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:38.839674 12704 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
W20260709 13:19:38.849433 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 474874us
I20260709 13:19:38.704470 17137 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.111s
I20260709 13:19:38.810381 17487 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:19:38.936865 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 133225us
I20260709 13:19:38.940797 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:38.942642 12781 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:38.844849 12775 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
I20260709 13:19:38.882750 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:38.845948 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:19:38.845657 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.944298 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:38.777863 17457 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 13:19:38.947257 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:38.949647 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:38.888195 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.952631 17223 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:38.953966 17487 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:38.954494 17487 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:38.958104 17307 consensus_queue.cc:237] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:38.960783 12709 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:38.961143 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:38.840526 17454 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:38.964323 12779 raft_consensus.cc:2393] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 16: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:38.781883 17330 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:38.966553 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:38.839152 17492 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:38.969398 12626 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:38.973399 12705 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:19:38.818562 17430 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:38.886009 17445 raft_consensus.cc:697] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:38.979565 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:38.845245 17460 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.985450 12708 raft_consensus.cc:2376] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13. Re-sending same reply.
I20260709 13:19:38.857064 17324 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:38.967545 17454 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:39.005992 12627 raft_consensus.cc:2393] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 13: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:19:39.000434 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.014456 12779 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:39.019569 12781 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:38.986665 17492 raft_consensus.cc:697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:39.003536 17469 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:39.027209 12781 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:39.036993 17430 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:39.040326 17430 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:39.046516 17494 log.cc:927] Time spent T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:19:39.045763 17445 consensus_queue.cc:237] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:39.012406 17147 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:39.053076 17147 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:39.053794 12706 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:19:39.044728 17324 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:39.065507 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:39.068789 17128 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.075659 17128 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:39.079015 17128 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:39.074538 12774 raft_consensus.cc:2368] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13.
W20260709 13:19:39.084954 17493 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 13:19:39.091200 17140 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13.
I20260709 13:19:39.003880 17460 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:39.041041 17430 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:39.130482 17492 consensus_queue.cc:237] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:39.148098 17460 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:39.057796 17469 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:39.163146 12672 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:39.187008 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 235821us
W20260709 13:19:39.171357 17314 consensus_meta.cc:287] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.626s	user 0.014s	sys 0.001s
I20260709 13:19:39.171970 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:39.172202 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:39.173214 17487 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:39.190928 13381 raft_consensus.cc:2393] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 12: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:39.174621 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:39.175799 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:39.191777 13378 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:39.191931 13383 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:39.176092 13379 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:39.176323 13376 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:39.183513 13349 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:39.184741 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.190824 17140 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:39.189306 17324 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:39.198235 17324 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:39.200629 17324 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:39.206416 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.215937 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:39.216606 12630 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:39.216918 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:39.217271 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:39.217691 12633 raft_consensus.cc:2393] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:39.217309 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:39.219182 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:39.220184 12625 raft_consensus.cc:2393] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 16: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:39.220932 12704 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:19:39.221743 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:39.222697 12625 raft_consensus.cc:2393] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 10: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:39.222642 13383 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:39.225154 12633 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:39.226800 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:39.225945 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:39.237051 12631 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:19:39.238303 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:39.242013 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:39.244313 12629 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:39.214015 17140 raft_consensus.cc:697] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:39.245716 17140 consensus_queue.cc:237] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:39.250825 12634 raft_consensus.cc:2368] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15.
W20260709 13:19:39.230116 17321 consensus_meta.cc:287] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.545s	user 0.013s	sys 0.000s
I20260709 13:19:39.226399 17324 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:39.267715 12627 raft_consensus.cc:1275] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:19:39.228405 17446 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:19:39.270499 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.269402 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:39.273154 13348 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:19:39.274070 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 48193.4us
I20260709 13:19:39.275017 13349 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:39.281823 12704 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:19:39.281960 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:39.275767 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:39.283520 12775 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:39.277554 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:39.269675 17460 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:39.296624 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:39.290231 12631 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:39.292857 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:39.287781 17469 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:39.289911 17457 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.294277 12704 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:19:39.298280 12627 raft_consensus.cc:2393] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:39.304134 13349 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:39.309682 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:39.305496 12629 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:39.313932 12672 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:39.322202 12630 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:39.294855 17274 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:19:39.331104 12597 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.047s	sys 0.057s
I20260709 13:19:39.312422 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:39.333071 12597 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.047s	sys 0.057s
W20260709 13:19:39.336169 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 537909us
I20260709 13:19:39.338798 13383 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:19:39.293440 17430 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:39.346889 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:39.300103 17486 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:39.337404 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:39.353094 13381 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:39.332155 17321 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.362469 17321 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:39.364863 17321 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:39.327054 17446 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:39.367623 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:39.273034 17324 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:39.330612 17460 raft_consensus.cc:697] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 7 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:39.345045 17507 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.373757 12746 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:39.344507 17469 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.310617 17457 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:39.351006 17432 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:39.318132 17269 consensus_queue.cc:1059] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
W20260709 13:19:39.382583 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 68756.3us
I20260709 13:19:39.373584 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:39.395076 13383 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:39.384924 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:39.397866 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:39.398049 12632 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:39.380165 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:39.357091 17486 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:39.388057 12673 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:39.391552 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:39.400988 12633 raft_consensus.cc:2368] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13.
I20260709 13:19:39.344376 17436 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:19:39.394357 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:39.409291 12632 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:19:39.400143 17507 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:39.379329 17484 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:39.391552 17444 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:39.425933 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:39.437287 12630 raft_consensus.cc:2393] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 15: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:39.406425 17486 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:39.445432 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:39.446399 12630 raft_consensus.cc:2393] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:39.418784 17128 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:39.446841 13383 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:39.418385 17430 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:19:39.450598 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 312085us
I20260709 13:19:39.421535 17511 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:39.412283 17422 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:19:39.452630 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:39.436383 17444 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:39.459932 12630 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:39.404567 17469 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:39.453068 17432 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.451242 17128 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.450274 17484 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:39.461694 12598 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:39.455869 17480 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:39.500691 12598 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:39.498148 17432 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:39.457041 17269 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:19:39.483741 17444 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:39.491571 17128 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:39.501765 17455 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:39.481904 17430 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:39.609993 17505 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:39.611277 17509 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:39.611856 17509 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:39.613325 17430 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:39.607035 17508 log.cc:927] Time spent T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.202s	user 0.000s	sys 0.003s
I20260709 13:19:39.508774 17484 raft_consensus.cc:697] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:39.472201 17511 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:39.502233 17415 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.100s
I20260709 13:19:39.586203 17457 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:39.637754 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 325399us
I20260709 13:19:39.618052 17505 raft_consensus.cc:2764] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:19:39.520448 17480 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:39.643957 13349 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:19:39.649487 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 229143us
I20260709 13:19:39.630191 17506 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:39.653842 12634 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:39.498883 17460 consensus_queue.cc:237] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:39.677994 13379 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
W20260709 13:19:39.686515 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 190718us
I20260709 13:19:39.654460 17509 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:39.690083 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.664399 12673 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:39.662966 17469 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:39.662220 17507 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:39.655397 17511 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:39.661769 17324 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:39.680570 17274 raft_consensus.cc:697] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:39.704950 12673 leader_election.cc:400] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13.
I20260709 13:19:39.729699 12673 leader_election.cc:403] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:19:39.726056 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:19:39.743851 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 47002.9us
W20260709 13:19:39.683610 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 253402us
I20260709 13:19:39.695415 17321 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:39.745381 12672 leader_election.cc:400] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13.
I20260709 13:19:39.654336 17484 consensus_queue.cc:237] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:39.750985 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 250331us
I20260709 13:19:39.755653 12598 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:39.727497 17527 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:19:39.758790 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.759181 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:39.759843 12634 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:39.734977 17530 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:39.748992 17457 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:39.750550 17324 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:39.770437 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:39.770490 12673 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:39.771158 12628 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:19:39.685587 17506 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:39.759182 17527 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.773698 17527 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:39.774962 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 131122us
I20260709 13:19:39.776896 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:39.800518 17469 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.801311 17526 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:39.768424 17455 raft_consensus.cc:697] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:39.802024 17526 raft_consensus.cc:697] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 12 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:39.802536 17455 consensus_queue.cc:237] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:39.803687 17329 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:39.803285 17526 consensus_queue.cc:237] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:39.804237 17329 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.804639 17329 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:39.801167 17469 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:39.748939 17321 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:39.829452 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.812511 12598 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:39.803638 17514 log.cc:927] Time spent T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.391s	user 0.002s	sys 0.000s
I20260709 13:19:39.806981 17469 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:39.769255 17530 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:39.828320 17524 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:19:39.831336 12778 raft_consensus.cc:1240] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:19:39.768596 17511 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:39.838541 12701 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:19:39.840360 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:39.886294 12704 raft_consensus.cc:2393] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:39.841630 12707 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:39.866533 12597 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:39.766153 17415 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:19:39.801327 17147 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:39.870904 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 100132us
I20260709 13:19:39.889849 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:39.890982 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 54064.4us
I20260709 13:19:39.872828 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:39.774307 17324 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:39.905164 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:39.872931 17507 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:39.908195 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:39.908918 12777 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:39.909276 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:39.910667 12598 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:39.919456 17330 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:39.913519 17223 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:39.896459 17274 consensus_queue.cc:237] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:39.925868 12597 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:39.927218 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:39.930820 12598 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:39.931703 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:39.903587 17147 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:39.929876 12673 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:39.947690 17147 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:39.891723 17445 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:19:39.975147 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 52565.5us
I20260709 13:19:39.957721 12598 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:39.977226 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:39.916056 17523 log.cc:927] Time spent T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
W20260709 13:19:39.928578 17522 log.cc:927] Time spent T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.125s	user 0.000s	sys 0.002s
I20260709 13:19:39.903574 17511 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:39.913378 17324 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:40.001120 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:40.006616 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 230686us
W20260709 13:19:40.007764 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 50188.2us
I20260709 13:19:40.008925 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:40.011298 12628 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 6.
I20260709 13:19:39.921509 17223 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:40.013048 12598 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:39.976351 17533 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:39.953192 17530 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:40.024108 12598 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:40.024222 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 179910us
I20260709 13:19:39.983557 17528 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:39.979058 17430 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:39.998911 17330 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:39.980142 17480 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:39.954109 17509 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:40.040729 12673 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:39.949505 17128 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:40.025107 17445 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:40.026579 17533 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:40.031388 17324 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:40.068548 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:40.038961 17528 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.002107 17329 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:40.070675 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:40.068869 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:40.056581 12747 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.65:43603 timed out after 1.636s (SENT) [suppressed 12 similar messages]
I20260709 13:19:40.038025 17469 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.020169 17527 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:40.086805 17525 log.cc:927] Time spent T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:19:40.095595 12630 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:19:40.100044 12633 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:19:40.102370 17086 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:40.102945 17086 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:40.100551 17542 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:40.106115 17147 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:40.106539 17147 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.106889 17147 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:40.087818 17533 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:40.117697 12706 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:40.058732 17543 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:40.118618 17543 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:40.119052 17543 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:40.119743 17137 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:40.122661 17086 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:40.121174 17137 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:40.054138 17330 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:40.129873 17330 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:40.131227 17128 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:40.131999 17552 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:40.094694 17528 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:40.133093 12705 raft_consensus.cc:1240] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:19:40.134783 12597 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:19:40.183595 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 46748.8us
W20260709 13:19:40.184444 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 49745.1us
I20260709 13:19:40.184831 12746 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:40.120113 17454 consensus_meta.cc:287] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 1.109s	user 0.008s	sys 0.005s
W20260709 13:19:40.109956 17432 consensus_meta.cc:287] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.524s	user 0.012s	sys 0.000s
I20260709 13:19:40.109915 17415 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.195377 17487 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:40.210250 17550 log.cc:927] Time spent T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:19:40.196198 17454 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:40.213773 17086 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:40.214416 17086 raft_consensus.cc:697] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:19:40.218494 17535 log.cc:927] Time spent T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:19:40.109274 17469 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:40.229220 12598 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:40.230638 12598 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:40.190119 17552 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:40.231830 17445 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:40.232643 17445 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:40.233054 17552 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:40.201655 17415 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:40.105310 17542 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:40.235795 17415 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:40.236224 17321 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:40.236939 17542 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:40.238498 17444 consensus_meta.cc:287] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.610s	user 0.014s	sys 0.000s
I20260709 13:19:40.238957 17444 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:40.243567 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:40.243806 12598 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:40.247478 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:19:40.248231 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 50662us
I20260709 13:19:40.197120 17532 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:40.233132 17445 raft_consensus.cc:697] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:40.266016 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:40.266719 12783 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:19:40.267201 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:40.266551 17445 consensus_queue.cc:237] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:40.268041 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:40.268937 17537 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:40.269374 17537 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.269685 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:40.270349 17421 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:40.262130 17469 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:40.272279 17408 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:40.272934 17408 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:40.275939 17457 consensus_meta.cc:287] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.503s	user 0.011s	sys 0.000s
I20260709 13:19:40.277715 17415 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:40.281117 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:40.296136 12780 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:19:40.297366 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:40.298305 12780 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:40.281921 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:40.299912 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:40.284279 12597 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:40.269767 17537 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:40.289192 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 248549us
I20260709 13:19:40.215323 17086 consensus_queue.cc:237] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:40.294158 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:40.253651 17509 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:40.319898 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:40.309237 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:40.309865 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:40.321491 12630 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:19:40.199712 17432 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:40.317236 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:40.318089 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:40.321550 12625 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:40.311738 12781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2741) took 1243 ms. Trace:
I20260709 13:19:40.252815 17444 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:40.328199 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:40.328987 12625 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:40.337077 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:40.339222 12631 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:40.295578 17457 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13.
I20260709 13:19:40.327723 12781 rpcz_store.cc:276] 0709 13:19:39.068850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.069231 (+   381us) service_pool.cc:224] Handling call
0709 13:19:39.069632 (+   401us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:39.094010 (+ 24378us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:39.418514 (+324504us) log.cc:844] Serialized 14 byte log entry
0709 13:19:39.869988 (+451474us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:39.877463 (+  7475us) log.cc:844] Serialized 52 byte log entry
0709 13:19:39.877738 (+   275us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:39.880520 (+  2782us) log.cc:844] Serialized 14 byte log entry
0709 13:19:40.310515 (+429995us) log.cc:844] Serialized 14 byte log entry
0709 13:19:40.311491 (+   976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:40.311543 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:40.311571 (+    28us) raft_consensus.cc:1685] finished
0709 13:19:40.311578 (+     7us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:40.311695 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74056,"prepare.queue_time_us":56720,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":365012,"raft.run_cpu_time_us":589,"raft.run_wall_time_us":74746,"spinlock_wait_cycles":13715200,"thread_start_us":1116626,"threads_started":4,"wal-append.queue_time_us":433627}
W20260709 13:19:40.344847 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 76850.3us
I20260709 13:19:40.292994 17532 raft_consensus.cc:697] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:40.306480 17223 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:40.361697 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:40.362222 13379 raft_consensus.cc:2368] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 14.
I20260709 13:19:40.334213 17330 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:40.342929 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:40.381618 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:40.382261 12631 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:19:40.367286 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:40.318615 17505 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:40.370398 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:40.371100 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:40.325989 17534 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.373260 12625 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:19:40.373512 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:40.336225 17509 raft_consensus.cc:697] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:40.324679 17541 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:40.316658 17415 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:40.339385 12709 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:40.376106 12633 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:40.381553 12702 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:40.374300 12630 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:40.396327 12708 raft_consensus.cc:2368] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12.
I20260709 13:19:40.387334 13385 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:40.378979 17527 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:40.372737 17546 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:40.378896 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:40.386353 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:40.366029 17128 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.402586 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:40.430039 13383 raft_consensus.cc:2410] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 13 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:19:40.401669 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:40.385118 17505 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:40.382162 17487 consensus_queue.cc:1048] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:19:40.429353 17533 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:40.429303 17509 consensus_queue.cc:237] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:40.444553 17533 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:40.397920 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:40.341503 17538 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 13:19:40.421144 17532 consensus_queue.cc:237] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:40.423604 17415 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:40.457973 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:40.425261 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:40.435451 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:40.459422 12631 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:40.403548 17542 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:40.452706 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 51112.5us
I20260709 13:19:40.463850 12633 raft_consensus.cc:2393] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:40.453812 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:40.454309 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:40.461156 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:40.468137 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:40.470919 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:40.466361 12631 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:19:40.473145 12747 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:40.474088 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:40.453151 17505 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:40.447224 17128 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:40.479315 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:40.456012 17554 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:40.480690 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:40.462569 17558 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.403986 17534 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:40.476429 12702 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:40.462453 17564 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:19:40.485261 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 107235us
I20260709 13:19:40.486141 12673 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:40.492290 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:40.492305 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:40.493993 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:40.494719 12633 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:19:40.475276 17536 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:40.495435 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:40.496106 12633 raft_consensus.cc:2393] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 16: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:40.499004 12747 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:40.482772 17486 consensus_meta.cc:287] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 1.031s	user 0.007s	sys 0.008s
I20260709 13:19:40.469904 17322 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:19:40.450119 17546 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:40.511658 12673 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:40.510339 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:40.523327 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:40.487653 17444 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:40.477936 17541 consensus_queue.cc:1059] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:19:40.475821 17415 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:19:40.495332 17469 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:40.529052 12780 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:40.538105 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:40.541731 17541 raft_consensus.cc:3055] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:40.543655 17541 raft_consensus.cc:740] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:40.545056 17541 consensus_queue.cc:260] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:40.546839 17541 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:40.498076 17564 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:40.538219 17469 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:40.558704 17469 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:40.559955 17564 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:40.560452 17534 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:40.518735 17487 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:40.540520 12673 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:40.562412 12709 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:40.482064 17543 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:40.581091 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:40.586275 12702 raft_consensus.cc:2393] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 14: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
W20260709 13:19:40.591691 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 51273.3us
I20260709 13:19:40.590951 13348 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:40.560616 17506 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:40.596778 12673 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:40.597343 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:40.597806 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:40.562131 17223 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:40.491051 17558 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:40.657428 12630 raft_consensus.cc:2376] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12. Re-sending same reply.
I20260709 13:19:40.512188 17486 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:40.659194 12633 raft_consensus.cc:2393] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 16: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:40.675920 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:40.665581 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 68865.7us
I20260709 13:19:40.521763 17314 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:19:40.677242 12630 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:40.538712 17147 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:40.677695 12673 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:40.598261 17577 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:40.696174 13381 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:40.696590 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:40.698748 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:40.698307 12673 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:40.701213 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:40.688014 17510 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:40.729220 17510 raft_consensus.cc:697] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:40.730216 17510 consensus_queue.cc:237] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:40.702143 12708 raft_consensus.cc:2368] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8.
I20260709 13:19:40.699488 17577 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.709765 17584 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:40.712618 17484 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:40.701115 17560 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:40.713712 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:40.714166 17543 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:40.732460 17584 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:40.734793 17584 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:40.736963 17408 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:40.716759 17546 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:40.732852 17577 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:40.693665 17568 log.cc:927] Time spent T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 13:19:40.715687 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:40.767115 12628 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:40.768159 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:40.768944 12633 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:40.686590 17223 raft_consensus.cc:697] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:40.770248 17223 consensus_queue.cc:237] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:40.716699 17486 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:40.722793 17128 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:40.727378 17558 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:40.701974 12630 raft_consensus.cc:2368] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15.
I20260709 13:19:40.728647 17147 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:40.739854 17484 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:40.774719 12673 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:40.875006 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:40.876282 12630 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 6 FOLLOWER]: Advancing to term 8
W20260709 13:19:40.944243 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:40.773026 17528 consensus_meta.cc:287] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.576s	user 0.011s	sys 0.000s
W20260709 13:19:40.957839 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 183214us
I20260709 13:19:40.962423 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:40.963634 12633 raft_consensus.cc:2410] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:19:40.969887 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:40.783838 17564 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.772390 17415 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.834717 17543 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:40.967383 17537 consensus_meta.cc:287] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.632s	user 0.011s	sys 0.000s
I20260709 13:19:40.971864 17564 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:40.955176 17348 raft_consensus.cc:3055] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:40.834702 17560 raft_consensus.cc:697] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:41.002290 12628 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:19:40.970165 17128 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:41.009606 12630 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 8.
I20260709 13:19:40.956501 17484 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:40.954699 17529 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:41.011085 12673 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:41.019430 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:41.021541 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:40.998279 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 401853us
I20260709 13:19:41.022305 12627 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:19:41.023741 12627 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:19:41.024920 12627 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:19:41.009509 17505 consensus_meta.cc:287] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.521s	user 0.011s	sys 0.000s
I20260709 13:19:41.032295 12627 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:19:41.004240 17348 raft_consensus.cc:740] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:41.033284 12673 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:41.038338 12630 raft_consensus.cc:1240] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:40.968081 17528 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:41.040316 12628 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:41.040570 12627 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:41.007736 17128 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.003228 17577 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:41.002696 17543 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:40.998255 17415 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:41.037611 17147 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:41.065433 17147 raft_consensus.cc:697] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:41.066596 17147 consensus_queue.cc:237] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:41.067414 17531 log.cc:927] Time spent T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 1.264s	user 0.000s	sys 0.016s
I20260709 13:19:41.037356 17510 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:41.049691 17486 consensus_queue.cc:1059] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:19:41.059284 17348 consensus_queue.cc:260] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:41.047663 17558 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.048147 17128 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:41.062979 17564 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:41.063745 17561 log.cc:927] Time spent T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.857s	user 0.012s	sys 0.000s
I20260709 13:19:41.070827 12707 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:19:41.003854 17537 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:41.076287 12702 raft_consensus.cc:2368] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13.
I20260709 13:19:41.082908 12629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 3028) took 1696 ms. Trace:
I20260709 13:19:41.081656 12633 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:19:41.083374 12629 rpcz_store.cc:276] 0709 13:19:39.381845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.384613 (+  2768us) service_pool.cc:224] Handling call
0709 13:19:39.385040 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:39.417957 (+ 32917us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:39.417992 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:39.440102 (+ 22110us) log.cc:844] Serialized 26 byte log entry
0709 13:19:39.695136 (+255034us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:39.695170 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:39.695259 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:41.066794 (+1371535us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:41.072506 (+  5712us) raft_consensus.cc:1685] finished
0709 13:19:41.072528 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:41.078109 (+  5581us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32496,"raft.queue_time_us":32877,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":19699,"spinlock_wait_cycles":77423488,"thread_start_us":232623,"threads_started":1,"wal-append.queue_time_us":351175}
I20260709 13:19:41.087720 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:41.036397 17505 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:41.033320 17529 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:41.092408 17510 raft_consensus.cc:697] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:41.093068 17348 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:41.111837 17469 consensus_meta.cc:287] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.553s	user 0.014s	sys 0.000s
I20260709 13:19:41.122706 12703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1910) took 1066 ms. Trace:
I20260709 13:19:41.123533 17537 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:41.126397 17584 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:41.117511 17510 consensus_queue.cc:237] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:41.132192 17529 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:41.080806 17560 consensus_queue.cc:237] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:41.123226 12703 rpcz_store.cc:276] 0709 13:19:40.056913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.057105 (+   192us) service_pool.cc:224] Handling call
0709 13:19:40.058501 (+  1396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:40.074499 (+ 15998us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:40.074529 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:40.080841 (+  6312us) log.cc:844] Serialized 26 byte log entry
0709 13:19:40.103558 (+ 22717us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:40.103607 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:40.103689 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:41.062956 (+959267us) spinlock_profiling.cc:243] Waited 43.5 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:41.117722 (+ 54766us) raft_consensus.cc:1685] finished
0709 13:19:41.117744 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:41.122614 (+  4870us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":47068,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":75,"spinlock_wait_cycles":100062592,"thread_start_us":16445,"threads_started":1,"wal-append.queue_time_us":96311}
I20260709 13:19:41.090498 17558 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:41.152040 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:19:41.152798 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:41.117736 17577 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:41.161681 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 88316.7us
I20260709 13:19:41.143666 17584 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:41.165657 17543 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:41.168037 17536 consensus_meta.cc:287] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.666s	user 0.009s	sys 0.000s
I20260709 13:19:41.168591 17536 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:41.121393 17469 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:41.191733 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:41.195956 17590 log.cc:927] Time spent T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:19:41.197999 17593 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:41.198792 17593 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:41.202302 17528 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:41.204761 17415 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:41.212347 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:41.206975 17558 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:41.213343 13380 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:41.192905 17584 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:41.233827 17584 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:41.236423 17584 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:41.212167 17543 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:41.254012 17505 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:41.244634 17543 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:41.317359 12707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1911) took 1113 ms. Trace:
I20260709 13:19:41.318079 12707 rpcz_store.cc:276] 0709 13:19:40.203413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.203577 (+   164us) service_pool.cc:224] Handling call
0709 13:19:40.203918 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:41.082971 (+879053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:41.316837 (+233866us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b54002486d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:41.317175 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1993135360}
I20260709 13:19:41.308521 17469 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:41.342653 12709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 2229) took 1021 ms. Trace:
I20260709 13:19:41.342653 17469 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:41.335548 17543 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:41.345855 12707 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:41.310020 17593 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:41.343466 12709 rpcz_store.cc:276] 0709 13:19:40.321318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.426186 (+104868us) service_pool.cc:224] Handling call
0709 13:19:41.342398 (+916212us) spinlock_profiling.cc:243] Waited 97.7 ms on lock 0x7b540006e354. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:41.342540 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":224759552}
I20260709 13:19:41.344707 17469 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:41.341176 17128 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:41.372156 12703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1921) took 1017 ms. Trace:
I20260709 13:19:41.375427 12703 rpcz_store.cc:276] 0709 13:19:40.354472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.164469 (+809997us) service_pool.cc:224] Handling call
0709 13:19:41.172454 (+  7985us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:41.172751 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:41.253421 (+ 80670us) log.cc:844] Serialized 14 byte log entry
0709 13:19:41.370855 (+117434us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:41.370912 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:41.370963 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:41.371987 (+  1024us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:41.372103 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":165317,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":3474,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":1602,"spinlock_wait_cycles":265984,"thread_start_us":190546,"threads_started":2,"wal-append.queue_time_us":120915}
I20260709 13:19:41.378553 12700 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:41.390960 17469 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:41.392956 17436 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:41.393831 17436 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:41.395840 17484 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:41.398207 17484 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:41.397456 17436 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:41.400254 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:41.401067 12627 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:41.401938 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:41.402748 12626 raft_consensus.cc:2393] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 14: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:41.403627 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:41.406994 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:41.407634 12634 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:41.409982 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:41.410353 12627 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:41.410717 12631 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:41.411494 17436 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:41.412124 17436 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:41.414718 17436 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:41.415271 17436 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:41.415798 17436 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:41.425004 12634 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:41.426059 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:41.418659 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:41.426661 12634 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:41.427636 12747 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:41.432076 12747 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:41.433151 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:41.433195 12631 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:19:41.434448 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:41.434964 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:41.435577 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:41.435928 12626 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:41.436457 12627 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:41.436772 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:41.433596 17415 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:41.440099 17415 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.440521 17415 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:41.438827 17086 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:41.438062 17536 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:41.438669 17128 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:41.444538 17607 log.cc:927] Time spent T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:19:41.444737 12705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1906) took 1603 ms. Trace:
I20260709 13:19:41.445151 12705 rpcz_store.cc:276] 0709 13:19:39.841625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.842710 (+  1085us) service_pool.cc:224] Handling call
0709 13:19:39.843026 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:40.139536 (+296510us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:41.444453 (+1304917us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b54001d8654. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:41.444661 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":666484352}
I20260709 13:19:41.446749 12628 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:41.439311 12626 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:41.438798 12747 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:41.447666 12629 raft_consensus.cc:1240] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:19:41.448091 12628 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:19:41.449661 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:41.450950 17436 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:41.452639 12700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1899) took 2159 ms. Trace:
I20260709 13:19:41.453182 12700 rpcz_store.cc:276] 0709 13:19:39.292896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.294608 (+  1712us) service_pool.cc:224] Handling call
0709 13:19:39.328160 (+ 33552us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:41.379056 (+2050896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:41.452298 (+ 73242us) spinlock_profiling.cc:243] Waited 2.05 s on lock 0x7b54002481d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:41.452527 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4714988288}
I20260709 13:19:41.458220 17532 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.458861 17532 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:41.464684 17532 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:41.463863 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:41.469666 12627 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:41.470070 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:41.473840 17436 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:41.471150 12703 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:19:41.475373 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:41.475988 12703 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:41.478552 12673 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:41.467199 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:41.472513 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:41.479396 12705 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:41.479488 12673 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:41.480355 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:41.472621 17554 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:41.473874 12704 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:41.486744 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:41.480271 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:41.472510 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:41.487902 12700 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:41.489998 12703 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:41.480522 12673 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:41.478552 13381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 755) took 1848 ms. Trace:
I20260709 13:19:41.481010 17128 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.481498 17314 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:41.481721 17457 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:41.493140 12673 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:41.471741 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:41.492033 13381 rpcz_store.cc:276] 0709 13:19:39.630599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.631259 (+   660us) service_pool.cc:224] Handling call
0709 13:19:39.642324 (+ 11065us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:40.696632 (+1054308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:41.478330 (+781698us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54000596d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:41.478488 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2420728064}
I20260709 13:19:41.502959 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:41.503750 12672 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:41.504347 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:41.504556 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:41.506800 12747 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:41.508890 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:41.508762 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:41.510106 12673 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:41.510655 12709 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:19:41.511745 12704 raft_consensus.cc:2393] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 14: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:41.511776 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:41.513410 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:41.516952 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:41.519160 12705 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:19:41.535306 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:41.574770 12704 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:19:41.567813 12705 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:41.522310 12747 leader_election.cc:400] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15.
W20260709 13:19:41.576283 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 61390.7us
W20260709 13:19:41.575150 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 59232.3us
I20260709 13:19:41.577651 12673 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:41.580989 12709 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:19:41.581039 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 56936.4us
I20260709 13:19:41.587852 12672 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:41.509579 17436 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:41.508463 17422 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:41.511849 17348 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:41.613217 12627 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:41.509233 17554 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:41.515445 17314 raft_consensus.cc:697] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:41.516251 17140 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:41.576010 17546 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:41.571229 17457 raft_consensus.cc:697] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:41.511924 17536 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:41.581015 17223 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:19:41.511240 17128 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:41.595095 17455 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:19:41.610795 17147 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:41.608566 17422 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:41.619252 17526 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.813s	user 0.030s	sys 0.070s
I20260709 13:19:41.590687 17330 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:19:41.629487 12598 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:41.590411 17086 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:19:41.626701 17546 raft_consensus.cc:697] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:41.621019 17510 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:41.643527 12598 leader_election.cc:400] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12.
I20260709 13:19:41.644156 12598 leader_election.cc:403] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:41.585072 17545 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:19:41.649839 12598 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:41.642486 17596 consensus_queue.cc:1059] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.027s
W20260709 13:19:41.644879 17526 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.836s	user 0.030s	sys 0.071s
I20260709 13:19:41.623050 17436 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:19:41.642143 17422 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:41.618597 17492 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:19:41.663396 17457 consensus_queue.cc:237] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:41.664970 17454 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:41.666196 17546 consensus_queue.cc:237] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:41.659173 17609 log.cc:927] Time spent T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.076s	user 0.002s	sys 0.000s
I20260709 13:19:41.650033 17532 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:41.671758 17128 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:41.650628 17534 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:41.656138 17415 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:41.656484 17321 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:41.678275 17510 raft_consensus.cc:697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:41.681605 12598 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:41.641861 17140 raft_consensus.cc:697] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:41.655893 17543 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12.
I20260709 13:19:41.704337 17532 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:41.698194 17505 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.714916 17505 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:41.715234 17510 consensus_queue.cc:237] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:41.711673 12598 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:41.720698 12598 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:41.730947 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 106353us
I20260709 13:19:41.735034 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:41.736204 12625 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:19:41.698164 17314 consensus_queue.cc:237] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:41.736673 17348 raft_consensus.cc:3055] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:41.910543 17348 raft_consensus.cc:740] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:41.854183 12630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1819) took 1332 ms. Trace:
I20260709 13:19:41.718369 17534 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:19:41.837033 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 253901us
W20260709 13:19:41.836388 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 252316us
I20260709 13:19:41.847530 12634 raft_consensus.cc:1275] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 13:19:41.852906 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 132300us
I20260709 13:19:41.917804 12598 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:41.921995 17532 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:41.923750 17086 consensus_queue.cc:1059] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:41.903512 17432 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:41.925239 17616 raft_consensus.cc:3055] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:41.925755 17616 raft_consensus.cc:740] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:41.925426 17321 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:41.926667 17616 consensus_queue.cc:260] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:41.928141 17616 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:41.932189 12598 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:41.913753 12630 rpcz_store.cc:276] 0709 13:19:40.521872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.042862 (+520990us) service_pool.cc:224] Handling call
0709 13:19:41.044345 (+  1483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:41.044658 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:41.044686 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:41.063779 (+ 19093us) log.cc:844] Serialized 26 byte log entry
0709 13:19:41.442826 (+379047us) spinlock_profiling.cc:243] Waited 7.92 ms on lock 0x7b180033ce80. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf520d1 00007f4c6d36da0f
0709 13:19:41.507772 (+ 64946us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:41.587919 (+ 80147us) log.cc:844] Serialized 14 byte log entry
0709 13:19:41.588497 (+   578us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:41.588577 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:41.844569 (+255992us) raft_consensus.cc:1685] finished
0709 13:19:41.853938 (+  9369us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:41.854118 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":79259,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":4415,"raft.queue_time_us":4441,"raft.run_cpu_time_us":605,"raft.run_wall_time_us":14576,"spinlock_wait_cycles":18215808,"thread_start_us":64711,"threads_started":1,"wal-append.queue_time_us":518452}
I20260709 13:19:41.920631 12634 raft_consensus.cc:1240] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:41.834817 17455 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:41.890985 17570 log.cc:927] Time spent T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 1.390s	user 0.006s	sys 0.011s
I20260709 13:19:41.731166 17511 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:41.954552 12701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1913) took 1725 ms. Trace:
I20260709 13:19:41.743505 17543 raft_consensus.cc:2764] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:19:41.943288 12598 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:41.940893 17532 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:41.923861 17307 raft_consensus.cc:3055] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:41.752295 17505 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:41.944599 17137 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:19:41.841446 17596 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:41.940606 17223 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.911630 17348 consensus_queue.cc:260] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:41.971138 12630 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:41.971810 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:41.960013 12634 raft_consensus.cc:1240] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:41.828533 17454 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:41.946934 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:41.955212 12701 rpcz_store.cc:276] 0709 13:19:40.228692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.241665 (+ 12973us) service_pool.cc:224] Handling call
0709 13:19:40.242063 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:40.242531 (+   468us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:40.242572 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:40.253552 (+ 10980us) log.cc:844] Serialized 26 byte log entry
0709 13:19:40.369740 (+116188us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:40.369798 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:40.369895 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:41.752781 (+1382886us) spinlock_profiling.cc:243] Waited 3.12 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:41.954053 (+201272us) raft_consensus.cc:1685] finished
0709 13:19:41.954084 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:41.954463 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5568,"raft.queue_time_us":104,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":12523,"spinlock_wait_cycles":7196800,"thread_start_us":107704,"threads_started":1,"wal-append.queue_time_us":205451}
I20260709 13:19:41.968318 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:41.973222 12627 raft_consensus.cc:2393] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 10: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:41.946447 17619 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:42.006811 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:41.979317 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:42.008649 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:41.963567 17534 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:41.973768 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:42.005919 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:19:42.006593 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 63398.3us
I20260709 13:19:41.963356 17307 raft_consensus.cc:740] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:41.946776 17432 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:19:41.854128 17140 consensus_queue.cc:237] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:42.012483 12598 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:41.969529 17329 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:42.018599 12598 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:42.022370 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:41.969249 17530 consensus_queue.cc:1059] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:19:42.025534 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:42.027099 12705 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:19:42.027431 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:42.027699 12705 raft_consensus.cc:606] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:42.028565 12705 raft_consensus.cc:614] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:41.968775 17086 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:19:41.957806 17455 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:42.030082 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:42.030802 13378 raft_consensus.cc:2393] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 10: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:42.007172 17619 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:42.011677 17534 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:42.038993 13378 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:19:42.007851 17454 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:42.026983 17543 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:42.027783 17505 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:42.025576 17511 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.976904 17223 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:42.008010 17330 consensus_queue.cc:1059] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:19:42.068185 17330 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.068740 17223 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:42.068668 17330 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:42.018997 17348 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:42.070721 17330 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:42.011545 17536 consensus_queue.cc:1059] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:19:42.075917 17536 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.076573 17536 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:42.079550 17536 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:42.080215 17536 raft_consensus.cc:3055] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:42.080701 17536 raft_consensus.cc:740] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:42.081579 17505 raft_consensus.cc:697] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:42.082588 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:42.082862 17505 consensus_queue.cc:237] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:41.977146 17596 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:42.086771 17596 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:42.059023 17534 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:42.061885 17543 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.090183 17543 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:42.093577 17330 raft_consensus.cc:3055] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:42.051616 17619 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:42.027407 17532 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:19:42.062057 17623 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:19:42.098338 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:42.130832 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:42.176034 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 164660us
W20260709 13:19:42.177304 13379 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51762 (request call id 797) took 2036 ms (client timeout 2010 ms). Trace:
W20260709 13:19:42.177750 13379 rpcz_store.cc:269] 0709 13:19:40.068041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.068274 (+   233us) service_pool.cc:224] Handling call
0709 13:19:42.104555 (+2036281us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b54005415d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:42.104714 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":642793216}
I20260709 13:19:42.179777 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:42.097786 17330 raft_consensus.cc:740] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:42.170008 17554 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:42.190850 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 243417us
I20260709 13:19:42.194864 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:42.195422 12776 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:19:42.196493 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:42.196708 12779 raft_consensus.cc:1240] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:42.197072 12778 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:42.199060 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:42.199585 12778 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:42.200162 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:42.203646 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:42.204936 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:42.206106 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:42.207571 17330 consensus_queue.cc:260] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:42.208878 17330 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:42.189173 17422 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:42.212209 17422 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:42.212759 12705 raft_consensus.cc:1240] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:42.213407 17454 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:42.175104 17307 consensus_queue.cc:260] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:42.214306 17307 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:42.215055 17551 log.cc:927] Time spent T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 2.015s	user 0.010s	sys 0.006s
I20260709 13:19:42.100899 17511 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:42.217484 17511 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:42.217975 17554 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:42.219199 12774 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:42.169600 17536 consensus_queue.cc:260] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:42.221549 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:42.224107 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:42.226554 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:42.227761 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:42.228428 12776 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:42.228814 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:42.229638 12779 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:42.230072 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:42.230707 12777 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 7.
I20260709 13:19:42.230786 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:42.231483 12774 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:42.238207 12775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2746) took 2428 ms. Trace:
I20260709 13:19:42.238654 12775 rpcz_store.cc:276] 0709 13:19:39.810116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.810289 (+   173us) service_pool.cc:224] Handling call
0709 13:19:39.811098 (+   809us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:39.811452 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:39.811478 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:39.830551 (+ 19073us) log.cc:844] Serialized 494 byte log entry
0709 13:19:39.981968 (+151417us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:39.982006 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:39.982078 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:42.170837 (+2188759us) spinlock_profiling.cc:243] Waited 40.6 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:42.237866 (+ 67029us) raft_consensus.cc:1685] finished
0709 13:19:42.237893 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:42.238117 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93289344,"thread_start_us":151026,"threads_started":1,"wal-append.queue_time_us":278635}
I20260709 13:19:42.241901 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:42.242616 17455 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:42.244846 17415 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:42.229768 17536 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:42.251627 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:42.252478 12701 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
W20260709 13:19:42.257802 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 194312us
I20260709 13:19:42.260341 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:42.256098 17511 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:42.263866 17511 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:42.264449 12706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1909) took 2229 ms. Trace:
I20260709 13:19:42.264837 12706 rpcz_store.cc:276] 0709 13:19:40.035367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.035766 (+   399us) service_pool.cc:224] Handling call
0709 13:19:40.036104 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:40.118215 (+ 82111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:42.264243 (+2146028us) spinlock_profiling.cc:243] Waited 81.4 ms on lock 0x7b54001095d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:42.264395 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":187203968}
I20260709 13:19:42.265681 17432 consensus_queue.cc:1059] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:42.266122 17511 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:42.266628 17432 raft_consensus.cc:3055] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:42.266991 17432 raft_consensus.cc:740] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:42.267717 17554 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:42.267834 17432 consensus_queue.cc:260] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:42.280859 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:42.281347 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:42.281737 12706 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:42.241271 17619 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:42.283409 12672 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:42.288488 12672 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:42.291391 12774 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:19:42.285674 17432 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:42.331842 12708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 2227) took 2011 ms (client timeout 2203 ms). Trace:
W20260709 13:19:42.333933 12708 rpcz_store.cc:269] 0709 13:19:40.320635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:40.492060 (+171425us) service_pool.cc:224] Handling call
0709 13:19:42.331624 (+1839564us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b540014bed4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:42.331781 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":480114432}
I20260709 13:19:42.334445 12709 raft_consensus.cc:1240] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:42.336301 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:42.316795 17314 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:42.338462 12774 raft_consensus.cc:2368] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12.
I20260709 13:19:42.340322 12706 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:19:42.340230 12672 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:42.345345 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:42.345546 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:42.346056 12783 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:42.346873 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:42.346066 12776 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:42.348295 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:42.324898 17554 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:42.327047 17422 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:42.357709 12633 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56832 (request call id 3059) took 1582 ms (client timeout 2050 ms). Trace:
I20260709 13:19:42.354812 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:42.358120 12633 rpcz_store.cc:269] 0709 13:19:40.775618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.011305 (+235687us) service_pool.cc:224] Handling call
0709 13:19:42.357516 (+1346211us) spinlock_profiling.cc:243] Waited 60.9 ms on lock 0x7b54002cef54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:42.357651 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":140060032}
I20260709 13:19:42.359012 12779 raft_consensus.cc:2410] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:19:42.360697 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:42.361295 12631 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:42.354206 12776 raft_consensus.cc:2368] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12.
I20260709 13:19:42.363138 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:42.333655 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:42.332984 16710 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.332916 17348 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:42.363804 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 2223) took 2044 ms. Trace:
I20260709 13:19:42.364470 12707 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:19:42.327137 17415 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:42.337237 17626 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.373219 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:42.400779 12633 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:42.375151 12781 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:19:42.375742 12777 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:42.387616 12598 leader_election.cc:400] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8.
I20260709 13:19:42.387951 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:42.392225 12778 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:19:42.402258 12598 leader_election.cc:403] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:42.403213 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:42.386176 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:42.395695 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:42.394515 12702 rpcz_store.cc:276] 0709 13:19:40.319687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.020857 (+701170us) service_pool.cc:224] Handling call
0709 13:19:42.363606 (+1342749us) spinlock_profiling.cc:243] Waited 53.7 ms on lock 0x7b540006d454. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:42.363739 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123574656}
I20260709 13:19:42.354344 17269 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:42.410157 12778 raft_consensus.cc:1275] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:42.400657 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:42.393537 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:42.412467 12631 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:42.436018 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:42.358180 17329 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:19:42.449609 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:42.450234 12781 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:42.363986 17554 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:42.412221 17321 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 7. Current term is 8.
I20260709 13:19:42.373970 17422 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:42.463725 12631 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:42.412823 17454 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:42.474749 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:42.404935 17415 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:42.399153 16710 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:42.412525 17626 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:42.477627 12631 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: [10.6-12.7]
I20260709 13:19:42.484031 12672 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:42.447486 17534 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:42.484141 12633 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:42.489903 12631 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:19:42.492985 12776 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:19:42.482725 17484 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.490980 17348 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:42.474212 17454 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:42.495262 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:42.519027 12778 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:19:42.494108 12776 pending_rounds.cc:85] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 8
I20260709 13:19:42.520677 12776 pending_rounds.cc:107] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 14.9
W20260709 13:19:42.520799 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:42.521061 12776 raft_consensus.cc:2889] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:42.528710 12781 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:42.537859 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:42.540573 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:42.542558 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:42.544070 12781 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:19:42.550402 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:42.555887 12776 raft_consensus.cc:1275] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:42.555536 12597 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:42.566777 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:42.569214 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:42.570479 12779 raft_consensus.cc:2393] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 11: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:42.540649 17505 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.538024 17619 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:42.574589 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:42.588912 12778 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
W20260709 13:19:42.589924 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 92421.5us
I20260709 13:19:42.591128 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:42.604349 17638 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:42.609334 17534 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:42.614535 17634 log.cc:927] Time spent T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
W20260709 13:19:42.619554 17543 consensus_meta.cc:287] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.529s	user 0.011s	sys 0.000s
I20260709 13:19:42.620162 17543 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:42.621690 12629 raft_consensus.cc:1240] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:19:42.617048 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:42.624708 12629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:19:42.626348 12629 raft_consensus.cc:606] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:42.626632 17626 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:42.627707 17632 log.cc:927] Time spent T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
W20260709 13:19:42.630968 17636 log.cc:927] Time spent T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:19:42.633666 12632 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:19:42.635143 12632 raft_consensus.cc:3055] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:42.635557 12632 raft_consensus.cc:740] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:42.644091 17137 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:42.649035 17454 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:42.645085 17137 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:42.656883 17633 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:19:42.653442 17137 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:42.665045 17454 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:42.742352 17454 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:42.745394 17454 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:42.747604 12632 consensus_queue.cc:260] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:42.749475 12632 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:42.665045 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 70858.1us
I20260709 13:19:42.666903 17269 raft_consensus.cc:2955] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Committing config change with OpId 9.9: config changed from index 4 to 9, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:42.670770 16710 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:42.545547 17511 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:42.673625 17558 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:19:42.676347 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 120937us
I20260709 13:19:42.677069 17532 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:42.545364 17321 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:42.684041 17422 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:42.686398 12629 raft_consensus.cc:1240] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:42.687917 17554 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:42.799069 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:42.700989 12707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1924) took 2185 ms. Trace:
I20260709 13:19:42.701653 12783 raft_consensus.cc:2410] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:19:42.518705 17484 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:42.704156 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:42.704630 17505 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:42.706813 17436 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.689016 12774 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:19:42.804420 12774 pending_rounds.cc:85] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 7
I20260709 13:19:42.804804 12774 pending_rounds.cc:107] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:19:42.805150 12774 raft_consensus.cc:2889] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:42.708559 17460 raft_consensus.cc:3055] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:42.713001 12631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:56830
I20260709 13:19:42.713845 17640 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.716233 12626 raft_consensus.cc:1275] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:42.780786 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:42.717592 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:42.723106 12633 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:19:42.724848 12747 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:42.810498 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 85752.2us
I20260709 13:19:42.725692 17626 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:42.732692 17638 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:42.733525 17581 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:42.734195 17571 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.734704 17630 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:42.736305 12775 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:19:42.740103 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:42.835238 12628 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:42.836032 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:42.836730 12628 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:42.752739 12702 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:19:42.760403 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:42.765888 12704 raft_consensus.cc:1275] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:42.838337 12625 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:19:42.766852 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 226325us
I20260709 13:19:42.798955 17558 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:42.841332 13349 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:42.842710 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:42.771786 12673 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:42.777333 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:42.778784 12597 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:42.782963 12598 leader_election.cc:400] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13.
I20260709 13:19:42.779966 13348 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:42.797096 12777 raft_consensus.cc:1240] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:42.691375 17415 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:42.845145 12705 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:42.800411 12707 rpcz_store.cc:276] 0709 13:19:40.515986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.347147 (+831161us) service_pool.cc:224] Handling call
0709 13:19:42.364202 (+1017055us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:42.395329 (+ 31127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:42.700777 (+305448us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540006d450. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:42.700922 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2338320768}
I20260709 13:19:42.851696 12632 raft_consensus.cc:1275] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 13:19:42.806579 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:42.852147 12632 pending_rounds.cc:85] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 9
I20260709 13:19:42.852500 12632 pending_rounds.cc:107] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:19:42.852723 12632 raft_consensus.cc:2889] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:42.811537 12630 raft_consensus.cc:1240] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:42.766423 17137 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.811363 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:42.812729 17436 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:42.826009 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:42.856854 12781 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:42.857121 12776 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:42.857539 13380 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:42.819244 12778 raft_consensus.cc:3055] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:42.858001 12778 raft_consensus.cc:740] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:42.829430 12672 leader_election.cc:400] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12.
I20260709 13:19:42.814067 12780 raft_consensus.cc:2955] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Committing config change with OpId 9.9: config changed from index 4 to 9, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:42.845391 12598 leader_election.cc:403] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:19:42.860368 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 77633.6us
I20260709 13:19:42.860826 17534 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13.
W20260709 13:19:42.861294 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:42.863690 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:42.864395 13377 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:42.865394 13382 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:19:42.866750 16710 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:42.867406 16710 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:42.845141 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 73443.7us
I20260709 13:19:42.872653 16710 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
W20260709 13:19:42.845247 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 66560us
W20260709 13:19:42.855582 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 75707.1us
I20260709 13:19:42.806463 17460 raft_consensus.cc:740] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:42.883736 12632 raft_consensus.cc:1240] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:42.884342 13377 raft_consensus.cc:1240] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:42.876507 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:42.803215 17623 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:19:42.890807 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:42.880440 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 69151.8us
I20260709 13:19:42.841531 17558 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:42.828508 17454 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:42.900076 13381 raft_consensus.cc:1275] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:19:42.900544 13381 pending_rounds.cc:85] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd: Aborting all ops after (but not including) 6
I20260709 13:19:42.900874 13381 pending_rounds.cc:107] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:19:42.901141 13381 raft_consensus.cc:2889] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:42.901418 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:42.883352 12672 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:42.849833 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:42.879172 12597 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:42.893404 12747 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:42.802368 17321 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:42.835470 17625 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:42.799474 17631 consensus_queue.cc:1059] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.108s
I20260709 13:19:42.851320 17306 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:42.914741 12597 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:42.858045 17530 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:42.858367 17140 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:42.873980 17432 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:42.799541 17511 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:42.856364 17532 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:42.919806 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:42.940371 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 48637.7us
I20260709 13:19:42.940047 12782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 2412) took 3014 ms. Trace:
I20260709 13:19:42.891356 17415 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:42.941208 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 57935.9us
I20260709 13:19:42.815587 17640 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:42.945186 12672 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:42.946035 12597 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:42.852072 17630 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:42.842713 17581 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:42.842952 17571 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:42.960353 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:42.833446 17638 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:42.915848 17543 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:42.941746 17554 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:42.971410 12631 raft_consensus.cc:606] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Received request to transfer leadership
I20260709 13:19:42.971833 12631 raft_consensus.cc:614] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:42.890870 17307 consensus_queue.cc:1059] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:19:42.921541 17534 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:42.875422 17137 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:42.961392 12672 leader_election.cc:400] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12.
I20260709 13:19:42.940819 17140 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.944103 17509 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:42.947408 17527 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:42.973145 12747 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:42.988015 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:42.971266 13384 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:42.959167 12597 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:42.945052 12782 rpcz_store.cc:276] 0709 13:19:39.925574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.925760 (+   186us) service_pool.cc:224] Handling call
0709 13:19:39.926118 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:40.030974 (+104856us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:40.044283 (+ 13309us) log.cc:844] Serialized 14 byte log entry
0709 13:19:41.441206 (+1396923us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x7b4800042258. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c722dbaa1 00007f4c722d2b37 00007f4c722d29ca 00007f4c723ca26f 00007f4c723c9f37 00007f4c723de55c 00007f4c723de508 00007f4c723de488 00007f4c723de443 00007f4c723dd6ec 00007f4c71307a96
0709 13:19:42.261675 (+820469us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b18004aad60. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf5223a 00007f4c6d3bca2a
0709 13:19:42.778111 (+516436us) log.cc:844] Serialized 14 byte log entry
0709 13:19:42.939686 (+161575us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:42.939756 (+    70us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:42.939806 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:42.939869 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:42.939985 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83745,"prepare.queue_time_us":428842,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":129325,"raft.run_cpu_time_us":15843,"raft.run_wall_time_us":1413626,"spinlock_wait_cycles":9956480,"thread_start_us":655155,"threads_started":3,"wal-append.queue_time_us":2474284}
I20260709 13:19:42.946461 17532 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:42.916323 17454 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:42.963224 17539 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:43.000128 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:42.964560 17641 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:42.990911 12597 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:42.991384 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:42.998682 12672 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:42.963162 17530 raft_consensus.cc:697] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:42.975363 17536 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:19:43.004734 12517 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 4c13fa2aff324f65ad0bf6191ea68fab with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 639ab2db265f4747bb7afb93e833f530 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:42.976207 17140 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:42.956753 17619 consensus_queue.cc:1059] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:19:42.980082 17527 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.979805 17509 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:42.970141 17580 consensus_queue.cc:1048] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:19:43.025036 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:43.034179 13379 raft_consensus.cc:2410] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:19:43.006089 17532 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:43.006026 17635 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:42.997515 17534 raft_consensus.cc:697] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:43.007393 12673 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:43.002638 17554 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:43.025383 17455 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:43.007264 17641 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:42.967612 17646 consensus_queue.cc:1059] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:19:42.979449 17626 consensus_queue.cc:1059] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:19:43.006726 17457 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:19:43.022658 17543 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:19:43.042021 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 52891.4us
I20260709 13:19:43.050375 12673 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:43.024075 17536 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:43.055800 17536 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:43.055984 17645 log.cc:927] Time spent T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:19:43.051483 17505 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:42.969787 17644 consensus_queue.cc:1048] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:19:43.061641 17529 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:43.062260 17529 raft_consensus.cc:697] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:43.063093 17529 consensus_queue.cc:237] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:43.063557 17534 consensus_queue.cc:237] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:43.034516 17623 consensus_queue.cc:1048] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:19:43.043386 17641 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:43.067937 12673 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:43.069345 17539 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:43.069769 17539 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:43.070118 17321 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:43.071065 17644 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:43.071605 17454 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:43.036497 17128 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:19:43.072794 17223 consensus_queue.cc:1059] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:43.073122 17128 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:43.073495 17642 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:43.074025 17510 raft_consensus.cc:3055] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:43.074429 17510 raft_consensus.cc:740] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:42.946578 16710 consensus_queue.cc:1059] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:19:43.030722 17527 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:43.075176 17510 consensus_queue.cc:260] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:43.076371 17510 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:43.076897 17436 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:43.079422 12634 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:19:43.080597 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:43.030840 17422 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:43.081728 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:43.033836 17509 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:43.082305 13377 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:19:43.082543 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:43.083113 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:43.083750 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:43.084190 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:43.085096 13380 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 13:19:43.053877 17314 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:43.070147 17539 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:43.085929 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:43.054261 17626 consensus_queue.cc:1243] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 639ab2db265f4747bb7afb93e833f530 log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:19:43.010694 17526 consensus_queue.cc:1059] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:19:43.088397 17581 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:43.088874 17526 raft_consensus.cc:3055] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:43.089250 17526 raft_consensus.cc:740] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:43.052049 17484 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:43.055301 17647 log.cc:927] Time spent T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:19:43.090065 17638 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:43.090372 17651 consensus_queue.cc:1059] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:43.091334 17652 consensus_queue.cc:1059] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:19:43.055168 17455 raft_consensus.cc:697] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:43.092952 17455 consensus_queue.cc:237] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:43.093357 17511 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:43.094122 17619 raft_consensus.cc:3055] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:43.094717 17619 raft_consensus.cc:740] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:43.095542 17637 raft_consensus.cc:3055] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:43.081883 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:43.053782 17530 consensus_queue.cc:237] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:43.108604 17274 raft_consensus.cc:3055] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:43.109078 17274 raft_consensus.cc:740] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:43.109992 17274 consensus_queue.cc:260] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:43.075634 17625 raft_consensus.cc:3055] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:43.094195 17543 raft_consensus.cc:697] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:43.096776 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:43.136433 12631 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:43.091500 17571 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:43.097210 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 52126.7us
I20260709 13:19:43.098323 12746 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:43.099781 12632 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:19:43.099839 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:43.140599 12672 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:43.106690 17637 raft_consensus.cc:740] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:19:43.153739 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 129573us
W20260709 13:19:43.156828 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 58588.5us
I20260709 13:19:43.157431 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:43.158545 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:43.107906 17640 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:43.050017 17546 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:43.047063 17613 consensus_queue.cc:1048] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:19:43.111413 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:43.101263 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:43.143203 12672 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:43.162462 13348 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:43.146054 12673 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:43.162982 12628 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:19:43.147545 17454 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:43.160526 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:43.162358 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:43.171026 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 157322us
I20260709 13:19:43.177017 12673 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:43.177589 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:43.126829 17626 raft_consensus.cc:3055] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:43.126286 17625 raft_consensus.cc:740] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:43.160423 17274 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:43.135366 17137 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:43.184469 12672 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:43.186992 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:43.160657 17314 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:43.161628 17546 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:43.171468 17422 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:43.198653 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:43.095590 17619 consensus_queue.cc:260] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:43.201877 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:43.208978 12625 raft_consensus.cc:2410] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:19:43.210680 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:43.160490 17528 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:19:43.182228 17626 raft_consensus.cc:740] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:43.221360 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:43.188288 17663 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:43.183130 17635 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:43.185895 17464 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:43.195030 16710 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:43.192943 17128 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:43.201735 17314 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:43.218541 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:43.204756 17546 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:43.171617 17545 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:19:43.260502 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:43.206799 17422 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:43.204701 17137 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:43.090090 17526 consensus_queue.cc:260] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:43.275871 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:43.274668 12782 raft_consensus.cc:1275] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:19:43.278801 12709 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 13:19:43.219852 17457 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:43.281342 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:43.214700 17086 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:43.200309 17596 consensus_queue.cc:1048] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:19:43.287581 12776 raft_consensus.cc:2393] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 13: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:43.284165 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:43.265007 17314 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:43.282797 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:43.263043 17128 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:43.275549 17422 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:43.312286 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:43.312923 13348 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:43.313099 12782 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
W20260709 13:19:43.315920 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.262598 17625 consensus_queue.cc:260] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:43.275996 16710 raft_consensus.cc:697] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:43.210839 17637 consensus_queue.cc:260] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:43.258194 17635 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:43.309139 17656 log.cc:927] Time spent T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.277s	user 0.002s	sys 0.000s
I20260709 13:19:43.327814 13348 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:43.286609 17137 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:43.329058 17580 consensus_queue.cc:1048] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:43.310134 17128 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:43.333112 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:43.337308 17664 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:43.326789 17638 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:43.312738 17626 consensus_queue.cc:260] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:43.289000 17527 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:43.257131 17543 consensus_queue.cc:237] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:43.366886 13348 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:43.397127 13348 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:43.386653 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:43.460009 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.376868 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:43.357671 12627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1833) took 1152 ms. Trace:
I20260709 13:19:43.390017 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:19:43.474800 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.371889 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:43.496855 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 99796.7us
I20260709 13:19:43.474700 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:43.367864 17137 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:43.374545 12625 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:43.364800 17526 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:43.541077 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 383169us
W20260709 13:19:43.548220 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.374210 17626 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:43.296614 17086 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:43.549582 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:43.474697 17505 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:43.542308 12627 rpcz_store.cc:276] 0709 13:19:42.205321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.213649 (+  8328us) service_pool.cc:224] Handling call
0709 13:19:42.213977 (+   328us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:42.517200 (+303223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:42.517247 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:42.538830 (+ 21583us) log.cc:844] Serialized 52 byte log entry
0709 13:19:42.869951 (+331121us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:43.008497 (+138546us) log.cc:844] Serialized 14 byte log entry
0709 13:19:43.021006 (+ 12509us) log.cc:844] Serialized 14 byte log entry
0709 13:19:43.179241 (+158235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.357202 (+177961us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b54002eaed4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:43.357257 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:43.357400 (+   143us) raft_consensus.cc:1685] finished
0709 13:19:43.357416 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:43.357599 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38350,"prepare.queue_time_us":181524,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":48,"raft.queue_time_us":78447,"raft.run_cpu_time_us":385,"raft.run_wall_time_us":8114,"spinlock_wait_cycles":5405568,"thread_start_us":829732,"threads_started":4,"wal-append.queue_time_us":343841}
I20260709 13:19:43.388468 17623 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:43.473542 17421 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:43.365608 17637 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:43.367952 17580 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:19:43.556720 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 380156us
I20260709 13:19:43.369489 17635 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:43.533035 17571 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:43.559653 12704 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:19:43.365685 17269 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:49378 (request call id 10) took 5482 ms. Trace:
I20260709 13:19:43.372637 17554 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.178s
I20260709 13:19:43.563452 12746 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:43.379375 17534 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:43.360574 17630 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:43.381229 17527 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:43.591583 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 219756us
I20260709 13:19:43.533766 17511 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:43.353551 17539 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:43.374229 16710 consensus_queue.cc:237] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:43.380334 17625 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:43.549505 17469 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:43.384446 17314 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:43.689621 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.383756 17509 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:43.692433 17663 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:43.389705 17532 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:43.357015 17641 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:43.363521 17536 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:43.726511 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.564569 17678 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:43.373189 17664 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:43.608450 17652 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:43.354620 17460 consensus_queue.cc:260] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:43.820968 12702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc7827272b424bad88349b6b7437844b"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:19:43.821910 12702 raft_consensus.cc:606] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:43.822356 12702 raft_consensus.cc:614] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:43.820968 12629 raft_consensus.cc:1240] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:43.831044 12626 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:43.769142 17527 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.731771 17534 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:43.716620 17571 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:43.762921 12672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.068s	sys 0.107s
W20260709 13:19:43.876609 12672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.069s	sys 0.107s
W20260709 13:19:43.877554 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 693132us
I20260709 13:19:43.735505 17630 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:43.769829 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 135792us
I20260709 13:19:43.769358 12708 raft_consensus.cc:2393] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:43.608558 17505 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:43.774739 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:43.774923 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:43.775104 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:43.776526 13385 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:19:43.725696 17623 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:43.749013 17644 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:43.851537 17663 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:19:43.852312 17140 consensus_meta.cc:287] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.818s	user 0.003s	sys 0.006s
W20260709 13:19:43.855762 17274 consensus_meta.cc:287] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.655s	user 0.009s	sys 0.000s
W20260709 13:19:43.860581 17546 consensus_meta.cc:287] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.585s	user 0.007s	sys 0.000s
I20260709 13:19:43.862838 17086 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:43.971146 12778 consensus_queue.cc:260] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:43.971704 17086 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:43.972395 12778 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:43.972528 17086 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:43.864267 17510 consensus_meta.cc:287] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.787s	user 0.008s	sys 0.002s
I20260709 13:19:43.974714 12706 raft_consensus.cc:2410] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 13:19:43.975540 17665 log.cc:927] Time spent T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.829s	user 0.012s	sys 0.009s
W20260709 13:19:43.714640 12703 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.016s	sys 0.024s
I20260709 13:19:43.977439 17546 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:43.757912 17469 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:43.893653 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.894420 13382 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:43.979308 17623 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:43.892784 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 330453us
I20260709 13:19:43.979768 17140 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:43.980168 12746 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:43.980324 17469 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:43.981284 12746 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:43.981969 17140 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:43.855862 17422 consensus_meta.cc:287] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.528s	user 0.008s	sys 0.000s
I20260709 13:19:43.864382 17314 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:43.982409 12746 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:43.906203 12672 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:43.982687 17422 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:43.983688 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 77605.1us
I20260709 13:19:43.984994 17422 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:43.962117 17554 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:43.985743 12701 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:43.986083 17554 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:43.949890 17644 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:43.987564 17509 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:43.989027 17536 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:43.989418 17644 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:43.989912 17536 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:43.944983 17630 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:43.990206 17644 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:43.990559 17536 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:43.991143 17536 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:43.991842 17644 consensus_queue.cc:1059] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:43.715763 17269 rpcz_store.cc:276] 0709 13:19:37.841500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:37.851263 (+  9763us) service_pool.cc:224] Handling call
0709 13:19:37.930366 (+ 79103us) log.cc:844] Serialized 494 byte log entry
0709 13:19:38.565087 (+634721us) spinlock_profiling.cc:243] Waited 8.92 ms on lock 0x7b1800428f20. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c53f3 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:19:43.323105 (+4758018us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3997,"raft.queue_time_us":12163,"raft.run_cpu_time_us":985,"raft.run_wall_time_us":95437,"spinlock_wait_cycles":20507520}
I20260709 13:19:43.992250 17630 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:43.902299 17673 log.cc:927] Time spent T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
I20260709 13:19:43.993716 17510 raft_consensus.cc:3055] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:43.994087 17510 raft_consensus.cc:740] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:43.994740 17505 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:43.994904 17510 consensus_queue.cc:260] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:43.865237 17527 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:43.996011 17510 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:43.997857 17527 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:43.893424 17571 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:43.999871 17532 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:43.880959 17137 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:44.004854 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:44.001665 17534 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:43.969663 17693 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:43.978099 17661 log.cc:927] Time spent T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.847s	user 0.005s	sys 0.018s
I20260709 13:19:43.978603 13383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51762 (request call id 819) took 1118 ms. Trace:
I20260709 13:19:43.757447 17511 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:44.008800 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:43.968762 17674 log.cc:927] Time spent T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.116s	user 0.002s	sys 0.000s
I20260709 13:19:44.026535 13383 rpcz_store.cc:276] 0709 13:19:42.859752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.861798 (+  2046us) service_pool.cc:224] Handling call
0709 13:19:42.862170 (+   372us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:43.005437 (+143267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:43.005473 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:43.024225 (+ 18752us) log.cc:844] Serialized 52 byte log entry
0709 13:19:43.158887 (+134662us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:43.158944 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.342904 (+183960us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b54003890d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:43.342961 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:43.974925 (+631964us) spinlock_profiling.cc:243] Waited 7.55 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:43.978391 (+  3466us) raft_consensus.cc:1685] finished
0709 13:19:43.978414 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:43.978551 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24454,"raft.queue_time_us":33172,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":5809,"spinlock_wait_cycles":302932736,"thread_start_us":100557,"threads_started":1,"wal-append.queue_time_us":104335}
I20260709 13:19:43.829439 17652 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:43.801256 17678 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:44.006552 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet af96e52b004d4c8c942cc0ec33c95c99 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 6)
I20260709 13:19:44.039847 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:19:44.013303 17128 consensus_meta.cc:287] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.645s	user 0.010s	sys 0.001s
I20260709 13:19:44.002100 17546 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:44.039162 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:44.039696 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:44.023049 12778 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:19:44.044301 12778 pending_rounds.cc:85] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 6
I20260709 13:19:44.044605 12778 pending_rounds.cc:107] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:19:44.013486 17137 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:44.045010 12778 raft_consensus.cc:2889] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:44.049341 12778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2791) took 1251 ms. Trace:
W20260709 13:19:44.034880 17543 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.013s	sys 0.004s
I20260709 13:19:44.050081 12778 rpcz_store.cc:276] 0709 13:19:42.798489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.798689 (+   200us) service_pool.cc:224] Handling call
0709 13:19:42.819113 (+ 20424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:43.015855 (+196742us) spinlock_profiling.cc:243] Waited 34 ms on lock 0x7b4c001f6e80. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6b67ea20 00007f4c6b68fa29 00007f4c6b655644 00007f4c6b67cf2d 00007f4c6b67cb93 00007f4c6d3ad18b 00007f4c6d3adcb5 00007f4c6d3a8f30 00007f4c6d3aa7de 00007f4c6d3b4b49 00007f4c6d3b5416
0709 13:19:43.398037 (+382182us) spinlock_profiling.cc:243] Waited 73.5 ms on lock 0x7b54001f16a0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d33006f 00007f4c6d3a8f60 00007f4c6d3aa7de 00007f4c6d3b4b49 00007f4c6d3b5416 00007f4c6d3b1d8f 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62
0709 13:19:43.970834 (+572797us) spinlock_profiling.cc:243] Waited 49.7 ms on lock 0x7b54001f16a0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d32f520 00007f4c6d3a8f70 00007f4c6d3aa7de 00007f4c6d3b4b49 00007f4c6d3b5416 00007f4c6d3b1d8f 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62
0709 13:19:44.049062 (+ 78228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.049279 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":361545600}
I20260709 13:19:44.040279 17641 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:44.051520 17137 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:44.070509 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:44.071345 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:44.003716 17663 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:44.058461 17269 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:44.078696 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:44.048789 17128 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:44.089239 12781 pending_rounds.cc:85] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 7
I20260709 13:19:44.089664 12781 pending_rounds.cc:107] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:19:44.089947 12781 raft_consensus.cc:2889] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:44.092767 12782 raft_consensus.cc:1275] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:19:44.100306 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:44.100997 12774 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:44.107728 17702 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:44.108526 17702 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:44.111500 12774 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:44.117882 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:44.118474 12780 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:44.119304 12625 raft_consensus.cc:3055] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:44.119656 12625 raft_consensus.cc:740] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:44.120262 12707 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:19:44.120486 12625 consensus_queue.cc:260] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:44.122499 17652 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:44.123067 17652 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:44.123646 17666 log.cc:927] Time spent T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:19:44.123522 17652 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:44.035151 17693 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:44.121462 12625 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:44.136354 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:44.137666 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:44.138401 12705 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:19:44.138612 12780 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:19:44.139353 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:44.143599 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:44.145087 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:44.145927 12775 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:19:44.146490 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:44.149276 12780 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:44.154393 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:44.077207 17415 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:44.169963 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.161249 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:44.161545 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:44.175047 12780 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:19:44.164273 12706 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:44.158172 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:44.184433 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.195262 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:44.229338 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.194643 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:44.174484 12705 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:19:44.080670 17539 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:44.165071 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 181495us
I20260709 13:19:44.314168 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:44.314867 12703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1940) took 1121 ms. Trace:
I20260709 13:19:44.315268 12703 rpcz_store.cc:276] 0709 13:19:43.193321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.196049 (+  2728us) service_pool.cc:224] Handling call
0709 13:19:43.198991 (+  2942us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:43.199214 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:43.976852 (+777638us) log.cc:844] Serialized 14 byte log entry
0709 13:19:44.314492 (+337640us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:44.314544 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:44.314586 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.314647 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:44.314757 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":187086,"prepare.queue_time_us":66373,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":2417,"thread_start_us":913250,"threads_started":2}
I20260709 13:19:44.327942 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:44.329947 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:44.329975 12780 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:19:44.330525 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:44.330629 12783 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:44.331154 12775 raft_consensus.cc:1813] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Handling vote request from an unknown peer 19cd864c053744bebd96093a7982537e
I20260709 13:19:44.331775 12775 raft_consensus.cc:2410] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 13:19:44.333119 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 138603us
I20260709 13:19:44.333705 13382 raft_consensus.cc:1813] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Handling vote request from an unknown peer 19cd864c053744bebd96093a7982537e
I20260709 13:19:44.334533 13382 raft_consensus.cc:2410] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:19:44.336328 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:44.336872 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:44.341413 12783 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:19:44.340871 12625 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:19:44.342739 12625 pending_rounds.cc:85] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 10
I20260709 13:19:44.342962 12625 pending_rounds.cc:107] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:19:44.343191 12625 raft_consensus.cc:2889] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:44.346693 12700 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:19:44.370172 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:44.398947 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:44.127521 17269 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:44.347692 17558 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:19:44.343633 17625 consensus_meta.cc:287] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.588s	user 0.005s	sys 0.006s
I20260709 13:19:44.317945 17314 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:44.331122 17348 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:44.313256 17415 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:44.408679 17698 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:44.414212 17678 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:44.468421 17348 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:44.471819 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.398550 17529 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
W20260709 13:19:44.474695 17687 log.cc:927] Time spent T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.677s	user 0.006s	sys 0.005s
I20260709 13:19:44.476785 17415 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:44.477782 12779 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:19:44.507367 12779 pending_rounds.cc:85] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 9
I20260709 13:19:44.507728 12779 pending_rounds.cc:107] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:19:44.508082 12779 raft_consensus.cc:2889] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:44.478106 17690 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.478952 17664 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:44.480434 13378 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51762 (request call id 816) took 1631 ms (client timeout 2010 ms). Trace:
W20260709 13:19:44.509696 13378 rpcz_store.cc:269] 0709 13:19:42.849369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.849518 (+   149us) service_pool.cc:224] Handling call
0709 13:19:44.480216 (+1630698us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b54005415d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:44.480367 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":883374336}
I20260709 13:19:44.482227 13381 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 13. Current term is 14. Ops: []
W20260709 13:19:44.513077 17697 log.cc:927] Time spent T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 13:19:44.484133 17700 log.cc:927] Time spent T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.355s	user 0.001s	sys 0.000s
I20260709 13:19:44.485816 17702 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:44.487362 17128 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:44.487852 17652 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:44.489398 17663 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:44.423491 17558 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:19:44.459933 17675 log.cc:927] Time spent T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:19:44.518338 12634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 3111) took 1172 ms. Trace:
I20260709 13:19:44.513000 17348 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:44.418758 17539 consensus_queue.cc:1048] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
W20260709 13:19:44.568890 17626 consensus_meta.cc:287] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.998s	user 0.009s	sys 0.003s
I20260709 13:19:44.577792 12774 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:19:44.578178 12774 pending_rounds.cc:85] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 6
I20260709 13:19:44.578414 12774 pending_rounds.cc:107] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:19:44.578675 12774 raft_consensus.cc:2889] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:44.521622 17690 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:44.522905 17269 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:44.524528 17708 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.525494 17455 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:44.526312 17683 log.cc:927] Time spent T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.755s	user 0.008s	sys 0.004s
I20260709 13:19:44.526782 17086 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:44.528498 17706 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.529119 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:44.530040 17707 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:44.535375 12704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1945) took 1285 ms. Trace:
W20260709 13:19:44.556216 12708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 1935) took 1394 ms (client timeout 1687 ms). Trace:
W20260709 13:19:44.601285 12708 rpcz_store.cc:269] 0709 13:19:43.161840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.162132 (+   292us) service_pool.cc:224] Handling call
0709 13:19:44.530588 (+1368456us) spinlock_profiling.cc:243] Waited 595 ms on lock 0x7b540045f6d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:44.556136 (+ 25548us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1368626432}
I20260709 13:19:44.507905 17511 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:44.552810 17510 consensus_meta.cc:287] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.556s	user 0.008s	sys 0.003s
I20260709 13:19:44.563269 12634 rpcz_store.cc:276] 0709 13:19:43.346147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.569722 (+223575us) service_pool.cc:224] Handling call
0709 13:19:43.608143 (+ 38421us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:43.672500 (+ 64357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:43.672545 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:43.733146 (+ 60601us) log.cc:844] Serialized 52 byte log entry
0709 13:19:44.025692 (+292546us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:44.025738 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.025813 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:44.518107 (+492294us) raft_consensus.cc:1685] finished
0709 13:19:44.518127 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:44.518275 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78731,"raft.queue_time_us":107913,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":10105,"thread_start_us":227804,"threads_started":1,"wal-append.queue_time_us":549203}
I20260709 13:19:44.467459 17631 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.089s
I20260709 13:19:44.562391 17558 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:44.592869 12775 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:44.600005 12704 rpcz_store.cc:276] 0709 13:19:43.249544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.249718 (+   174us) service_pool.cc:224] Handling call
0709 13:19:43.250297 (+   579us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:43.716839 (+466542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.535032 (+818193us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b54001cd9d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:44.535295 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":710812032}
I20260709 13:19:44.505455 17698 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:44.625393 17580 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.626698 12629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 3107) took 1499 ms. Trace:
I20260709 13:19:44.604365 17274 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.641664 12775 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:44.631073 12629 rpcz_store.cc:276] 0709 13:19:43.127317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.140798 (+ 13481us) service_pool.cc:224] Handling call
0709 13:19:43.141052 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:43.823797 (+682745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.626441 (+802644us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b5400370854. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:44.626636 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1512137088}
I20260709 13:19:44.599185 17455 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:44.584312 17526 consensus_meta.cc:287] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 1.026s	user 0.009s	sys 0.002s
W20260709 13:19:44.633496 17637 consensus_meta.cc:287] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.934s	user 0.011s	sys 0.002s
I20260709 13:19:44.593819 17708 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:44.664278 12628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 3108) took 1536 ms. Trace:
I20260709 13:19:44.666997 12628 rpcz_store.cc:276] 0709 13:19:43.128134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.163918 (+ 35784us) service_pool.cc:224] Handling call
0709 13:19:43.164321 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:43.325381 (+161060us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:43.325591 (+   210us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:43.326185 (+   594us) log.cc:844] Serialized 26 byte log entry
0709 13:19:43.536911 (+210726us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:43.536973 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:43.537078 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:44.525552 (+988474us) spinlock_profiling.cc:243] Waited 93.9 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:44.658674 (+133122us) raft_consensus.cc:1685] finished
0709 13:19:44.658707 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:44.664179 (+  5472us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20620,"raft.queue_time_us":28505,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":8559,"spinlock_wait_cycles":215857408,"thread_start_us":359531,"threads_started":2,"wal-append.queue_time_us":350817}
I20260709 13:19:44.610419 17706 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:44.610908 17707 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:44.670302 13377 raft_consensus.cc:2410] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
W20260709 13:19:44.663403 16710 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.006s	sys 0.012s
I20260709 13:19:44.644143 17274 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:44.677920 12746 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:44.679134 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:44.682229 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.635278 17580 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:44.683318 17269 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:44.684429 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:44.685038 17269 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:44.685266 12672 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:44.685745 17348 consensus_queue.cc:1048] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:44.686424 12672 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:44.686702 17637 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:44.687251 12672 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:44.687304 17637 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:44.688124 17348 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:44.688369 12672 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:44.688410 17539 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:44.690994 17539 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:44.691370 17539 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:44.690418 17707 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:44.675725 16710 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.006s	sys 0.013s
I20260709 13:19:44.697443 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:44.701109 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:44.701412 17705 log.cc:927] Time spent T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:19:44.702023 13383 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:19:44.692535 17086 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:44.697824 17128 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:44.707074 17706 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:44.708855 17708 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:44.710595 17455 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:44.713697 17690 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:44.714351 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:44.714998 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.715381 17698 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:44.688158 17684 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:44.718251 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:44.718629 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:44.719357 17460 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:44.720599 12627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 3110) took 1441 ms. Trace:
I20260709 13:19:44.721133 17640 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:44.720956 12627 rpcz_store.cc:276] 0709 13:19:43.279427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.566965 (+287538us) service_pool.cc:224] Handling call
0709 13:19:43.567767 (+   802us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:43.568204 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:43.568239 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:43.673829 (+105590us) log.cc:844] Serialized 52 byte log entry
0709 13:19:43.868868 (+195039us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:44.069676 (+200808us) log.cc:844] Serialized 14 byte log entry
0709 13:19:44.720311 (+650635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.720382 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:44.720418 (+    36us) raft_consensus.cc:1685] finished
0709 13:19:44.720429 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:44.720549 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":650399,"prepare.queue_time_us":542422,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":755222,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":40284,"thread_start_us":247794,"threads_started":2,"wal-append.queue_time_us":280653}
I20260709 13:19:44.723438 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:44.727123 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:44.727255 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:44.727869 12629 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:44.728811 12746 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:44.729265 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:44.730046 12634 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:44.727892 12625 raft_consensus.cc:2393] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 17: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:44.727067 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:44.721992 17640 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:44.723171 12631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 3117) took 1332 ms. Trace:
I20260709 13:19:44.737651 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:44.739480 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:44.740127 12630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 884) took 1346 ms. Trace:
I20260709 13:19:44.741989 12627 raft_consensus.cc:2393] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 13: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:44.740660 12630 rpcz_store.cc:276] 0709 13:19:43.394111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.081841 (+687730us) service_pool.cc:224] Handling call
0709 13:19:44.127929 (+ 46088us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.128130 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:44.455593 (+327463us) log.cc:844] Serialized 14 byte log entry
0709 13:19:44.739681 (+284088us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:44.739758 (+    77us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:44.739819 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.739908 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:44.740054 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88925,"prepare.queue_time_us":283656,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"thread_start_us":341011,"threads_started":2,"wal-append.queue_time_us":238453}
I20260709 13:19:44.681910 17526 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:44.743458 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.746429 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:44.748464 12627 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:19:44.736486 12631 rpcz_store.cc:276] 0709 13:19:43.390845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.058598 (+667753us) service_pool.cc:224] Handling call
0709 13:19:44.058931 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.059143 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:44.366499 (+307356us) log.cc:844] Serialized 14 byte log entry
0709 13:19:44.722860 (+356361us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:44.722915 (+    55us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:44.722953 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.723020 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:44.723121 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":356074,"prepare.queue_time_us":352426,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":356237,"thread_start_us":292655,"threads_started":1}
I20260709 13:19:44.692864 17348 raft_consensus.cc:697] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:44.781247 12629 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
W20260709 13:19:44.781989 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 48651.6us
W20260709 13:19:44.750245 17571 consensus_meta.cc:287] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.750s	user 0.012s	sys 0.000s
I20260709 13:19:44.792147 12630 raft_consensus.cc:1275] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:19:44.790447 12627 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:44.792770 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:44.794589 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:44.796891 12672 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:44.752425 17637 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:44.748483 17128 raft_consensus.cc:697] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:44.745988 17684 raft_consensus.cc:697] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:44.746304 17455 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:19:44.799378 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 82843us
W20260709 13:19:44.808290 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 84915.6us
I20260709 13:19:44.809446 12629 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:19:44.777624 17510 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.781673 17708 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:44.795418 17529 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.795718 17571 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:44.808085 17644 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.815376 12634 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:44.818394 17510 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:44.798755 17269 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:44.807893 17527 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:19:44.818779 17681 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:44.844429 13380 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:44.831178 17684 consensus_queue.cc:237] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:44.767628 17526 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:44.813370 17455 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:44.816947 17708 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:44.843678 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 46894.9us
I20260709 13:19:44.814455 17454 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:44.848591 12597 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:44.853055 17505 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:19:44.856938 17510 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:44.857682 17509 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:44.858166 17511 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.858516 17274 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:44.847177 17137 consensus_meta.cc:287] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.767s	user 0.011s	sys 0.003s
I20260709 13:19:44.883113 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:44.845446 17534 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.884994 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:44.899646 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 48123.8us
I20260709 13:19:44.888325 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:44.860863 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:44.874089 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:44.864871 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:44.899674 12598 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:44.905311 12781 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:44.922286 12776 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:44.905709 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:44.922057 12780 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:44.905989 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:44.922957 12774 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:19:44.922979 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:44.851052 17455 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:44.926930 12780 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:19:44.930907 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 122311us
I20260709 13:19:44.901060 17641 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:44.900943 17692 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:44.901340 17532 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:44.851243 17546 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:44.903163 17664 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.844267 17644 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:44.905319 17701 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:44.967844 17701 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:44.851920 17348 consensus_queue.cc:237] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:44.847563 17269 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:44.955340 17436 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.844547 17529 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:44.931154 17543 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:44.921115 17137 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:44.959754 17716 log.cc:927] Time spent T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:19:44.960258 17640 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:44.960829 17571 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:44.958168 17546 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:44.903456 17509 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:44.927928 17454 raft_consensus.cc:697] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:44.928280 17534 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:44.958518 17692 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:44.967147 17644 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:44.976784 17436 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:44.977802 17729 log.cc:927] Time spent T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:19:44.975915 17580 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:44.940709 17641 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:45.007308 17641 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:45.007499 17693 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:45.009963 17454 consensus_queue.cc:237] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:44.905135 17511 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:45.016016 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:44.986310 12598 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:45.016721 12629 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
W20260709 13:19:45.017838 13380 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51754 (request call id 875) took 1244 ms (client timeout 1640 ms). Trace:
W20260709 13:19:45.018252 13380 rpcz_store.cc:269] 0709 13:19:43.773437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.774771 (+  1334us) service_pool.cc:224] Handling call
0709 13:19:45.017678 (+1242907us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b540051fc54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:45.017783 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1783760256}
I20260709 13:19:45.019037 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:45.019709 12630 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 17.
I20260709 13:19:45.020382 17662 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:45.020797 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:44.921197 17128 consensus_queue.cc:237] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:45.026520 17727 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:19:44.951869 17321 consensus_queue.cc:1048] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:19:44.959697 17664 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:44.958822 17532 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:45.026603 12775 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:45.034488 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:44.991446 17528 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:45.060959 12597 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:44.994936 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 63387.5us
I20260709 13:19:45.067716 12597 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:45.052981 12776 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 8.
I20260709 13:19:45.084756 12775 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
W20260709 13:19:45.089833 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.999791 17543 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:19:45.103475 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:45.106323 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:45.107503 12672 leader_election.cc:336] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Timed out: RequestConsensusVote RPC to 127.11.63.68:36021 timed out after 2.010s (SENT)
I20260709 13:19:45.107965 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:45.108901 12779 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:45.063751 17140 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:45.112048 12779 raft_consensus.cc:1240] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:45.118237 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:45.087177 17640 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:19:45.122385 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 188121us
I20260709 13:19:45.064479 17526 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:45.083444 17528 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:45.192291 17528 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:45.193698 17546 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:45.084105 17678 consensus_queue.cc:1059] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:19:45.195250 17678 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:45.196183 17678 consensus_queue.cc:1059] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:45.208304 17708 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:45.214432 13377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51754 (request call id 877) took 1440 ms. Trace:
I20260709 13:19:45.215030 13377 rpcz_store.cc:276] 0709 13:19:43.774218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.774400 (+   182us) service_pool.cc:224] Handling call
0709 13:19:45.214184 (+1439784us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b54005f0354. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:45.214354 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1784398080}
W20260709 13:19:45.222018 17680 log.cc:927] Time spent T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 1.371s	user 0.008s	sys 0.005s
I20260709 13:19:45.223898 12701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1944) took 2003 ms. Trace:
I20260709 13:19:45.224399 12701 rpcz_store.cc:276] 0709 13:19:43.220231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.220585 (+   354us) service_pool.cc:224] Handling call
0709 13:19:43.220895 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:43.986290 (+765395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.223589 (+1237299us) spinlock_profiling.cc:243] Waited 765 ms on lock 0x7b5400107cd4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:45.223827 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1758582144}
I20260709 13:19:45.227725 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:45.234084 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:45.242017 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:45.246852 17619 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:45.250097 17701 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:45.251670 13381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 870) took 2289 ms. Trace:
I20260709 13:19:45.302052 13381 rpcz_store.cc:276] 0709 13:19:42.961989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.962149 (+   160us) service_pool.cc:224] Handling call
0709 13:19:44.482024 (+1519875us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.511006 (+ 28982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.251364 (+740358us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54005415d0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:45.251538 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3456570752}
I20260709 13:19:45.263057 17623 raft_consensus.cc:3055] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:45.304088 17623 raft_consensus.cc:740] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:45.305258 17623 consensus_queue.cc:260] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:45.307277 17701 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:45.308023 17701 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:45.129279 17640 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:45.349395 12632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 3112) took 1990 ms. Trace:
I20260709 13:19:45.125809 17739 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:45.363668 12632 rpcz_store.cc:276] 0709 13:19:43.359152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.970796 (+611644us) service_pool.cc:224] Handling call
0709 13:19:43.971126 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:44.716340 (+745214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:44.716370 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:44.716816 (+   446us) log.cc:844] Serialized 26 byte log entry
0709 13:19:44.812613 (+ 95797us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:44.812769 (+   156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.812843 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:45.340404 (+527561us) raft_consensus.cc:1685] finished
0709 13:19:45.349171 (+  8767us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:45.349325 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":744819,"raft.queue_time_us":746533,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"thread_start_us":95350,"threads_started":1,"wal-append.queue_time_us":184185}
W20260709 13:19:45.384176 13348 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.660s (SENT) [suppressed 15 similar messages]
W20260709 13:19:45.392830 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 306846us
I20260709 13:19:45.394665 12597 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:45.130484 17140 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:45.397819 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 379713us
I20260709 13:19:45.349668 17529 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:45.392779 17539 consensus_meta.cc:287] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.701s	user 0.011s	sys 0.003s
W20260709 13:19:45.409600 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.378010 17640 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:45.407522 12597 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:45.382112 17511 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:45.462733 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 55289.4us
I20260709 13:19:45.465226 13349 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:45.466538 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:45.389809 17492 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:45.470945 12626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 876) took 2440 ms. Trace:
I20260709 13:19:45.471335 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:45.407078 17738 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 13:19:45.463768 12597 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:45.477236 13377 raft_consensus.cc:2393] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 13: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:45.431668 17543 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:45.477411 12597 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:45.482626 17736 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:45.483214 17736 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:45.483664 17736 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:45.468533 17137 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:45.385622 17739 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:45.493902 17739 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:45.494668 17739 raft_consensus.cc:3055] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:45.495056 17739 raft_consensus.cc:740] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:45.496052 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:45.474675 17511 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:45.498744 17511 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:45.499137 17511 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:45.499946 12778 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:45.470021 17691 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:19:45.502012 17664 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:45.504160 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:45.506356 17692 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:45.470163 17086 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:45.485685 17733 log.cc:927] Time spent T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:19:45.474411 12626 rpcz_store.cc:276] 0709 13:19:42.998074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.998776 (+   702us) service_pool.cc:224] Handling call
0709 13:19:42.999053 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:43.831657 (+832604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.438202 (+1606545us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b540027b854. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:45.438430 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1912768896}
W20260709 13:19:45.510363 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.512468 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:45.437817 17436 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:45.548435 12776 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:19:45.570713 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:19:45.574360 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 54705.9us
W20260709 13:19:45.592028 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.481029 13377 raft_consensus.cc:2955] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Committing config change with OpId 9.9: config changed from index 4 to 9, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:45.618283 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:45.620426 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:45.472393 17581 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:45.476725 17307 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:45.436939 17539 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:45.607519 17708 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:45.603099 17543 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:45.630725 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:45.627597 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:45.641884 13378 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:45.643002 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:45.647302 13379 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:45.607159 17137 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:45.607465 17692 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:45.629256 17739 consensus_queue.cc:260] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:45.625402 17581 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:45.655494 12782 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.022s	sys 0.059s
W20260709 13:19:45.679837 12782 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.024s	sys 0.060s
I20260709 13:19:45.635938 17307 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:45.681399 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:45.692004 13385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 778) took 1008 ms. Trace:
I20260709 13:19:45.644601 17554 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:45.480782 17492 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:45.608255 17086 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:45.690423 17739 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:45.698952 17532 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:45.699975 17701 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:45.644345 17543 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:45.701751 17543 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:45.660766 17137 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:45.708190 17137 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:45.679399 17692 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:45.710880 17539 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:45.648557 17321 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:19:45.712854 17692 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:45.714323 17543 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:45.715207 17086 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:45.716343 17543 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:45.655624 17662 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.011s	sys 0.017s
I20260709 13:19:45.692363 13385 rpcz_store.cc:276] 0709 13:19:44.683910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.685364 (+  1454us) service_pool.cc:224] Handling call
0709 13:19:44.685684 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:44.692641 (+  6957us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:44.692669 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:44.693097 (+   428us) log.cc:844] Serialized 26 byte log entry
0709 13:19:44.990608 (+297511us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:44.990670 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.990770 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:45.691717 (+700947us) raft_consensus.cc:1685] finished
0709 13:19:45.691746 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:45.691941 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6605,"raft.queue_time_us":8303,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":7988,"thread_start_us":238848,"threads_started":1,"wal-append.queue_time_us":429350}
I20260709 13:19:45.694880 17307 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:45.648641 17708 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:45.677538 17581 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:45.718025 17662 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.012s	sys 0.018s
W20260709 13:19:45.720021 17460 consensus_meta.cc:287] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 1.000s	user 0.012s	sys 0.000s
I20260709 13:19:45.717482 17492 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:45.683208 17691 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
W20260709 13:19:45.726804 17641 consensus_meta.cc:287] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.719s	user 0.010s	sys 0.000s
I20260709 13:19:45.758545 17736 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:45.766464 17708 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:45.770577 17736 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:45.816004 12597 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:19:45.828243 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:45.829386 12600 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.010s	sys 0.001s
I20260709 13:19:45.830042 12628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d46747dede234934bea0f0c324106c28"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:19:45.837834 12632 raft_consensus.cc:1240] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 7. Current term is 8. Ops: []
W20260709 13:19:45.842139 12672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.044s	sys 0.043s
W20260709 13:19:45.842456 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 709050us
I20260709 13:19:45.878718 12778 raft_consensus.cc:1240] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:45.822790 12529 catalog_manager.cc:5697] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:45.882875 12774 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:19:45.883569 12783 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.761s	user 0.009s	sys 0.025s
W20260709 13:19:45.888259 12528 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54792 (request call id 228) took 14634 ms (client timeout 15000 ms). Trace:
I20260709 13:19:45.896270 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:45.897760 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:45.905822 13380 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.000s	sys 0.019s
I20260709 13:19:45.913977 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:45.915249 17086 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:45.919579 12775 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15. Ops: []
W20260709 13:19:45.923349 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 46491.6us
I20260709 13:19:45.924192 13378 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
W20260709 13:19:45.724259 17454 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.018s	sys 0.032s
W20260709 13:19:45.936435 17510 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.011s	sys 0.039s
I20260709 13:19:45.719340 17511 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:45.733879 17537 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:45.937860 17527 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.004s	sys 0.014s
I20260709 13:19:45.938186 17719 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:45.938422 17140 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:45.938840 17558 consensus_meta.cc:287] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 1.317s	user 0.012s	sys 0.001s
W20260709 13:19:45.941854 17314 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.000s	sys 0.020s
W20260709 13:19:46.077909 17314 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.001s	sys 0.020s
I20260709 13:19:46.077992 17558 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:46.078724 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:45.942016 17619 consensus_meta.cc:287] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.644s	user 0.005s	sys 0.005s
I20260709 13:19:45.941217 17630 consensus_queue.cc:1059] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:19:46.080515 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:46.080883 17558 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:46.081259 17505 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.082180 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:46.082003 17505 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:46.012681 12628 raft_consensus.cc:606] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:46.082924 12775 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:46.083189 12628 raft_consensus.cc:614] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:46.084091 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:46.084375 12633 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
W20260709 13:19:46.037494 12672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.044s	sys 0.044s
I20260709 13:19:45.945349 17321 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:46.085457 17321 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:46.088580 17321 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:46.050664 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:46.090945 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:46.092357 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:46.092555 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:46.050462 17537 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:46.094662 12778 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:19:46.095151 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:45.739908 17641 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:46.099223 17543 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:46.100077 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:46.100278 17543 raft_consensus.cc:697] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:19:46.060806 17454 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.019s	sys 0.032s
W20260709 13:19:46.101930 12783 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.982s	user 0.011s	sys 0.026s
I20260709 13:19:46.102566 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:46.103814 12708 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:19:46.051913 13379 raft_consensus.cc:1240] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: []
W20260709 13:19:46.062093 17527 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.004s	sys 0.014s
W20260709 13:19:46.065697 17510 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.929s	user 0.011s	sys 0.040s
I20260709 13:19:46.066687 17719 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:46.108824 17534 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:46.110661 17719 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:19:46.089176 17460 raft_consensus.cc:3055] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:46.111363 17460 raft_consensus.cc:740] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:45.945509 17662 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:46.112984 17662 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:46.114835 17269 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:46.117704 12633 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39590 (request call id 879) took 3019 ms (client timeout 1553 ms). Trace:
W20260709 13:19:46.118257 12633 rpcz_store.cc:269] 0709 13:19:43.098490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.098906 (+   416us) service_pool.cc:224] Handling call
0709 13:19:46.117378 (+3018472us) spinlock_profiling.cc:243] Waited 2.92 s on lock 0x7b54002db9d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:46.117592 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6718801280}
W20260709 13:19:46.119788 17455 consensus_meta.cc:287] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 1.189s	user 0.006s	sys 0.015s
I20260709 13:19:46.120258 17455 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:46.121701 12781 raft_consensus.cc:1240] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: []
W20260709 13:19:46.122920 17750 log.cc:927] Time spent T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.368s	user 0.003s	sys 0.000s
W20260709 13:19:46.041599 12600 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.011s	sys 0.001s
I20260709 13:19:46.108855 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:46.125000 12700 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
W20260709 13:19:46.125306 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:46.126127 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:46.041168 13380 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.000s	sys 0.020s
I20260709 13:19:46.127512 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:46.071704 17708 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:46.072548 17692 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.085350 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:46.076956 17511 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:45.750777 17684 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.013s	sys 0.026s
I20260709 13:19:46.094712 13376 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:19:46.100100 17641 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:46.066957 12529 catalog_manager.cc:5697] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 9 to 10, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 10 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:46.125031 12781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2832) took 1005 ms. Trace:
I20260709 13:19:46.069957 17532 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:46.013792 12528 rpcz_store.cc:269] 0709 13:19:31.253405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:31.253625 (+   220us) service_pool.cc:224] Handling call
0709 13:19:45.887938 (+14634313us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:37.537434 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:37.538716 (+  1282us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:37.538774 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:19:37.538795 (+    21us) tablet.cc:662] Decoding operations
0709 13:19:37.614784 (+ 75989us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:37.639058 (+ 24274us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:37.646283 (+  7225us) tablet.cc:685] Acquiring locks for 31 operations
0709 13:19:37.786704 (+140421us) tablet.cc:701] Row locks acquired
0709 13:19:37.786761 (+    57us) write_op.cc:260] PREPARE: finished
0709 13:19:37.786964 (+   203us) write_op.cc:270] Start()
0709 13:19:37.787139 (+   175us) write_op.cc:276] Timestamp: P: 1783603177786897 usec, L: 0
0709 13:19:37.787179 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 13:19:37.909763 (+122584us) log.cc:844] Serialized 12092 byte log entry
0709 13:19:39.130536 (+1220773us) op_driver.cc:464] REPLICATION: finished
0709 13:19:39.669756 (+539220us) write_op.cc:301] APPLY: starting
0709 13:19:39.683655 (+ 13899us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:41.127492 (+1443837us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:41.127614 (+   122us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:41.656201 (+528587us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:41.663869 (+  7668us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=62,key_file_lookups=62,delta_file_lookups=62,mrs_lookups=0
0709 13:19:41.663972 (+   103us) write_op.cc:312] APPLY: finished
0709 13:19:41.664962 (+   990us) log.cc:844] Serialized 268 byte log entry
0709 13:19:41.865388 (+200426us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:41.883623 (+ 18235us) write_op.cc:454] Released schema lock
0709 13:19:41.888569 (+  4946us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:41.902238 (+ 13669us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:41.902275 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":335114,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":476414,"apply.run_cpu_time_us":128396,"apply.run_wall_time_us":2261598,"cfile_cache_hit":126,"cfile_cache_hit_bytes":133831,"num_ops":31,"prepare.queue_time_us":269895,"prepare.run_cpu_time_us":28534,"prepare.run_wall_time_us":692357,"raft.queue_time_us":614029,"raft.run_cpu_time_us":46708,"raft.run_wall_time_us":328728,"replication_time_us":1343160,"thread_start_us":1250817,"threads_started":5,"wal-append.queue_time_us":598775,"wal-append.run_cpu_time_us":34885,"wal-append.run_wall_time_us":2690727}]]}
W20260709 13:19:46.135516 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 46023.6us
W20260709 13:19:46.141664 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 44231.2us
I20260709 13:19:46.141180 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:46.144345 12700 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:19:46.144256 17692 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:46.145639 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:46.146756 17321 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:46.147550 17692 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:46.149442 17539 consensus_queue.cc:1059] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:46.149461 17492 consensus_queue.cc:1059] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:19:46.151285 17708 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:46.151865 17492 raft_consensus.cc:3055] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:46.152318 17492 raft_consensus.cc:740] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:46.153700 17678 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:46.150673 17539 raft_consensus.cc:3055] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:19:46.141134 17684 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.014s	user 0.013s	sys 0.028s
I20260709 13:19:46.137398 12781 rpcz_store.cc:276] 0709 13:19:45.120024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.120158 (+   134us) service_pool.cc:224] Handling call
0709 13:19:45.120420 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:46.122141 (+1001721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.124662 (+  2521us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b540034a654. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:46.124951 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2301812992}
I20260709 13:19:46.112737 17460 consensus_queue.cc:260] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:46.154481 17678 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:46.154481 17539 raft_consensus.cc:740] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:46.153489 17492 consensus_queue.cc:260] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:46.110563 17086 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:46.147620 17321 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:46.185938 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:46.197320 12597 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:46.206195 12780 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.020s	sys 0.030s
I20260709 13:19:46.165283 12529 catalog_manager.cc:5697] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 9 to 12, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 12 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
W20260709 13:19:46.206614 12780 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.020s	sys 0.030s
I20260709 13:19:46.211206 17678 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:46.188263 17662 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:46.213668 17539 consensus_queue.cc:260] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:46.215442 17539 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:46.216869 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 50488.2us
W20260709 13:19:46.217816 17734 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.032s	sys 0.031s
W20260709 13:19:46.218384 17734 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.032s	sys 0.031s
I20260709 13:19:46.219225 17321 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:46.197022 17543 consensus_queue.cc:237] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:46.220474 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:46.221602 12700 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
W20260709 13:19:46.222157 12779 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.018s	sys 0.032s
W20260709 13:19:46.222571 12779 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.723s	user 0.018s	sys 0.032s
I20260709 13:19:46.216871 17623 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:46.218264 12673 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:46.226531 17644 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:19:46.227854 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.229874 12746 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:19:46.230703 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 67687.3us
I20260709 13:19:46.230926 17692 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:46.231820 17692 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.232446 17692 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:46.233705 12673 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:46.233968 17536 consensus_queue.cc:1059] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:19:46.234927 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 59427.4us
I20260709 13:19:46.235065 17536 raft_consensus.cc:3055] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:46.235551 17536 raft_consensus.cc:740] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:46.237265 17548 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:46.237914 17548 raft_consensus.cc:697] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:46.238317 12782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2826) took 1127 ms. Trace:
I20260709 13:19:46.238500 17455 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:46.191380 17532 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:46.205720 17708 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:46.239403 17548 consensus_queue.cc:237] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:46.240576 17223 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.064s	sys 0.063s
I20260709 13:19:46.204391 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:46.246940 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 80939.8us
W20260709 13:19:46.209239 17640 consensus_meta.cc:287] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.739s	user 0.011s	sys 0.006s
I20260709 13:19:46.256695 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:46.233139 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:46.247385 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:46.242702 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:46.245766 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:46.246395 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:46.256675 12673 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:46.265846 12708 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 17.
I20260709 13:19:46.247447 12529 catalog_manager.cc:5697] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 6 to 9, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 9 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:19:46.246901 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:46.256645 17321 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:46.246196 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:46.254820 17581 consensus_meta.cc:287] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.527s	user 0.012s	sys 0.000s
I20260709 13:19:46.245075 17692 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:46.273531 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:46.242214 17492 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:46.270264 12597 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:46.257364 17505 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:46.236874 17536 consensus_queue.cc:260] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:46.241480 17532 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:46.269853 17223 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.887s	user 0.065s	sys 0.065s
I20260709 13:19:46.255201 17738 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:46.251166 17086 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:46.290941 17137 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:46.292409 17536 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:46.302331 17321 raft_consensus.cc:697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:46.306118 12705 raft_consensus.cc:2410] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:19:46.306950 17321 consensus_queue.cc:237] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:46.297644 13382 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.008s	sys 0.006s
W20260709 13:19:46.309828 13382 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.008s	sys 0.007s
W20260709 13:19:46.298029 13381 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.710s	user 0.008s	sys 0.007s
I20260709 13:19:46.310633 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:46.311008 13381 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.827s	user 0.008s	sys 0.008s
I20260709 13:19:46.238821 12782 rpcz_store.cc:276] 0709 13:19:45.111148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.113685 (+  2537us) service_pool.cc:224] Handling call
0709 13:19:45.114072 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:45.114455 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:19:45.132649 (+ 18194us) log.cc:844] Serialized 14 byte log entry
0709 13:19:45.916356 (+783707us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:45.917368 (+  1012us) log.cc:844] Serialized 26 byte log entry
0709 13:19:45.917648 (+   280us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:45.917707 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.917876 (+   169us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:46.237907 (+320031us) raft_consensus.cc:1685] finished
0709 13:19:46.237928 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:46.238103 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":245088,"prepare.queue_time_us":18046,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":174316,"raft.queue_time_us":246278,"raft.run_cpu_time_us":510,"raft.run_wall_time_us":6785,"thread_start_us":551439,"threads_started":1,"wal-append.queue_time_us":1101593}
I20260709 13:19:46.292136 12529 catalog_manager.cc:5697] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 5 to 6, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 6 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
W20260709 13:19:46.297659 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 370596us
I20260709 13:19:46.323518 12706 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:46.324602 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:46.274286 17640 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:46.344188 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:46.336944 12529 catalog_manager.cc:5697] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 8 to 9, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 9 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
W20260709 13:19:46.350087 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 81271us
I20260709 13:19:46.351923 12703 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:46.288771 17581 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:46.355309 17137 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.355737 17581 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:46.356066 17137 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:46.358227 17137 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:46.362166 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 105626us
W20260709 13:19:46.366062 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.367935 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:46.379379 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 146390us
I20260709 13:19:46.343521 17532 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:19:46.395125 17307 consensus_meta.cc:287] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.670s	user 0.014s	sys 0.001s
I20260709 13:19:46.395614 17307 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:46.396052 17537 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:46.397830 13376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51762 (request call id 817) took 3548 ms. Trace:
I20260709 13:19:46.398113 17731 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:46.398239 13376 rpcz_store.cc:276] 0709 13:19:42.849726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.874188 (+ 24462us) service_pool.cc:224] Handling call
0709 13:19:42.874493 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:46.143700 (+3269207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.397477 (+253777us) spinlock_profiling.cc:243] Waited 3.22 s on lock 0x7b54003ab954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:46.397728 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7404295552}
W20260709 13:19:46.399290 17748 log.cc:927] Time spent T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 13:19:46.408730 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:46.408993 13383 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.013s	sys 0.011s
I20260709 13:19:46.409811 12783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2831) took 1291 ms. Trace:
W20260709 13:19:46.421344 13383 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.013s	sys 0.011s
I20260709 13:19:46.410315 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:46.424642 12633 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
W20260709 13:19:46.383386 17464 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.900s	user 0.006s	sys 0.056s
I20260709 13:19:46.428260 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:46.430413 17464 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.947s	user 0.006s	sys 0.056s
W20260709 13:19:46.434159 17753 log.cc:927] Time spent T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.221s	user 0.002s	sys 0.000s
I20260709 13:19:46.437207 12777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2827) took 1322 ms. Trace:
I20260709 13:19:46.437708 12777 rpcz_store.cc:276] 0709 13:19:45.114992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.115134 (+   142us) service_pool.cc:224] Handling call
0709 13:19:45.133802 (+ 18668us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:45.186310 (+ 52508us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:45.186354 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:45.186931 (+   577us) log.cc:844] Serialized 26 byte log entry
0709 13:19:45.654230 (+467299us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:45.654284 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.654384 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:46.433224 (+778840us) spinlock_profiling.cc:243] Waited 34.3 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:46.436672 (+  3448us) raft_consensus.cc:1685] finished
0709 13:19:46.436699 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:46.437010 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":48141,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":115,"spinlock_wait_cycles":78928256,"thread_start_us":466845,"threads_started":1,"wal-append.queue_time_us":1025048}
I20260709 13:19:46.442639 17460 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:46.452937 12707 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: []
W20260709 13:19:46.456727 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 46065us
I20260709 13:19:46.450769 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:46.459439 17692 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:46.459884 17436 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:46.460346 17436 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:46.460230 17692 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:46.460740 17436 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:46.417946 12783 rpcz_store.cc:276] 0709 13:19:45.118209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.118777 (+   568us) service_pool.cc:224] Handling call
0709 13:19:45.119018 (+   241us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:45.119339 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:45.119356 (+    17us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:45.119679 (+   323us) log.cc:844] Serialized 26 byte log entry
0709 13:19:45.213448 (+ 93769us) spinlock_profiling.cc:243] Waited 5.2 ms on lock 0x7b4800042258. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c722dbaa1 00007f4c722d2b37 00007f4c722d29ca 00007f4c723ca26f 00007f4c723c9f37 00007f4c723de55c 00007f4c723de508 00007f4c723de488 00007f4c723de443 00007f4c723dd6ec 00007f4c71307a96
0709 13:19:46.102909 (+889461us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:46.103007 (+    98us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.103155 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:46.398375 (+295220us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:46.406510 (+  8135us) raft_consensus.cc:1685] finished
0709 13:19:46.406531 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:46.409738 (+  3207us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3343,"raft.run_cpu_time_us":2442,"raft.run_wall_time_us":87540,"spinlock_wait_cycles":63970816,"thread_start_us":982658,"threads_started":1,"wal-append.queue_time_us":1097194}
I20260709 13:19:46.485417 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:46.487243 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:46.487115 12529 catalog_manager.cc:5697] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 8 to 12, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 12 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:46.458395 17307 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:46.482314 12704 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:19:46.467173 17692 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:46.502117 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:46.504002 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:46.492527 12597 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:46.506089 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:46.509388 12706 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:19:46.508585 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:46.512804 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:46.513707 12781 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:46.514029 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 120871us
W20260709 13:19:46.517661 17128 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.019s	sys 0.038s
I20260709 13:19:46.518164 17509 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:46.518237 12598 leader_election.cc:336] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.591s (SENT)
I20260709 13:19:46.518592 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:46.518908 12598 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:46.519254 13385 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:46.495451 17505 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:46.529809 17505 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:46.518030 17128 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.019s	sys 0.038s
W20260709 13:19:46.532517 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 66607us
I20260709 13:19:46.538419 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:46.539161 12706 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13. Ops: [12.8-12.8]
W20260709 13:19:46.540140 17771 log.cc:927] Time spent T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:19:46.539186 12529 catalog_manager.cc:5697] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 6 to 9, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 9 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:46.545739 17678 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:46.520221 17511 consensus_queue.cc:1059] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:19:46.546216 17678 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:46.546659 17511 raft_consensus.cc:3055] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:46.547076 17511 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:46.551120 12708 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:19:46.551961 12708 raft_consensus.cc:606] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:19:46.552398 12708 raft_consensus.cc:614] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:46.556540 12781 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
W20260709 13:19:46.557286 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 150843us
I20260709 13:19:46.546679 17678 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:46.546233 17505 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:46.547996 17509 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:19:46.574152 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:46.600138 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 44534.4us
I20260709 13:19:46.575516 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:19:46.576293 17646 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:46.558396 17640 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:46.556350 17511 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:46.554266 17537 consensus_queue.cc:1059] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:19:46.580876 13383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51762 (request call id 832) took 1071 ms. Trace:
I20260709 13:19:46.611182 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:46.580691 17348 consensus_queue.cc:1048] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:46.581593 17546 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:46.582576 12673 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:46.583439 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:19:46.586014 12598 leader_election.cc:336] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.505s (SENT)
I20260709 13:19:46.598718 17436 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:46.617684 12598 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:46.618072 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:46.625597 12780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 656) took 1131 ms. Trace:
I20260709 13:19:46.632961 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:46.641314 12633 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:46.626169 12780 rpcz_store.cc:276] 0709 13:19:45.493897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.603850 (+109953us) service_pool.cc:224] Handling call
0709 13:19:45.604284 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:45.604572 (+   288us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:45.604612 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:45.622704 (+ 18092us) log.cc:844] Serialized 26 byte log entry
0709 13:19:46.207162 (+584458us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:46.207229 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.207325 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:46.599964 (+392639us) raft_consensus.cc:1685] finished
0709 13:19:46.599992 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:46.625493 (+ 25501us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":553639,"threads_started":1,"wal-append.queue_time_us":865829}
I20260709 13:19:46.611656 13383 rpcz_store.cc:276] 0709 13:19:45.509871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.509994 (+   123us) service_pool.cc:224] Handling call
0709 13:19:45.619744 (+109750us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:45.620209 (+   465us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:45.620252 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:45.627285 (+  7033us) log.cc:844] Serialized 26 byte log entry
0709 13:19:46.502376 (+875091us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:46.520005 (+ 17629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.536754 (+ 16749us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:46.579993 (+ 43239us) spinlock_profiling.cc:243] Waited 9.52 ms on lock 0x7b540001b6a0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d327dbe 00007f4c6d332930 00007f4c6d3328d8 00007f4c6d332858 00007f4c6d332813 00007f4c6d331a9c 00007f4c7321d626 00007f4c73217dd8 00007f4c6d36e7cb 00007f4c6d3712c0
0709 13:19:46.580554 (+   561us) raft_consensus.cc:1685] finished
0709 13:19:46.580573 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:46.580781 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10240,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":6131,"spinlock_wait_cycles":21896576,"thread_start_us":794552,"threads_started":1,"wal-append.queue_time_us":858370}
I20260709 13:19:46.621171 17511 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:46.647229 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:46.648716 12634 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:46.650439 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:46.651208 12706 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:46.652657 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:46.655639 13349 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:46.659133 12628 raft_consensus.cc:2410] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 15 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
W20260709 13:19:46.659327 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.670535 13382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51762 (request call id 833) took 1138 ms. Trace:
I20260709 13:19:46.649364 17640 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:46.675729 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:46.678660 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:46.678833 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 92883.5us
I20260709 13:19:46.679617 12782 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:46.685900 12783 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:46.684015 13381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51762 (request call id 828) took 1214 ms. Trace:
I20260709 13:19:46.687752 12706 raft_consensus.cc:2410] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:19:46.674906 17734 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:46.687067 12779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 658) took 1192 ms. Trace:
I20260709 13:19:46.690155 12747 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:46.689914 12781 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:19:46.677934 17307 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:46.671124 13382 rpcz_store.cc:276] 0709 13:19:45.532132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.532899 (+   767us) service_pool.cc:224] Handling call
0709 13:19:45.533320 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:45.606007 (+ 72687us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:45.606053 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:45.629959 (+ 23906us) log.cc:844] Serialized 26 byte log entry
0709 13:19:46.310378 (+680419us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:46.607626 (+297248us) log.cc:844] Serialized 14 byte log entry
0709 13:19:46.645673 (+ 38047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.660413 (+ 14740us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:46.660501 (+    88us) raft_consensus.cc:1685] finished
0709 13:19:46.660517 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:46.670446 (+  9929us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":105692,"prepare.queue_time_us":239416,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":99885,"raft.run_cpu_time_us":324,"raft.run_wall_time_us":26962,"thread_start_us":953010,"threads_started":2,"wal-append.queue_time_us":737990}
W20260709 13:19:46.697041 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.687642 17505 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:19:46.702929 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 67412.7us
I20260709 13:19:46.703280 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:46.703835 17436 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:46.704946 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 122542us
W20260709 13:19:46.710638 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.711313 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:46.686489 13381 rpcz_store.cc:276] 0709 13:19:45.469229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.469406 (+   177us) service_pool.cc:224] Handling call
0709 13:19:45.483610 (+ 14204us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:45.483881 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:46.374558 (+890677us) log.cc:844] Serialized 14 byte log entry
0709 13:19:46.641880 (+267322us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:46.648443 (+  6563us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:46.661278 (+ 12835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.680193 (+ 18915us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:46.683952 (+  3759us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1250,"prepare.queue_time_us":970734,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":34024,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":22267,"thread_start_us":1014030,"threads_started":2,"wal-append.queue_time_us":302154}
I20260709 13:19:46.694024 17307 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:46.698963 17734 raft_consensus.cc:697] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:46.691871 12779 rpcz_store.cc:276] 0709 13:19:45.494595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:45.497332 (+  2737us) service_pool.cc:224] Handling call
0709 13:19:45.497738 (+   406us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:45.498191 (+   453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:45.498229 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:45.498906 (+   677us) log.cc:844] Serialized 52 byte log entry
0709 13:19:46.223139 (+724233us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:46.223531 (+   392us) log.cc:844] Serialized 14 byte log entry
0709 13:19:46.242795 (+ 19264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.242936 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:46.686841 (+443905us) raft_consensus.cc:1685] finished
0709 13:19:46.686864 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:46.687011 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18922,"prepare.queue_time_us":201,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":24889,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":244,"thread_start_us":723651,"threads_started":1,"wal-append.queue_time_us":981205}
I20260709 13:19:46.710001 17536 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:46.736946 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:46.739233 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 63792.8us
I20260709 13:19:46.717113 17307 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:46.722036 17505 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:46.762463 12597 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:46.769200 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:46.777221 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.770596 12597 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:46.791182 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:46.796628 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 57236.6us
I20260709 13:19:46.800860 12597 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:46.801947 17662 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:46.807479 17662 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:46.808081 17662 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:46.808194 17539 consensus_meta.cc:287] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.592s	user 0.012s	sys 0.004s
I20260709 13:19:46.808740 17678 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:46.782404 17640 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:46.824906 12706 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:46.818408 12707 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
W20260709 13:19:46.838757 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 150035us
W20260709 13:19:46.841339 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 140871us
I20260709 13:19:46.781549 17505 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:46.843017 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:46.851699 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:46.846122 17492 consensus_meta.cc:287] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.545s	user 0.012s	sys 0.001s
I20260709 13:19:46.854866 12628 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:46.843125 17543 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:46.870451 17543 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:46.870922 17543 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:46.873492 17777 log.cc:927] Time spent T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
W20260709 13:19:46.874666 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:46.875504 17784 log.cc:927] Time spent T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.113s	user 0.000s	sys 0.002s
I20260709 13:19:46.881880 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:46.885888 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:46.869280 17526 raft_consensus.cc:3055] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:46.886713 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:46.886797 17526 raft_consensus.cc:740] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:46.887336 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:46.887956 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:46.888782 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:46.890113 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:46.890946 12708 raft_consensus.cc:2393] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 16: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:46.891824 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:46.892447 12703 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:46.893268 17623 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:46.890103 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:46.886062 17678 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:46.910434 17776 log.cc:927] Time spent T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 13:19:46.913393 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:46.918746 17780 log.cc:927] Time spent T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:19:46.925361 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:46.904702 17581 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.926338 17581 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:46.859236 17734 consensus_queue.cc:237] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:46.935930 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:46.936789 13381 raft_consensus.cc:2410] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:19:46.937628 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:46.940160 12709 raft_consensus.cc:1240] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:46.893899 17623 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:46.941332 12705 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:19:46.941759 12705 pending_rounds.cc:85] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 6
I20260709 13:19:46.941991 12705 pending_rounds.cc:107] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:19:46.942278 12705 raft_consensus.cc:2889] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:46.943373 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:46.943303 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:46.944583 12708 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:46.955513 13382 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:46.960392 17321 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.948122 17526 consensus_queue.cc:260] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:46.961022 17321 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:46.963301 17321 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:46.965813 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:46.967736 12700 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:46.972839 12708 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:19:46.960620 17623 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:46.979301 12634 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:19:46.979792 12633 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 18.
I20260709 13:19:46.979758 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:46.983703 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:46.999576 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:46.999899 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 204314us
I20260709 13:19:46.934641 17505 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:47.000676 12597 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:47.009750 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:47.025647 12632 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:47.023037 17321 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:47.029425 12701 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:47.029647 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:47.029793 17321 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:47.030396 17321 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:47.031716 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:47.032025 12777 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:19:47.035172 12579 auto_rebalancer.cc:893] Removing replace marker failed: Replica 19cd864c053744bebd96093a7982537e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:19:47.035732 12579 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:19:47.036110 12579 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:19:47.042982 17321 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:47.044309 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:47.044996 12634 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:47.046425 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:47.046742 17731 consensus_meta.cc:287] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.648s	user 0.018s	sys 0.000s
I20260709 13:19:47.047152 17731 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:47.047466 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:47.014637 17526 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:47.046119 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:47.049348 12626 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
W20260709 13:19:47.052737 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.053443 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:47.053594 17678 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:47.054785 17678 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:47.047158 12628 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:47.057327 12627 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:47.059196 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:47.059979 12626 raft_consensus.cc:2368] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14.
I20260709 13:19:47.046072 17543 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:47.061496 17581 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:47.062207 17543 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:47.065629 12774 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 2457) took 1140 ms. Trace:
I20260709 13:19:47.066130 12631 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:47.066493 17662 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:47.066030 12774 rpcz_store.cc:276] 0709 13:19:45.924973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.095276 (+170303us) service_pool.cc:224] Handling call
0709 13:19:46.095850 (+   574us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:46.146128 (+ 50278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:46.146164 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:46.146838 (+   674us) log.cc:844] Serialized 52 byte log entry
0709 13:19:46.562188 (+415350us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:46.562240 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:46.587981 (+ 25741us) spinlock_profiling.cc:243] Waited 49.8 ms on lock 0x7b5400128754. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:46.588033 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:47.065288 (+477255us) raft_consensus.cc:1685] finished
0709 13:19:47.065313 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:47.065520 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1311,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":138,"spinlock_wait_cycles":114437888,"thread_start_us":414854,"threads_started":1,"wal-append.queue_time_us":915924}
I20260709 13:19:47.067449 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:47.067991 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 44193.4us
I20260709 13:19:47.068210 12631 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:19:47.069327 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:47.070014 12627 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:47.066942 12628 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:47.071550 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:47.071628 13349 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:47.013629 17137 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:19:47.074188 17678 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:47.074759 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:47.075336 12775 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:47.082309 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:47.083957 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:47.084728 12702 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 15: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:47.084677 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:47.083844 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:47.085316 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:47.085903 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:47.086197 17321 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:47.086409 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:47.086652 12780 raft_consensus.cc:2393] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 13: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:47.089198 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:47.089823 13383 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:47.091796 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:47.092455 12774 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:47.092916 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:47.094111 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:47.094326 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:47.094651 12777 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:47.095614 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:47.096174 12778 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:47.096362 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:47.092131 13381 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:19:47.098047 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:47.098748 12776 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:19:47.099172 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:47.099517 12783 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:19:47.100296 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:47.101014 12774 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:19:47.101547 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:47.102190 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:47.102649 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:47.102895 12633 raft_consensus.cc:2393] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 13: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:47.103459 12626 raft_consensus.cc:2410] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:19:47.103647 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:47.105629 12625 raft_consensus.cc:2410] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:19:47.108217 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:47.108922 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:47.098117 12780 raft_consensus.cc:1240] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:47.110739 12778 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:47.137787 12625 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 18: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:19:47.149027 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.154341 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:47.162093 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.151865 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:47.149200 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:47.171149 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 79760.4us
I20260709 13:19:47.175509 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:47.178373 12777 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:19:47.178550 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:47.179031 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:47.179750 12633 raft_consensus.cc:2393] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 14: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:47.139575 17460 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:47.191743 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:47.170174 17640 raft_consensus.cc:697] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:47.196734 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:47.198336 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:47.193276 12673 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:47.139725 17623 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
W20260709 13:19:47.202178 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 50395us
I20260709 13:19:47.209334 12673 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:47.236837 17678 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:19:47.239470 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.234477 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:47.196480 17086 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:47.250305 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:47.245580 12626 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:19:47.242882 13348 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:47.236083 17772 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:19:47.253322 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 50635.6us
I20260709 13:19:47.208117 13378 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:19:47.238281 17734 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:47.207578 12597 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:19:47.266680 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 59149.3us
I20260709 13:19:47.268150 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:47.250118 17128 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:47.268945 12626 raft_consensus.cc:2393] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 15: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:47.278656 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:47.277204 17532 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:47.279929 17511 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.280301 17532 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:47.280937 17532 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:47.283026 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:47.283968 12625 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 15: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:47.286734 12672 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:47.207867 17460 raft_consensus.cc:697] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:47.287732 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:47.289183 12781 raft_consensus.cc:1240] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:47.279621 17773 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:47.292102 17773 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:47.292526 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:47.280527 17511 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:47.283979 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:47.293712 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 84483.2us
I20260709 13:19:47.293732 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:47.295358 17511 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:47.295637 12707 raft_consensus.cc:2393] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 17: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:47.270759 12597 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:47.296640 12597 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:47.297739 17623 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:47.298780 17128 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:47.302109 17773 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:47.302600 12597 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:47.303207 17086 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:47.306013 17086 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:47.310482 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:47.311617 17532 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:47.312511 12672 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:47.313768 17532 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:47.315107 17460 consensus_queue.cc:237] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:47.318488 17719 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.319159 17719 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:47.319715 17684 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:47.320225 17684 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:47.320664 17684 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:47.322324 12780 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:47.322474 17719 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:47.258824 17526 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:47.323515 17526 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:47.324012 17526 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:47.326026 17731 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:47.256528 17734 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:47.327129 17734 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:47.328440 17731 consensus_queue.cc:1059] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:47.330379 17128 raft_consensus.cc:3055] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:47.330868 17128 raft_consensus.cc:740] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:19:47.332162 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 61226.1us
I20260709 13:19:47.332026 17128 consensus_queue.cc:260] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:47.336473 17086 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.336699 13349 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:47.337431 17086 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:47.338158 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 60770.2us
I20260709 13:19:47.337949 17719 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:47.339701 17719 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:47.340092 13349 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:47.340984 17719 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:47.341027 17801 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:47.341629 17719 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:47.268605 17772 raft_consensus.cc:697] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:47.342152 17719 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:47.342505 17532 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:47.343055 17532 raft_consensus.cc:697] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:47.343401 17128 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:47.343101 17772 consensus_queue.cc:237] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:47.344066 17532 consensus_queue.cc:237] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:47.336473 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:47.341805 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 44288.6us
I20260709 13:19:47.345929 17640 consensus_queue.cc:237] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:47.348026 17678 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:47.348651 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:47.270247 17662 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:47.350205 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:47.341928 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:47.341908 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 44025.6us
I20260709 13:19:47.280377 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:47.284915 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:47.374506 12775 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:47.376415 12631 raft_consensus.cc:2393] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 14: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:47.365334 17543 consensus_queue.cc:1059] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 13:19:47.378863 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:47.379055 12632 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:19:47.379170 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:47.379475 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:47.380172 12633 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:47.380363 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:47.380901 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:47.381206 12627 raft_consensus.cc:1240] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:47.365147 17762 consensus_queue.cc:1059] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:19:47.383646 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:47.341848 17801 raft_consensus.cc:697] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:47.399645 12779 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:47.399981 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:47.402774 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:47.358969 17086 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:47.408417 12598 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:47.414631 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:47.415473 12777 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:47.416152 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:47.416931 12780 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:47.341109 17623 raft_consensus.cc:697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:47.443949 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:47.444861 12633 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
I20260709 13:19:47.452322 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:47.453549 12783 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:47.456802 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:19:47.470119 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 59663us
I20260709 13:19:47.410630 17678 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:47.416210 17539 raft_consensus.cc:3055] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:47.476596 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:47.414435 17646 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:47.479032 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 70731.4us
I20260709 13:19:47.470583 17799 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:47.474990 17539 raft_consensus.cc:740] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:19:47.486212 17307 consensus_meta.cc:287] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.712s	user 0.011s	sys 0.001s
I20260709 13:19:47.520459 12598 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:47.521296 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:47.543812 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:47.544572 12779 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:19:47.457885 17719 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:47.525971 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:47.547217 17719 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:47.547938 13381 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:47.528451 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:47.548151 12777 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:19:47.531975 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:47.480551 17678 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:47.551064 17678 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:47.552047 17539 consensus_queue.cc:260] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:47.553622 17539 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:47.559744 12597 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:47.498929 17646 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:47.565800 17646 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:47.545454 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:47.563149 17684 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:47.579540 13349 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:19:47.582594 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 180509us
I20260709 13:19:47.582651 12597 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:47.601711 12631 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:47.613778 12626 raft_consensus.cc:1240] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: []
W20260709 13:19:47.618386 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 98035.8us
W20260709 13:19:47.618933 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.619292 12598 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:47.616500 12597 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:47.567677 17307 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:47.601446 17543 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:47.604612 17630 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:19:47.659806 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 52236.3us
I20260709 13:19:47.664345 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:47.665246 12632 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:19:47.611636 17678 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:47.678310 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:47.679788 13383 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:47.568953 17623 consensus_queue.cc:237] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:47.694442 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:47.703219 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 90525.2us
I20260709 13:19:47.661777 17509 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.658708 17797 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:19:47.769994 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 271050us
I20260709 13:19:47.701349 12633 raft_consensus.cc:2393] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 11: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:47.661100 17630 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:47.681615 17678 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:19:47.749516 17809 log.cc:927] Time spent T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
I20260709 13:19:47.569119 17801 consensus_queue.cc:237] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:47.671133 17543 raft_consensus.cc:697] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:47.840997 12626 raft_consensus.cc:1240] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: []
W20260709 13:19:47.820735 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 46318us
W20260709 13:19:47.846221 13348 leader_election.cc:336] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.660s (SENT)
I20260709 13:19:47.850919 13348 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:47.743950 17532 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:19:47.853219 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 233986us
I20260709 13:19:47.695060 17731 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.159s
I20260709 13:19:47.845805 17630 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:47.846520 17678 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:47.893839 12598 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:47.830569 17797 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:47.819907 17662 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:19:47.920498 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.831519 17511 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
W20260709 13:19:47.926277 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 45530.5us
I20260709 13:19:47.925630 17460 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:47.902012 17797 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:47.937708 17813 log.cc:927] Time spent T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 13:19:47.939611 17819 log.cc:927] Time spent T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:19:47.939782 12747 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:47.889941 17749 consensus_queue.cc:1059] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
W20260709 13:19:47.941395 17811 log.cc:927] Time spent T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
W20260709 13:19:47.941751 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 116190us
W20260709 13:19:47.947997 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 123158us
I20260709 13:19:47.865327 17509 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:47.932654 17086 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.963562 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:47.947862 17492 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.009s	sys 0.026s
I20260709 13:19:47.959900 17692 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:47.974012 17692 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:47.974387 17820 raft_consensus.cc:3055] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:47.974576 17692 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:47.974947 17820 raft_consensus.cc:740] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:47.975926 17820 consensus_queue.cc:260] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:47.963198 17137 consensus_queue.cc:1059] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
W20260709 13:19:47.972414 17812 log.cc:927] Time spent T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 13:19:47.947862 17684 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:48.017383 17436 raft_consensus.cc:3055] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:48.017913 17436 raft_consensus.cc:740] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:48.018975 17436 consensus_queue.cc:260] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:48.021860 17436 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:47.977479 17820 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:48.043587 17526 consensus_meta.cc:287] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.719s	user 0.012s	sys 0.000s
I20260709 13:19:48.044198 17526 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:48.045959 13379 raft_consensus.cc:2393] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 14: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
W20260709 13:19:48.048714 17734 consensus_meta.cc:287] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.721s	user 0.012s	sys 0.000s
I20260709 13:19:48.049425 17734 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.059927 17734 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:48.060050 17509 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:48.060738 17307 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:47.967907 17543 consensus_queue.cc:237] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.089188 12703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1955) took 1149 ms. Trace:
I20260709 13:19:47.966279 17086 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.089754 12703 rpcz_store.cc:276] 0709 13:19:46.939823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:46.941666 (+  1843us) service_pool.cc:224] Handling call
0709 13:19:46.941959 (+   293us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:46.942429 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:46.942457 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:46.995324 (+ 52867us) log.cc:844] Serialized 52 byte log entry
0709 13:19:47.266255 (+270931us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:47.409204 (+142949us) log.cc:844] Serialized 14 byte log entry
0709 13:19:47.669073 (+259869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:47.669158 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:48.086141 (+416983us) spinlock_profiling.cc:243] Waited 6.17 ms on lock 0x7b180012f820. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c53f3 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:19:48.088798 (+  2657us) raft_consensus.cc:1685] finished
0709 13:19:48.088909 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:48.089086 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49162,"prepare.queue_time_us":144726,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":6744,"raft.queue_time_us":142434,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":147,"spinlock_wait_cycles":14189440,"thread_start_us":435197,"threads_started":2,"wal-append.queue_time_us":450471}
I20260709 13:19:48.091645 17086 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:48.102459 12780 consensus_meta.cc:287] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.637s	user 0.012s	sys 0.000s
I20260709 13:19:48.103143 12780 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:48.112463 13383 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:48.116214 17511 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:48.117058 17511 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:48.120867 17511 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:48.061270 17509 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:19:48.142253 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 211475us
W20260709 13:19:48.147532 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:48.151535 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:48.133670 17128 consensus_meta.cc:287] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.790s	user 0.013s	sys 0.000s
I20260709 13:19:48.124174 17692 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:48.171449 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 231830us
I20260709 13:19:48.171628 12700 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:48.174134 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:48.176510 12781 raft_consensus.cc:1813] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Handling vote request from an unknown peer 19cd864c053744bebd96093a7982537e
I20260709 13:19:48.175026 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:48.177687 12780 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:48.175132 13348 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:48.180081 12781 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 10: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:48.181391 12673 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:48.185231 12747 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:48.190774 12597 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:48.191893 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:48.172495 17464 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:19:48.196969 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.193497 13348 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:48.171171 17532 raft_consensus.cc:697] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:19:48.225438 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.190593 17824 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.236406 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:48.237123 12707 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:19:48.236802 17824 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:48.237457 17838 log.cc:927] Time spent T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.088s	user 0.001s	sys 0.001s
I20260709 13:19:48.238257 12625 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:19:48.239022 17824 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:48.187788 17831 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:48.239449 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:48.240274 12630 raft_consensus.cc:2368] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 13. Current term is 14.
I20260709 13:19:48.239835 17831 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:48.239389 12626 raft_consensus.cc:1275] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
W20260709 13:19:48.236423 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 55153.3us
I20260709 13:19:48.242295 12673 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:48.190896 17511 raft_consensus.cc:3055] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:19:48.244839 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 379457us
I20260709 13:19:48.248466 13348 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:48.244078 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:48.249737 12631 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:48.251451 12634 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:19:48.199302 17772 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:48.232872 17086 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:48.264284 12782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2844) took 1085 ms. Trace:
I20260709 13:19:48.241024 17731 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:48.243345 17646 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.241209 17823 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:48.269394 12673 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:48.246536 17511 raft_consensus.cc:740] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:48.193521 17630 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:48.261373 17773 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:48.260032 17772 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:48.272176 12782 rpcz_store.cc:276] 0709 13:19:47.178624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.178725 (+   101us) service_pool.cc:224] Handling call
0709 13:19:47.238322 (+ 59597us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:47.244505 (+  6183us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:47.517229 (+272724us) log.cc:844] Serialized 14 byte log entry
0709 13:19:48.263913 (+746684us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:48.263971 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:48.264024 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:48.264089 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:48.264222 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9524,"prepare.queue_time_us":21578,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":38635,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":79,"thread_start_us":917802,"threads_started":3,"wal-append.queue_time_us":661619}
I20260709 13:19:48.265004 17762 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:19:48.289386 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 112919us
I20260709 13:19:48.296847 12633 raft_consensus.cc:1275] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:48.314816 12779 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:48.293537 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:48.316279 17630 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:48.316433 13379 raft_consensus.cc:1813] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Handling vote request from an unknown peer 19cd864c053744bebd96093a7982537e
I20260709 13:19:48.316916 13377 raft_consensus.cc:1240] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:48.273873 17731 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:48.327390 13379 raft_consensus.cc:2410] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 13:19:48.327900 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 58596.4us
I20260709 13:19:48.279940 17799 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:48.330789 12673 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:48.270363 17492 consensus_queue.cc:1059] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:19:48.328780 17830 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:19:48.314839 17837 log.cc:927] Time spent T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:19:48.291989 17772 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:48.334962 17835 log.cc:927] Time spent T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:19:48.244557 17464 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:48.339263 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 59596us
I20260709 13:19:48.291527 17762 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:48.326764 17839 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:48.352869 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 459122us
I20260709 13:19:48.353519 12598 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:48.273698 17822 consensus_queue.cc:1059] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:19:48.332347 17731 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:48.288141 17734 consensus_queue.cc:1059] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
W20260709 13:19:48.347083 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 54194.4us
I20260709 13:19:48.287043 17646 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:48.383548 12703 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:48.346160 17799 raft_consensus.cc:697] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:19:48.385726 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 155719us
I20260709 13:19:48.354116 17762 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:48.391386 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.395541 12747 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:48.400833 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 47370.2us
I20260709 13:19:48.401762 12598 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:48.391392 17692 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:48.325764 17532 consensus_queue.cc:237] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.393471 17492 raft_consensus.cc:3055] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:48.415310 13349 leader_election.cc:400] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 13. Current term is 14.
I20260709 13:19:48.419057 13349 leader_election.cc:403] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:19:48.423305 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:48.424042 12627 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:48.358278 17839 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:48.412905 17825 log.cc:927] Time spent T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
I20260709 13:19:48.354768 17511 consensus_queue.cc:260] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.450116 12597 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:48.428093 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:48.451886 12597 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:48.420732 17662 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:48.486761 12631 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
W20260709 13:19:48.487298 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 156631us
I20260709 13:19:48.420828 17719 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 13. Current term is 14.
I20260709 13:19:48.418824 17492 raft_consensus.cc:740] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:48.409255 17823 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 13:19:48.403535 17797 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.437757 17646 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:48.426769 17840 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:19:48.438992 17678 consensus_meta.cc:287] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.533s	user 0.007s	sys 0.005s
I20260709 13:19:48.491614 17662 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:48.425773 17831 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:48.500813 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 110870us
I20260709 13:19:48.494700 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:48.485970 17822 raft_consensus.cc:3055] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:48.494112 17581 consensus_queue.cc:1048] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:19:48.469846 17543 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:48.522442 12673 leader_election.cc:400] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14.
I20260709 13:19:48.543054 12673 leader_election.cc:403] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:48.495581 17511 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:48.428260 17086 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:19:48.528522 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.494621 17844 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:48.433801 17849 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:19:48.539031 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 103212us
I20260709 13:19:48.540786 12632 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:48.546871 17128 consensus_queue.cc:1059] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:19:48.524259 17840 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:48.536877 17662 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:48.557886 17842 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:19:48.545701 17543 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:48.565829 17543 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:48.572429 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 71937.6us
I20260709 13:19:48.573248 12673 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:48.574002 12783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 675) took 1033 ms. Trace:
I20260709 13:19:48.574136 17861 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:19:48.574424 12783 rpcz_store.cc:276] 0709 13:19:47.541104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.547811 (+  6707us) service_pool.cc:224] Handling call
0709 13:19:47.548086 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:47.935455 (+387369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:47.935501 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:47.950413 (+ 14912us) log.cc:844] Serialized 26 byte log entry
0709 13:19:48.247024 (+296611us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:48.247066 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:48.247138 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:48.573762 (+326624us) raft_consensus.cc:1685] finished
0709 13:19:48.573785 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:48.573941 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":380389,"raft.queue_time_us":389118,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":137,"thread_start_us":280767,"threads_started":1,"wal-append.queue_time_us":460433}
I20260709 13:19:48.538277 17822 raft_consensus.cc:740] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:48.576962 17822 consensus_queue.cc:260] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.578397 17822 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:48.579026 12780 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:19:48.578723 17840 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:48.580279 17464 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:48.580827 17464 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:48.581194 17464 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:48.581547 17464 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:48.581789 12629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 3158) took 1050 ms. Trace:
I20260709 13:19:48.582012 17539 consensus_queue.cc:1059] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:48.582156 12629 rpcz_store.cc:276] 0709 13:19:47.531778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.531921 (+   143us) service_pool.cc:224] Handling call
0709 13:19:47.547666 (+ 15745us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:47.578596 (+ 30930us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:47.614961 (+ 36365us) log.cc:844] Serialized 14 byte log entry
0709 13:19:48.173151 (+558190us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:48.173227 (+    76us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:48.173293 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:48.581422 (+408129us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b54000dcd54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:48.581467 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:48.581600 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87798,"prepare.queue_time_us":41280,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":160182,"raft.run_cpu_time_us":52,"raft.run_wall_time_us":52,"spinlock_wait_cycles":46243712,"thread_start_us":503159,"threads_started":2,"wal-append.queue_time_us":379729}
I20260709 13:19:48.584097 17734 raft_consensus.cc:3055] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:48.584543 17734 raft_consensus.cc:740] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:48.578323 12747 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:48.578519 17839 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:48.587742 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:48.536226 17678 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.553804 17849 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:48.436442 17799 consensus_queue.cc:237] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:48.554610 17492 consensus_queue.cc:260] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.589130 12780 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:48.596135 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:48.596711 12783 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:19:48.594213 12627 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:19:48.599058 13349 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:48.604146 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:19:48.604525 13349 leader_election.cc:336] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.535s (SENT)
I20260709 13:19:48.605198 13349 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:48.578711 17797 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:48.607460 17678 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:48.607900 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:48.609056 17852 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:48.609716 17852 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:48.609905 17692 consensus_queue.cc:1059] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:48.611482 17844 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:48.611685 17086 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:48.611908 12672 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:48.612723 17731 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.613829 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:48.614756 17831 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:48.614925 17731 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:48.615283 17843 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:48.589895 17640 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:48.615908 17843 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.616285 17640 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.616429 17843 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:48.595516 17849 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:48.617863 17692 raft_consensus.cc:3055] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:48.618340 17692 raft_consensus.cc:740] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:48.618839 17640 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:48.588275 17321 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:48.619298 17692 consensus_queue.cc:260] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.619660 17321 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.543484 17859 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 13:19:48.596060 17839 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:19:48.624648 17772 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:48.625281 17719 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:48.625944 17719 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:48.627928 17824 raft_consensus.cc:3055] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:48.628296 17719 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:48.628619 17492 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:48.628993 17581 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:48.573318 17762 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.629639 17581 raft_consensus.cc:697] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 11 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:48.581784 17684 consensus_queue.cc:1059] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:19:48.630811 17684 raft_consensus.cc:3055] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:48.631173 17684 raft_consensus.cc:740] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:48.631050 17581 consensus_queue.cc:237] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:48.632207 17684 consensus_queue.cc:260] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.610206 17852 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:48.633741 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.589957 17853 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:48.636559 17321 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:48.585536 17734 consensus_queue.cc:260] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.638974 17734 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:48.640555 12703 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:48.641674 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:48.642310 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:48.642477 12782 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:48.643406 12597 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:48.644173 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:48.645004 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:48.645550 12700 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:48.645957 17537 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:48.611521 17848 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:19:48.645984 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:19:48.648169 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 43680.5us
W20260709 13:19:48.649667 13349 leader_election.cc:336] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.539s (SENT)
I20260709 13:19:48.650017 17862 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:48.650586 17862 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:48.651000 17862 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:48.651469 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:48.652125 12707 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:19:48.609153 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:48.629720 17692 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:48.662773 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:48.663115 17762 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:48.664681 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:48.664736 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:48.654507 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:48.666219 12706 raft_consensus.cc:2410] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260709 13:19:48.667459 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 58386us
I20260709 13:19:48.668260 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:48.657136 12780 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
W20260709 13:19:48.659319 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 257745us
I20260709 13:19:48.674381 12700 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 18.
I20260709 13:19:48.676350 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:48.677330 12707 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:19:48.677738 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:48.678417 12705 raft_consensus.cc:2376] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 18. Re-sending same reply.
I20260709 13:19:48.669795 17536 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:48.680068 17536 raft_consensus.cc:697] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:48.683022 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:48.683729 12627 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:19:48.687006 17772 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:48.689669 17853 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:48.628306 17824 raft_consensus.cc:740] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:48.698518 17824 consensus_queue.cc:260] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.699842 17824 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:48.701591 17843 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:48.633388 17684 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:48.703773 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:48.704572 12598 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:48.706514 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:48.706671 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 57031.7us
I20260709 13:19:48.707346 12597 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:48.701807 17543 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.708468 17861 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14.
I20260709 13:19:48.645313 17840 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.709908 17543 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:48.710242 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:48.710753 17840 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:48.710845 12628 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:48.711207 12747 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:48.712123 17848 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:48.712505 12747 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:48.681152 17536 consensus_queue.cc:237] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.714251 17086 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:48.719864 17848 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:48.720002 17852 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.722600 17852 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:48.725770 17692 consensus_queue.cc:1059] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:48.726672 17692 raft_consensus.cc:3055] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:48.726986 17692 raft_consensus.cc:740] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:48.727376 17862 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.727946 17692 consensus_queue.cc:260] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:48.729142 17692 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:48.734436 12703 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:19:48.720551 12628 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:19:48.714381 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:48.736960 17772 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:48.737372 17772 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:48.737763 17772 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:48.749104 17862 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:48.706012 17526 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:48.756517 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:48.757228 12700 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
W20260709 13:19:48.760957 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 184932us
W20260709 13:19:48.764674 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 57400.5us
W20260709 13:19:48.768989 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 64511.9us
I20260709 13:19:48.765895 12597 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:19:48.772111 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.785475 17623 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:48.786023 17623 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:48.786409 17623 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:48.787801 12598 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:48.792150 17128 consensus_queue.cc:1059] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:48.796938 17734 consensus_queue.cc:1059] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:48.798681 17128 consensus_queue.cc:1048] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:48.799492 17865 raft_consensus.cc:3055] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:48.799405 17830 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:48.799949 17865 raft_consensus.cc:740] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:48.800801 17128 raft_consensus.cc:3055] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:48.790516 17848 consensus_queue.cc:1059] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 13:19:48.805105 12780 raft_consensus.cc:1275] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:19:48.801265 17128 raft_consensus.cc:740] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:48.774922 17321 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:48.811735 17128 consensus_queue.cc:260] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:48.813099 12673 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:48.801079 17865 consensus_queue.cc:260] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.815647 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:48.816490 17865 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:48.817907 17820 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:48.818553 17820 raft_consensus.cc:697] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:48.819976 17820 consensus_queue.cc:237] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.821768 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:48.822958 12782 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:48.825284 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:48.826089 12627 raft_consensus.cc:2393] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 15: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:48.826654 12781 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:48.832520 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:48.833526 12703 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:48.837527 17128 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:48.844127 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.845743 12630 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:19:48.845635 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:48.848390 12627 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 13:19:48.848953 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:48.854493 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 66783.3us
I20260709 13:19:48.855396 12598 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:48.856554 17840 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:48.856673 12598 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:48.857892 17872 log.cc:927] Time spent T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:19:48.810846 17532 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:48.861131 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:48.872418 12629 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:48.861636 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:48.873327 12707 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:19:48.884311 17464 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.857591 17840 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:48.889008 17464 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:48.889834 17692 raft_consensus.cc:3055] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:48.890362 17692 raft_consensus.cc:740] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:48.891350 17692 consensus_queue.cc:260] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:48.893886 12706 raft_consensus.cc:1275] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:48.893747 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:48.894771 12707 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:48.834815 17086 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:48.860702 17799 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:48.898599 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:48.899051 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:48.914155 12780 raft_consensus.cc:1240] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:48.902406 12703 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:19:48.915712 12781 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:48.905305 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:48.905682 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:48.917989 12704 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:48.860284 17539 consensus_queue.cc:1059] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:19:48.921350 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:48.924378 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 94777.8us
I20260709 13:19:48.922116 12709 raft_consensus.cc:2376] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14. Re-sending same reply.
I20260709 13:19:48.845193 17511 consensus_queue.cc:1048] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:19:48.940191 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:19:48.903918 12598 connection.cc:256] RPC call timeout handler was delayed by -1.1696557189998202s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17830000000000001s for a response
W20260709 13:19:48.940984 12598 connection.cc:256] RPC call timeout handler was delayed by -1.0499012119998952s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1763s for a response
I20260709 13:19:48.858886 17797 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:19:48.885655 17532 raft_consensus.cc:697] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:19:48.945358 13349 leader_election.cc:336] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.716s (SENT)
I20260709 13:19:48.946743 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:48.964046 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:48.925624 12747 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:48.878873 17861 consensus_queue.cc:1059] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:19:48.952343 12776 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:48.907473 17853 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:48.968029 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:48.969731 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:48.970145 12703 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:48.968758 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:48.967931 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:48.975461 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:48.976486 12634 raft_consensus.cc:2410] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:19:48.977847 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:48.980602 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:48.899410 17086 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:48.933889 17539 consensus_queue.cc:1243] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:19:48.986893 17086 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:48.990125 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:48.991957 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:48.992161 17772 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:48.992852 12598 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:48.993897 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:48.995711 12775 raft_consensus.cc:1275] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:48.994907 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:48.995229 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:48.998411 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:48.993508 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:49.042847 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.044726 12778 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:49.045197 12705 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:49.046258 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 120765us
I20260709 13:19:49.046952 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:49.047669 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:49.057850 12707 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:19:49.056761 12629 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:49.058614 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:49.060454 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:49.063473 12701 raft_consensus.cc:2393] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 11: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:49.052024 12625 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:19:49.053292 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:49.063450 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:49.066269 12780 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:48.997206 12598 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:49.053831 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:49.054062 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 67913.8us
I20260709 13:19:48.997174 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:49.124593 12775 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 16. Current term is 17. Ops: [15.10-16.11]
I20260709 13:19:48.994660 17799 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:49.128383 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 131335us
W20260709 13:19:49.132082 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 186724us
I20260709 13:19:49.132406 12598 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:49.132822 13349 leader_election.cc:336] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.526s (SENT)
I20260709 13:19:49.088562 12778 raft_consensus.cc:1240] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:49.134186 12747 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:49.053158 17848 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:49.144883 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:49.046187 17861 consensus_queue.cc:1243] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:19:49.146714 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:49.162339 12780 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:49.162735 12776 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
W20260709 13:19:49.062450 17662 consensus_meta.cc:287] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.505s	user 0.010s	sys 0.000s
I20260709 13:19:49.168670 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:49.169874 12631 raft_consensus.cc:2393] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 15: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:49.060515 17731 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.152930 12708 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:19:49.155994 12747 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:49.176427 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:49.177459 12708 raft_consensus.cc:2393] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 15: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:49.176805 12747 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:49.122465 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:48.967332 17526 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:49.160187 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:49.161551 12633 raft_consensus.cc:1240] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:49.166600 12704 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:19:49.046379 17860 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:49.192703 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:19:49.146358 17849 consensus_meta.cc:287] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.528s	user 0.000s	sys 0.011s
I20260709 13:19:49.139453 17509 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:49.192979 17536 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:19:49.199458 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 66658.1us
I20260709 13:19:49.199548 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:49.209741 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:49.194195 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:49.200050 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:49.209668 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:49.203940 12747 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:49.210807 13383 raft_consensus.cc:2368] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14.
I20260709 13:19:49.212189 13377 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:49.162467 17853 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:49.217933 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:49.213907 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:49.180737 17840 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:19:49.216926 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.221040 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:49.200645 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:49.139204 17863 consensus_queue.cc:1059] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 13:19:49.220680 17086 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.236876 12705 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:19:49.233815 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:49.226169 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:49.245326 12700 raft_consensus.cc:1813] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Handling vote request from an unknown peer 19cd864c053744bebd96093a7982537e
I20260709 13:19:49.233896 12701 raft_consensus.cc:2368] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16.
I20260709 13:19:49.238368 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:49.238912 12709 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:19:49.239185 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:49.242558 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:49.232978 13349 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:49.247392 13377 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:19:49.248255 12709 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:19:49.248590 12707 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:19:49.243347 13379 raft_consensus.cc:1240] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:49.243491 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:49.246313 12700 raft_consensus.cc:2410] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:19:49.249835 12709 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:49.251016 12702 raft_consensus.cc:2393] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 15: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:49.251911 12705 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:19:49.253336 13349 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:49.231567 17830 raft_consensus.cc:3055] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:49.257784 12747 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:49.258638 12703 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:19:49.259065 17526 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:49.224237 17882 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:19:49.236378 17492 consensus_meta.cc:287] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.607s	user 0.011s	sys 0.000s
I20260709 13:19:49.180506 17662 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:49.236014 17848 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:49.264261 17526 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:49.219115 17860 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:49.208331 17849 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:49.284113 12782 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:49.261117 17830 raft_consensus.cc:740] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:19:49.288334 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 194615us
I20260709 13:19:49.237152 17840 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:49.182272 17731 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:49.286270 12774 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:49.287364 12597 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:49.270107 12747 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:49.292375 12630 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:49.302618 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:49.302886 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:49.305297 17623 consensus_meta.cc:287] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.518s	user 0.007s	sys 0.000s
I20260709 13:19:49.341243 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:49.290603 12673 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:49.268970 17772 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:49.276189 17526 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:49.325258 12701 raft_consensus.cc:1240] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:49.325472 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:49.257068 17086 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:49.260015 17842 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:19:49.327191 12702 raft_consensus.cc:1240] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:19:49.327986 17860 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:49.329041 12747 connection.cc:256] RPC call timeout handler was delayed by -1.0533449179999934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1544s for a response
W20260709 13:19:49.368180 12747 connection.cc:256] RPC call timeout handler was delayed by -1.0380449179999687s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15610000000000002s for a response
W20260709 13:19:49.368652 12747 connection.cc:256] RPC call timeout handler was delayed by -1.0317449179999585s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15680000000000002s for a response
I20260709 13:19:49.329275 17853 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:49.330866 12709 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:49.331106 17662 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:49.333091 17849 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:49.215615 17532 consensus_queue.cc:237] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:49.286367 17823 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:19:49.336345 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.342727 12704 raft_consensus.cc:2376] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14. Re-sending same reply.
I20260709 13:19:49.295723 17840 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:49.349021 17492 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:19:49.345626 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 75742.2us
I20260709 13:19:49.420351 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:49.308543 17882 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:49.349534 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:49.382843 17874 log.cc:927] Time spent T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.242s	user 0.001s	sys 0.000s
I20260709 13:19:49.383242 17830 consensus_queue.cc:260] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:49.381296 17734 consensus_queue.cc:1048] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:19:49.388805 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:49.343061 17797 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:49.403360 17853 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:49.443447 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:49.324566 17848 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:19:49.422482 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:19:49.436090 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 86632.2us
W20260709 13:19:49.343724 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 53204.9us
I20260709 13:19:49.432344 17882 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:49.453976 17882 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:49.456370 17536 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:49.451164 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:49.461189 17853 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:49.466114 17128 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:49.466609 17128 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:49.466955 17772 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:49.467209 17843 consensus_queue.cc:1059] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:49.439143 17537 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:49.350260 17865 consensus_queue.cc:1059] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.085s
W20260709 13:19:49.469658 17885 log.cc:927] Time spent T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 13:19:49.466976 17128 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:49.349679 17623 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:49.472448 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:49.367843 17820 consensus_queue.cc:1059] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.087s
I20260709 13:19:49.464983 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:49.479966 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 347674us
I20260709 13:19:49.456990 17536 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:49.516573 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 63340.2us
W20260709 13:19:49.470963 17684 consensus_meta.cc:287] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.713s	user 0.011s	sys 0.000s
I20260709 13:19:49.448235 17799 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:19:49.584046 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.587841 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:49.588729 17128 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:49.497339 12672 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:49.449971 17492 raft_consensus.cc:697] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:49.444777 17086 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:49.595860 12783 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:19:49.607430 12632 raft_consensus.cc:1240] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 13:19:49.608165 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 110917us
I20260709 13:19:49.445775 17731 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:49.467581 17772 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:49.470729 17797 raft_consensus.cc:697] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:49.604323 17128 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:49.471870 17537 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:49.626539 17537 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:49.450759 17890 raft_consensus.cc:2764] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
W20260709 13:19:49.615784 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 177694us
I20260709 13:19:49.623715 17492 consensus_queue.cc:237] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:49.625659 17865 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:19:49.626184 17900 log.cc:927] Time spent T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.148s	user 0.001s	sys 0.001s
I20260709 13:19:49.642673 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:49.644459 12708 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:49.601205 17849 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:19:49.639230 17892 log.cc:927] Time spent T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 13:19:49.722046 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:49.723054 13383 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
W20260709 13:19:49.740483 17883 log.cc:927] Time spent T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.359s	user 0.001s	sys 0.000s
I20260709 13:19:49.746971 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:49.747135 12746 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:49.769817 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:49.673288 17849 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:49.776139 17849 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:49.779771 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:49.780314 12778 raft_consensus.cc:2368] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 13. Current term is 14.
I20260709 13:19:49.780690 12781 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:49.790359 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:49.793670 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 192637us
I20260709 13:19:49.794138 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:49.794497 12598 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:49.692636 17509 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:19:49.813380 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.670114 17842 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.127s
I20260709 13:19:49.816604 12778 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
W20260709 13:19:49.822696 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 78585.5us
I20260709 13:19:49.823840 12747 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:49.825651 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:49.842386 12781 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:19:49.856741 12632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 3169) took 1035 ms. Trace:
W20260709 13:19:49.859362 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 64934.6us
I20260709 13:19:49.859436 12783 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:19:49.860363 12598 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:49.857211 12632 rpcz_store.cc:276] 0709 13:19:48.821989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.823954 (+  1965us) service_pool.cc:224] Handling call
0709 13:19:48.824364 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:49.607888 (+783524us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:49.856433 (+248545us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b54001590d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:49.856670 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1800311168}
I20260709 13:19:49.865438 12598 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:49.865461 12747 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:49.873035 12598 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:49.779145 17797 consensus_queue.cc:237] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:49.844345 17799 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:49.856544 17907 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:49.868544 17848 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:19:49.895874 17904 log.cc:927] Time spent T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
W20260709 13:19:49.896894 17897 log.cc:927] Time spent T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:19:49.893116 12747 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:49.875420 17509 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:49.912354 17916 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.875305 17532 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:49.914176 12626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 3173) took 1090 ms. Trace:
I20260709 13:19:49.915653 17911 consensus_queue.cc:1048] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:19:49.900089 17880 log.cc:927] Time spent T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.773s	user 0.002s	sys 0.002s
I20260709 13:19:49.902019 17086 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:49.929702 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:49.930610 17907 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:49.930994 17907 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:49.931375 17907 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:49.937837 17905 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:49.939076 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:49.939644 17843 raft_consensus.cc:3055] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:49.940116 17843 raft_consensus.cc:740] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:49.940204 17879 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:49.943521 17623 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:49.945462 12779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 2510) took 1034 ms. Trace:
I20260709 13:19:49.945891 12779 rpcz_store.cc:276] 0709 13:19:48.911030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.911178 (+   148us) service_pool.cc:224] Handling call
0709 13:19:48.940110 (+ 28932us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:49.272797 (+332687us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:49.272844 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:49.292317 (+ 19473us) log.cc:844] Serialized 26 byte log entry
0709 13:19:49.525815 (+233498us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:49.525878 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:49.525980 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:49.945257 (+419277us) raft_consensus.cc:1685] finished
0709 13:19:49.945279 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:49.945399 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":28721,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"thread_start_us":450603,"threads_started":2,"wal-append.queue_time_us":270985}
I20260709 13:19:49.948909 17879 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:49.950519 17899 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:49.952395 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:49.956673 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:49.925432 12626 rpcz_store.cc:276] 0709 13:19:48.823616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.844892 (+ 21276us) service_pool.cc:224] Handling call
0709 13:19:48.845231 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:48.854028 (+  8797us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:49.146071 (+292043us) log.cc:844] Serialized 14 byte log entry
0709 13:19:49.593666 (+447595us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:49.618712 (+ 25046us) log.cc:844] Serialized 26 byte log entry
0709 13:19:49.618859 (+   147us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:49.618894 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:49.618961 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:49.913938 (+294977us) raft_consensus.cc:1685] finished
0709 13:19:49.913980 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:49.914110 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":140050,"prepare.queue_time_us":405653,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":97333,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":15067,"thread_start_us":523570,"threads_started":2,"wal-append.queue_time_us":295221}
W20260709 13:19:49.957924 17824 consensus_meta.cc:287] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 1.258s	user 0.013s	sys 0.000s
I20260709 13:19:49.927420 17536 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:49.961205 17536 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:49.961555 12707 raft_consensus.cc:1275] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:19:49.963174 17526 consensus_meta.cc:287] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.595s	user 0.008s	sys 0.000s
I20260709 13:19:49.918947 17840 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:49.963575 17526 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:49.964749 17537 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:49.965500 17840 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:49.925145 17916 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:49.967344 17537 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:49.897503 17882 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:49.969581 12777 raft_consensus.cc:2393] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 14: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:49.971517 17882 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:49.968072 17916 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:49.967362 17526 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:49.973196 12631 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:49.976137 12703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 2303) took 1005 ms. Trace:
I20260709 13:19:49.976523 12703 rpcz_store.cc:276] 0709 13:19:48.970996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.124106 (+153110us) service_pool.cc:224] Handling call
0709 13:19:49.975969 (+851863us) spinlock_profiling.cc:243] Waited 75 ms on lock 0x7b54001cd4d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:49.976079 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172610816}
I20260709 13:19:49.978051 17905 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.978590 17905 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:49.980845 17905 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:49.982204 12774 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2856) took 1071 ms. Trace:
I20260709 13:19:49.982609 12774 rpcz_store.cc:276] 0709 13:19:48.911075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:48.915109 (+  4034us) service_pool.cc:224] Handling call
0709 13:19:49.286080 (+370971us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:49.296553 (+ 10473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:49.981985 (+685432us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b540044bbd0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:49.982130 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":792756992}
W20260709 13:19:49.986629 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 113712us
I20260709 13:19:49.987432 12598 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:49.988230 17801 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:49.989846 12598 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:49.990525 17801 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:49.995584 17907 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:50.002974 17903 log.cc:927] Time spent T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.170s	user 0.002s	sys 0.000s
I20260709 13:19:49.941342 17843 consensus_queue.cc:260] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:50.015091 17848 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:50.015659 17848 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:49.958061 17899 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:50.019651 12598 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:50.023109 17907 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:50.023690 17640 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:50.024055 12673 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:50.024683 17909 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:50.025424 17831 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.030609 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:50.031510 12628 raft_consensus.cc:2393] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 14: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:50.039110 12776 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 16. Current term is 17. Ops: [15.10-16.11]
I20260709 13:19:50.042431 12779 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:50.044148 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:50.044771 12775 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:19:50.044642 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:50.045434 17921 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:50.046342 17859 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.046964 17859 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:50.049803 17916 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.050760 17824 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:50.022943 12598 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:50.051358 17824 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:50.053737 17865 consensus_queue.cc:1059] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:50.055233 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:50.055586 17865 raft_consensus.cc:3055] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:50.055994 17865 raft_consensus.cc:740] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:50.056041 12780 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:19:50.057349 17865 consensus_queue.cc:260] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:50.059444 17865 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:50.050413 17916 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:50.061425 17916 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:50.069456 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:50.070403 12774 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:50.073366 12781 raft_consensus.cc:3055] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:50.073580 12775 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:50.074043 12781 raft_consensus.cc:740] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:19:50.074494 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 51638.3us
W20260709 13:19:50.079885 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.025969 17831 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:50.084945 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:50.112058 12631 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:50.089669 12672 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:50.113392 17863 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:50.113951 17863 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:50.114367 17863 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:50.115599 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:50.115814 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:50.116266 12779 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:50.102602 12632 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13. Ops: []
W20260709 13:19:50.102938 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.072494 17772 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:50.086869 12781 consensus_queue.cc:260] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:50.118350 17917 consensus_queue.cc:1059] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:50.118877 12781 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:50.119413 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.119630 17891 consensus_queue.cc:1059] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:50.107766 12633 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:19:50.071681 17859 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:50.109598 12626 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:19:50.082314 17847 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:50.116314 12775 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:50.127496 17464 raft_consensus.cc:3055] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:50.127936 17464 raft_consensus.cc:740] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:50.129177 12781 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:19:50.128826 17464 consensus_queue.cc:260] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:50.133383 12781 pending_rounds.cc:85] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 6
I20260709 13:19:50.134440 12781 pending_rounds.cc:107] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:19:50.134691 12781 raft_consensus.cc:2889] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:50.091502 17824 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:50.113410 12598 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:50.151728 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:50.166296 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 52960.5us
I20260709 13:19:50.168130 12598 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:50.169102 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:50.173348 12781 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
W20260709 13:19:50.173892 13349 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.007s	sys 0.019s
W20260709 13:19:50.178731 13349 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.008s	sys 0.020s
W20260709 13:19:50.179136 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 539757us
W20260709 13:19:50.182348 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 430776us
W20260709 13:19:50.184260 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 227652us
I20260709 13:19:50.187315 12747 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:50.187942 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:50.188629 12747 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:50.191079 17909 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:50.189538 17934 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:50.196216 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:50.197253 13382 raft_consensus.cc:2393] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 14: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:50.204633 13348 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:50.215904 12598 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:50.216048 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:50.216940 12706 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:19:50.212586 13349 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:50.219306 12703 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:50.222990 12598 leader_election.cc:400] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16.
I20260709 13:19:50.223353 12598 leader_election.cc:403] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:19:50.223567 12704 raft_consensus.cc:1240] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:50.226291 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:50.228870 13348 leader_election.cc:400] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 13. Current term is 14.
I20260709 13:19:50.212138 17801 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:50.231885 13349 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:50.234969 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:50.215294 17772 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:50.242973 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:50.235638 12634 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:50.237795 13349 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:50.227346 17847 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:19:50.248368 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 129742us
I20260709 13:19:50.252338 12672 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:50.253697 12598 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:50.255007 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:50.256003 12702 raft_consensus.cc:2393] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 14: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
W20260709 13:19:50.256870 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.230414 17321 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16.
I20260709 13:19:50.220983 17640 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:19:50.259107 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:50.227638 17840 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:19:50.241967 17460 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:50.242234 17719 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:50.227136 17799 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:19:50.251451 17526 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:50.279721 12709 raft_consensus.cc:1240] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:50.282092 17492 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:50.287811 12672 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:50.304452 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:50.315127 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 108629us
W20260709 13:19:50.316972 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.273595 17917 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:19:50.300519 17853 consensus_meta.cc:287] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.837s	user 0.010s	sys 0.003s
W20260709 13:19:50.334084 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 80514.8us
I20260709 13:19:50.303299 17801 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:50.264767 17849 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:50.348659 12747 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:50.321959 17917 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:50.324591 17847 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:50.356038 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:50.357774 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.358130 17801 raft_consensus.cc:697] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:50.264530 17863 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:50.357213 12781 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:19:50.319368 17492 raft_consensus.cc:697] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:50.372229 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:50.344174 17460 raft_consensus.cc:697] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 13 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:19:50.372102 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 84416.7us
W20260709 13:19:50.376349 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.349515 17940 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:50.378881 12672 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:50.358923 17086 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:50.363252 17901 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:50.353708 17749 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:19:50.387187 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 74616.6us
I20260709 13:19:50.363917 17917 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:50.385473 12672 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:50.338135 17853 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:50.358343 17909 consensus_queue.cc:1059] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:19:50.354096 17907 consensus_queue.cc:1059] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:19:50.391131 12673 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:50.395066 12782 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:50.376273 12778 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 10: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:50.368428 17920 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:50.371398 17938 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:50.358203 17879 consensus_queue.cc:1059] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:19:50.371291 17536 consensus_queue.cc:1059] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:19:50.390630 17824 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:50.397254 17797 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.390547 17749 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:50.389362 17719 raft_consensus.cc:697] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:50.409299 12781 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:50.428984 12701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 1987) took 1127 ms. Trace:
I20260709 13:19:50.386963 17922 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:50.393369 17526 raft_consensus.cc:697] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:50.383909 17801 consensus_queue.cc:237] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:50.426976 17492 consensus_queue.cc:237] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:50.441767 17852 raft_consensus.cc:3055] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:50.442186 17852 raft_consensus.cc:740] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:50.443156 17852 consensus_queue.cc:260] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:50.429497 12701 rpcz_store.cc:276] 0709 13:19:49.301752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.301909 (+   157us) service_pool.cc:224] Handling call
0709 13:19:49.302221 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:49.348578 (+ 46357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:50.428692 (+1080114us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b54003ec1d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:50.428911 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52508928}
I20260709 13:19:50.403195 17920 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:50.413163 17871 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:50.391232 17940 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:50.403337 17938 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:50.416440 17943 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:50.417531 17930 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
W20260709 13:19:50.418891 17929 log.cc:927] Time spent T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:19:50.420866 17927 log.cc:927] Time spent T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 13:19:50.423081 17932 log.cc:927] Time spent T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:19:50.373685 17847 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:50.398623 17901 raft_consensus.cc:697] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:50.433179 17863 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:50.426916 17086 raft_consensus.cc:3055] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:50.438040 17830 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:50.425186 17749 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:50.446039 17871 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:50.439986 17719 consensus_queue.cc:237] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:50.411665 17460 consensus_queue.cc:237] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:50.362386 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 114273us
I20260709 13:19:50.406672 17824 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:50.447798 17909 raft_consensus.cc:3055] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:19:50.450187 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:50.453595 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 90429.7us
I20260709 13:19:50.439651 17853 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:50.455288 17852 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:50.424314 17797 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:50.491241 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:50.480360 17871 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:50.473799 17086 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:50.495673 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:50.497689 13377 raft_consensus.cc:1240] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 13. Current term is 14. Ops: []
W20260709 13:19:50.492722 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 134357us
I20260709 13:19:50.484275 17824 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:50.506932 12626 raft_consensus.cc:1240] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:50.489743 17909 raft_consensus.cc:740] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:50.512615 12626 raft_consensus.cc:1240] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:50.513711 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:50.496047 17849 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:50.518208 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:50.513967 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:50.494961 17941 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.517067 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:50.519150 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:50.516856 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:50.520947 12629 raft_consensus.cc:1240] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:19:50.521055 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:50.522781 12631 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:50.548506 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:50.495088 17536 consensus_queue.cc:1059] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:19:50.506659 17853 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.521718 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:50.523340 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:50.557185 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:50.560114 12776 raft_consensus.cc:1275] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:19:50.560588 17913 consensus_queue.cc:1059] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:19:50.561596 17947 raft_consensus.cc:3055] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:50.561970 17947 raft_consensus.cc:740] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:50.563148 17831 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:50.562844 17947 consensus_queue.cc:260] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:50.564235 17947 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:50.566788 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 284820us
I20260709 13:19:50.515357 17942 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.587083 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:50.514742 17772 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 13:19:50.601169 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:50.602694 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:50.613487 12633 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:19:50.616556 12626 raft_consensus.cc:2393] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 11: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:19:50.623385 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 110504us
I20260709 13:19:50.624436 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:50.496172 17526 consensus_queue.cc:237] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:50.515358 17901 consensus_queue.cc:237] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:50.627696 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 67977.9us
I20260709 13:19:50.604879 17934 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:50.634049 17934 consensus_queue.cc:1059] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:50.573271 17947 consensus_queue.cc:1059] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 13:19:50.636134 12775 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:19:50.636436 17913 raft_consensus.cc:3055] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:19:50.636792 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.637214 17913 raft_consensus.cc:740] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:50.638779 17879 raft_consensus.cc:3055] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:50.639199 17879 raft_consensus.cc:740] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:50.640048 12747 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:50.640527 17879 consensus_queue.cc:260] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:50.641158 17662 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.642063 17879 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:50.641798 17662 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:50.644799 17662 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:50.573983 17853 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:50.638223 17913 consensus_queue.cc:260] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:50.648758 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 244875us
I20260709 13:19:50.650998 12673 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:50.556383 17086 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:50.616794 17917 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:50.553668 17941 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:50.683534 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.685082 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:50.686193 13383 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:50.601063 17942 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:50.713425 12632 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:19:50.649531 17662 consensus_queue.cc:1059] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:19:50.717620 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:50.686777 17086 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:50.605803 17909 consensus_queue.cc:260] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:50.752852 17909 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:50.738003 17937 log.cc:927] Time spent T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
W20260709 13:19:50.739182 17949 log.cc:927] Time spent T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:19:50.740080 17938 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:19:50.687592 17952 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:19:50.740515 12708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 2305) took 1763 ms (client timeout 1899 ms). Trace:
I20260709 13:19:50.693315 17913 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:50.785151 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:50.790664 12708 rpcz_store.cc:269] 0709 13:19:48.971688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.192046 (+220358us) service_pool.cc:224] Handling call
0709 13:19:50.733726 (+1541680us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b540006d454. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:50.734541 (+   815us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":901121920}
I20260709 13:19:50.747581 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:50.778698 12707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 2321) took 1794 ms. Trace:
W20260709 13:19:50.771205 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 120333us
I20260709 13:19:50.789975 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:50.806123 13378 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
W20260709 13:19:50.734669 17904 log.cc:927] Time spent T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.052s	user 0.000s	sys 0.004s
I20260709 13:19:50.803565 12707 rpcz_store.cc:276] 0709 13:19:48.984322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.249531 (+265209us) service_pool.cc:224] Handling call
0709 13:19:49.249871 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:49.961970 (+712099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:50.778417 (+816447us) spinlock_profiling.cc:243] Waited 711 ms on lock 0x7b5400359154. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:50.778624 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1635943168}
I20260709 13:19:50.812152 13383 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:50.781500 17762 raft_consensus.cc:3055] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:50.741705 17950 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:50.787415 17859 raft_consensus.cc:3055] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:50.849949 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:50.742291 17797 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:50.771396 17861 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:19:50.848145 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 223150us
I20260709 13:19:50.801267 17940 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:50.866151 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:50.866494 17128 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:50.867733 12776 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:50.800314 17943 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:50.759414 17843 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:50.760828 17920 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:50.759774 17941 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:50.754127 17922 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:50.859710 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:50.860172 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:50.861058 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:50.861862 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:50.822639 17731 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:19:50.870719 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.844799 17930 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.866847 13376 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:50.847925 17762 raft_consensus.cc:740] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:50.846680 17866 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:50.881749 12706 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:50.888393 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:50.882552 12626 raft_consensus.cc:1240] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:50.850323 17859 raft_consensus.cc:740] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:50.892556 12774 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:19:50.884729 12705 raft_consensus.cc:2410] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:19:50.894767 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:50.883340 12701 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:19:50.881829 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:50.897853 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:50.905468 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.874043 17797 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:19:50.892352 17848 consensus_meta.cc:287] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.871s	user 0.014s	sys 0.000s
I20260709 13:19:50.903414 12780 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
W20260709 13:19:50.898522 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 48660.9us
I20260709 13:19:50.915225 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:50.915942 12706 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:50.916061 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:50.820648 17952 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:50.923475 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:50.926110 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.874332 17931 consensus_queue.cc:1059] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:19:50.927464 17931 consensus_queue.cc:1243] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:19:50.931843 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.937908 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:50.901342 17934 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.940119 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:50.938740 12778 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:50.941510 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:50.945550 12776 raft_consensus.cc:2393] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 13: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:50.950153 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:50.952088 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:50.954427 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:50.953218 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:50.956946 12779 raft_consensus.cc:1240] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:50.958140 12779 raft_consensus.cc:1240] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:50.959333 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:50.907135 17839 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:50.966266 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:19:50.950879 12672 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.68:36021 timed out after 1.529s (SENT) [suppressed 26 similar messages]
I20260709 13:19:50.930279 17662 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:50.903040 17930 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:50.881044 17955 consensus_queue.cc:1059] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.089s
W20260709 13:19:50.993422 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.907441 17731 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:50.917291 17797 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:50.923231 17848 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:50.999691 17948 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:50.987964 17917 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:50.995951 17662 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.001518 17952 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:50.961871 17934 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:50.979579 17839 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:50.995640 17950 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:50.899504 17922 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.003507 17853 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:51.003973 17961 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:19:51.004833 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:50.350404 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:50.910646 17866 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.018155 17536 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:51.048924 12774 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13. Ops: []
W20260709 13:19:51.014281 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 86013.4us
I20260709 13:19:51.024057 12704 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:51.030170 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.038177 12777 raft_consensus.cc:2368] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 13. Current term is 14.
I20260709 13:19:51.003167 17973 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:50.989194 17965 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:50.764186 17882 consensus_queue.cc:1175] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "639ab2db265f4747bb7afb93e833f530" responder_term: 11 status { last_received { term: 11 index: 6 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:19:51.043954 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 93094.4us
I20260709 13:19:51.018487 17879 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:51.016198 17543 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
W20260709 13:19:51.069824 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.015738 17862 consensus_queue.cc:1048] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
W20260709 13:19:51.095482 17830 consensus_meta.cc:287] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.619s	user 0.010s	sys 0.003s
I20260709 13:19:51.119035 17953 consensus_queue.cc:1048] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.149s
I20260709 13:19:51.068866 17911 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:51.092403 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 142746us
I20260709 13:19:51.120764 13378 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
W20260709 13:19:51.067869 17852 consensus_meta.cc:287] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.574s	user 0.008s	sys 0.004s
I20260709 13:19:51.024439 17948 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.126065 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:51.130139 12704 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:19:51.021245 12778 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:19:51.133510 17975 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:51.137674 17911 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.145661 12672 leader_election.cc:400] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14.
I20260709 13:19:51.147385 12706 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:19:51.148904 12706 pending_rounds.cc:85] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 8
I20260709 13:19:51.149147 12706 pending_rounds.cc:107] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:19:51.149386 12706 raft_consensus.cc:2889] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:51.152828 12597 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:51.129629 17511 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:19:51.159830 17911 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:51.156607 12598 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:51.160964 17848 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:51.164289 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:51.130180 17921 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.171629 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:51.181216 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:51.182739 17917 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:51.141789 17866 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:51.152341 17952 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.152817 17830 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.082619 17965 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:51.193039 17976 log.cc:927] Time spent T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:19:51.194518 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 235240us
I20260709 13:19:51.191366 12598 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:51.198101 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:51.199469 12598 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:51.183260 17860 consensus_queue.cc:1059] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:19:51.203325 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:51.184631 17799 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:19:51.205212 12782 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.020s	sys 0.012s
W20260709 13:19:51.205668 12782 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.020s	sys 0.012s
I20260709 13:19:51.155073 17966 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:51.208796 17860 raft_consensus.cc:3055] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:51.209205 17860 raft_consensus.cc:740] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:51.209640 17975 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:51.210145 17975 raft_consensus.cc:697] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:19:51.211447 17824 consensus_meta.cc:287] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.697s	user 0.010s	sys 0.000s
I20260709 13:19:51.211895 17967 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.212435 17967 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.214301 12672 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:51.215013 17966 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.215068 17965 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:51.215478 17966 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:51.215634 17866 consensus_queue.cc:1059] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:51.216166 17662 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:51.190623 17801 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:51.210984 17975 consensus_queue.cc:237] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.214653 17967 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:51.237884 17865 raft_consensus.cc:3055] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:51.186863 17921 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.238353 17865 raft_consensus.cc:740] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:51.239401 17865 consensus_queue.cc:260] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.240284 17934 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:51.240626 17865 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:51.238306 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:51.241407 12631 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:19:51.242689 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:51.254261 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:51.207376 17543 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:51.254952 13378 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
W20260709 13:19:51.259395 17749 consensus_meta.cc:287] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.779s	user 0.011s	sys 0.001s
I20260709 13:19:51.259958 17749 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.261925 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:51.195657 17830 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.207566 17799 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.264294 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:51.222087 17849 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:51.203267 17952 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.267948 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:51.269373 13378 raft_consensus.cc:2376] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13. Re-sending same reply.
I20260709 13:19:51.215495 17931 consensus_queue.cc:1059] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:19:51.274294 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:51.256036 17831 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.255682 17959 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.211925 17824 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.274399 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:51.283542 12782 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:51.265192 17930 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:51.282054 12632 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:19:51.269183 17799 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:51.288865 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:51.292603 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:51.293056 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:51.298775 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:51.210153 17860 consensus_queue.cc:260] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.306382 17128 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.307324 17128 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.277420 17849 raft_consensus.cc:697] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:51.310562 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:51.281292 17797 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:51.295559 17839 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:51.271246 17894 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.111s
W20260709 13:19:51.326745 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.336109 17985 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:51.336694 17985 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.337258 12782 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:19:51.308831 17731 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:51.285354 17831 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.313575 17822 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:51.306202 17922 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:51.355796 17985 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:51.338253 17977 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:51.356748 17977 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.357092 17977 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:51.357170 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:51.302081 17959 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.359926 17959 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:51.360579 17913 consensus_meta.cc:287] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.540s	user 0.008s	sys 0.000s
I20260709 13:19:51.361152 13378 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:19:51.361043 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:51.362623 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:51.366791 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.341938 17749 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:51.340027 17940 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.370432 12672 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:51.339888 17963 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:51.370687 12705 raft_consensus.cc:2376] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14. Re-sending same reply.
I20260709 13:19:51.371332 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:51.376950 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:51.377990 12703 raft_consensus.cc:2368] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 10. Current term is 11.
I20260709 13:19:51.387693 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:51.387961 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:51.388375 17831 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:51.388494 17464 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:51.389144 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:51.389833 12776 raft_consensus.cc:1217] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Deduplicated request from leader. Original: 14.9->[15.10-16.11]   Dedup: 15.10->[16.11-16.11]
I20260709 13:19:51.390343 12776 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 16. Current term is 17. Ops: [15.10-16.11]
I20260709 13:19:51.393149 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:51.393662 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:51.363950 17948 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:51.366255 17822 raft_consensus.cc:697] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:51.388507 12703 raft_consensus.cc:2393] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 11: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:51.368800 17848 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:51.394399 12781 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
W20260709 13:19:51.384316 17871 consensus_meta.cc:287] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.880s	user 0.012s	sys 0.001s
I20260709 13:19:51.399972 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:51.400584 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:51.400745 12781 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:19:51.402498 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:51.402776 13348 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:51.380285 17797 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:51.403894 17930 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:51.404450 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:51.404254 17797 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.405860 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:51.406502 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:51.407338 12782 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:51.407347 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:51.408697 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:51.409329 12701 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:51.375860 17940 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.409309 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:51.415959 12704 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:19:51.404354 17930 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:51.417466 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:51.417948 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.419168 17963 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.419632 17963 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.421655 17543 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:51.403903 17822 consensus_queue.cc:237] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.421838 17963 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:51.425724 17965 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:51.426251 17965 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:51.426663 17965 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:51.429657 17983 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:51.430330 17983 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.432781 17983 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:51.434083 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:51.439934 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:51.441706 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:51.442467 12703 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:19:51.409164 17913 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:51.445897 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:51.446522 12782 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:19:51.447368 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:51.400434 17848 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.449625 12597 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:51.452186 12747 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:51.394042 17849 consensus_queue.cc:237] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.463083 17799 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.444240 17824 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:51.465231 17799 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:51.418809 17952 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:51.439368 17930 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:51.406280 17871 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:51.459122 12808 heartbeater.cc:646] Failed to heartbeat to 127.11.63.126:38321 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.11.63.126:38321 timed out after 15.000s (SENT)
I20260709 13:19:51.449115 17913 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:51.462117 17909 consensus_meta.cc:287] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.709s	user 0.013s	sys 0.000s
I20260709 13:19:51.475394 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:51.486140 12779 raft_consensus.cc:1275] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:19:51.486588 17797 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:51.486999 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:51.487636 12704 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:19:51.488490 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:51.490515 17952 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.491346 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:51.491179 17952 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.494529 17952 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:51.494638 17762 consensus_queue.cc:260] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 9, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.495728 17762 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:51.498705 17539 raft_consensus.cc:3055] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:51.499135 17539 raft_consensus.cc:740] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:51.499432 17848 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:51.470608 17543 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:51.500018 17539 consensus_queue.cc:260] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.500563 17543 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.501148 17543 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:51.502939 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:51.505651 17847 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:51.507094 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:51.507658 12776 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:51.471345 17983 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:51.517970 12707 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:51.475015 12672 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:51.448599 17731 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:51.480877 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:51.519917 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 64693.5us
I20260709 13:19:51.520699 12782 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:19:51.522140 17917 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.522718 17917 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.523278 17539 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:51.524930 17917 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:51.484447 17913 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:51.527299 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:51.533221 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:51.537508 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:51.538726 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:51.539552 17983 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:51.540136 17983 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.522411 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:51.530421 12701 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:51.545248 12709 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 13:19:51.545666 12709 pending_rounds.cc:85] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 10
I20260709 13:19:51.545878 12709 pending_rounds.cc:107] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:19:51.525909 17948 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:51.546154 12709 raft_consensus.cc:2889] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:51.548512 12672 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:51.552848 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.547195 17822 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:51.568006 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:51.582106 12701 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:19:51.584468 17871 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:51.558696 17891 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:51.589480 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:51.579888 17822 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.559935 17948 raft_consensus.cc:697] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:51.605948 12775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36488 (request call id 2523) took 1085 ms. Trace:
I20260709 13:19:51.606549 12775 rpcz_store.cc:276] 0709 13:19:50.520209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.522930 (+  2721us) service_pool.cc:224] Handling call
0709 13:19:51.605653 (+1082723us) spinlock_profiling.cc:243] Waited 78.3 ms on lock 0x7b5400128754. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:51.605810 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":180123008}
I20260709 13:19:51.608140 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:51.608907 12775 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:19:51.618175 17921 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:51.621670 17822 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:51.624266 17830 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:51.624437 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:51.625394 12777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54346 (request call id 689) took 1269 ms. Trace:
I20260709 13:19:51.580046 17952 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:51.625783 12777 rpcz_store.cc:276] 0709 13:19:50.355845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.357291 (+  1446us) service_pool.cc:224] Handling call
0709 13:19:51.625197 (+1267906us) spinlock_profiling.cc:243] Waited 664 ms on lock 0x7b540008a554. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:51.625334 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1528289664}
I20260709 13:19:51.626294 17952 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.628489 17952 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:51.629397 13348 leader_election.cc:400] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 13. Current term is 14.
I20260709 13:19:51.550694 17966 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.633409 12672 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:51.590261 17086 consensus_meta.cc:287] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.838s	user 0.011s	sys 0.002s
I20260709 13:19:51.636765 17961 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.639175 17961 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.591152 17891 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.650935 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:51.651542 13383 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:51.653601 12776 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:51.656596 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:51.656777 12672 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:51.657253 12776 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:51.652992 12746 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:51.661669 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:51.662103 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:51.662477 12634 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:51.664193 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:51.673182 12625 raft_consensus.cc:1275] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:19:51.675128 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:51.676939 12632 raft_consensus.cc:1217] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Deduplicated request from leader. Original: 8.6->[10.7-13.8]   Dedup: 10.7->[13.8-13.8]
I20260709 13:19:51.676924 12672 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:51.712436 17864 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:51.713871 17871 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.714339 17871 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.714872 17965 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.716027 17946 raft_consensus.cc:3055] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:51.716455 17946 raft_consensus.cc:740] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:51.635178 17840 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:51.717022 12700 raft_consensus.cc:1275] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:19:51.717648 12633 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:51.718468 12632 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: [10.7-13.8]
I20260709 13:19:51.637744 17943 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:51.719935 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:51.720813 12708 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:19:51.716794 17965 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:51.652063 17961 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:51.656109 17891 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:19:51.726945 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 50124.8us
W20260709 13:19:51.737709 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.725443 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:51.761092 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:51.727246 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:51.655949 17958 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:51.725044 17799 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:51.762674 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:51.711278 12776 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:19:51.676403 17921 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.767892 12634 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:19:51.768388 12632 raft_consensus.cc:2393] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 15: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:51.769742 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:51.717620 17975 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:51.724867 17864 raft_consensus.cc:697] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:51.778156 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:51.723909 17943 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.779285 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:51.716476 17871 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:51.773483 12672 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:51.757313 17511 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:51.786134 12627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 941) took 1103 ms. Trace:
I20260709 13:19:51.760310 17464 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:51.667618 17983 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:51.795271 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:51.769495 17731 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.805298 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:51.791504 12776 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:51.759891 17964 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:51.725311 17930 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.793299 12627 rpcz_store.cc:276] 0709 13:19:50.682382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.682525 (+   143us) service_pool.cc:224] Handling call
0709 13:19:50.682942 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:50.683201 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:19:50.693204 (+ 10003us) log.cc:844] Serialized 14 byte log entry
0709 13:19:51.603536 (+910332us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b1800352ea0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf5223a 00007f4c6d3bca2a
0709 13:19:51.785777 (+182241us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:51.785834 (+    57us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:51.785880 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.785947 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:51.786073 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":9807,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":7756,"raft.queue_time_us":4386,"spinlock_wait_cycles":39892224,"thread_start_us":181955,"threads_started":1}
I20260709 13:19:51.779855 17975 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.818560 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:51.790743 17943 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:51.725310 17840 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.820576 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:51.722896 17948 consensus_queue.cc:237] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.799686 17464 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.800275 17966 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:51.658309 17859 consensus_queue.cc:260] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.822781 12634 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:51.827260 12672 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:51.797150 17992 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:51.776513 17799 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.808512 17871 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.835587 17843 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:51.851670 17843 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.852176 17843 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:51.799739 17511 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.780774 17921 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:51.838193 17996 log.cc:927] Time spent T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:19:51.814720 17946 consensus_queue.cc:260] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.773844 17958 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.799134 17864 consensus_queue.cc:237] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.833496 17464 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:51.840677 17913 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.815243 17987 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:51.842151 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:51.845114 17992 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.843189 17977 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.820513 17964 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.824960 17975 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:51.849285 17692 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:51.825448 17086 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.807271 17731 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:51.857281 17799 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:51.858723 12673 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:51.883008 17946 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:51.893775 12777 raft_consensus.cc:1240] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:51.867543 17987 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.878702 17992 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:51.869447 17511 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:51.866225 17859 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:51.896536 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.871440 17921 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:51.870435 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:51.896888 17831 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:51.900632 17964 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:51.873764 17913 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:51.875028 17958 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:51.875804 17543 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.878669 12634 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:51.854609 17871 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.885294 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:51.891270 12625 raft_consensus.cc:2393] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 18: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:51.861126 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:51.840447 17930 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:51.905186 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.882977 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:51.910392 12775 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:19:51.911495 12775 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:51.914952 17731 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:51.916237 17128 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:51.888253 17086 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:51.918502 12775 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:19:51.904465 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:51.919697 12631 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:51.924444 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:51.925380 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:51.925976 12631 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:19:51.926935 12627 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 19: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:51.923695 12632 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:19:51.932241 12632 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:51.941429 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:51.898018 17822 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:51.948686 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.952247 17840 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:52.037811 12673 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:52.038198 12631 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:19:52.039003 17128 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:19:52.039320 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 88086.8us
W20260709 13:19:52.038973 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:52.053861 17977 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:52.054467 17840 consensus_queue.cc:1059] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:52.054837 17961 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:52.055389 17961 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:52.055790 17840 raft_consensus.cc:3055] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:52.056187 17840 raft_consensus.cc:740] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:52.056639 17943 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.045284 17962 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:52.058084 17907 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.058195 17962 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.058694 17907 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:52.059749 12781 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:19:52.060390 17962 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:52.054503 17977 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:19:52.057019 17840 consensus_queue.cc:260] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.062832 17840 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:52.057413 17961 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:52.065412 17943 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:52.066841 17871 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:52.067335 12774 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2894) took 1130 ms. Trace:
I20260709 13:19:52.067996 17543 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:52.053159 17992 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.069698 12702 raft_consensus.cc:2393] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 16: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:52.070634 17992 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:52.073312 12705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 2342) took 1140 ms. Trace:
I20260709 13:19:52.073760 12705 rpcz_store.cc:276] 0709 13:19:50.932584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.292296 (+359712us) service_pool.cc:224] Handling call
0709 13:19:52.073132 (+780836us) spinlock_profiling.cc:243] Waited 77.4 ms on lock 0x7b540006d454. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:52.073253 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":178116480}
I20260709 13:19:52.073802 17464 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.075278 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:52.076632 17464 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:52.080902 12708 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:19:52.084484 17943 consensus_queue.cc:1048] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:52.085343 17662 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.083003 12779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2902) took 1121 ms. Trace:
I20260709 13:19:52.086138 12779 rpcz_store.cc:276] 0709 13:19:50.948562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.973713 (+ 25151us) service_pool.cc:224] Handling call
0709 13:19:50.987901 (+ 14188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:51.486529 (+498628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.069359 (+582830us) spinlock_profiling.cc:243] Waited 498 ms on lock 0x7b540042c2d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:52.069529 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1144780672}
I20260709 13:19:52.067723 12774 rpcz_store.cc:276] 0709 13:19:50.936778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.936929 (+   151us) service_pool.cc:224] Handling call
0709 13:19:50.969325 (+ 32396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:51.049519 (+ 80194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.067098 (+1017579us) spinlock_profiling.cc:243] Waited 79.3 ms on lock 0x7b540019d2d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:52.067268 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182465280}
I20260709 13:19:52.088222 17662 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.090050 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:52.076738 12705 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:52.106753 13379 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:52.108052 12705 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:52.110074 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:52.110728 13378 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:52.111524 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:52.112497 12781 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:52.114121 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:52.114847 12781 raft_consensus.cc:1813] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Handling vote request from an unknown peer 19cd864c053744bebd96093a7982537e
I20260709 13:19:52.115403 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:52.085958 18001 log.cc:927] Time spent T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.176s	user 0.000s	sys 0.005s
I20260709 13:19:52.118511 12781 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 11: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:52.111214 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:52.120177 12706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 2341) took 1202 ms. Trace:
I20260709 13:19:52.120378 12700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 2335) took 1249 ms. Trace:
I20260709 13:19:52.120954 12673 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:52.121743 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:52.122229 13379 raft_consensus.cc:1813] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Handling vote request from an unknown peer 19cd864c053744bebd96093a7982537e
I20260709 13:19:52.122579 13379 raft_consensus.cc:2410] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:19:52.124609 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:52.120570 12706 rpcz_store.cc:276] 0709 13:19:50.918133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.918274 (+   141us) service_pool.cc:224] Handling call
0709 13:19:50.946406 (+ 28132us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:51.149826 (+203420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.119965 (+970139us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b5400107cd4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:52.120122 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":452152576}
I20260709 13:19:52.120961 12700 rpcz_store.cc:276] 0709 13:19:50.871184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.871301 (+   117us) service_pool.cc:224] Handling call
0709 13:19:50.871569 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:51.719653 (+848084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.120148 (+400495us) spinlock_profiling.cc:243] Waited 845 ms on lock 0x7b5400172ad4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:52.120315 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1943232768}
I20260709 13:19:52.136615 12673 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:52.140749 12673 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:52.108460 17734 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.141988 13379 raft_consensus.cc:1240] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:52.150813 12630 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:52.143136 12700 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:52.143056 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:19:52.152621 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:52.153627 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:52.153755 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:52.154963 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:52.155347 17822 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:52.155535 12781 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:52.150691 12780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2904) took 1182 ms. Trace:
I20260709 13:19:52.158744 12780 rpcz_store.cc:276] 0709 13:19:50.968551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.019986 (+ 51435us) service_pool.cc:224] Handling call
0709 13:19:51.038308 (+ 18322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:51.122124 (+ 83816us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:51.122153 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:51.122193 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:51.122234 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.150387 (+1028153us) spinlock_profiling.cc:243] Waited 54.6 ms on lock 0x7b540008a054. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:52.150453 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:52.150616 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":125593472}
I20260709 13:19:52.160204 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:52.160782 12780 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:52.143795 12626 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:52.155077 12706 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:52.146503 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:52.163668 12779 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:52.188243 13378 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
W20260709 13:19:52.192152 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 52439.8us
I20260709 13:19:52.192252 17128 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:52.147624 17871 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:52.219473 17128 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:52.205994 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:52.220083 12781 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:19:52.140434 17961 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:52.226224 12673 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:52.224030 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:52.224442 17128 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:52.140947 17992 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:52.227761 13378 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:52.172327 17801 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.229552 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:52.162319 17662 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:52.196781 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:52.232697 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:52.235110 12628 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:52.235003 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:52.238961 17537 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:52.239498 17537 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:52.240047 17623 raft_consensus.cc:3055] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:52.240500 17623 raft_consensus.cc:740] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:52.241827 17951 raft_consensus.cc:3055] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:52.172124 17734 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.241313 17623 consensus_queue.cc:260] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:52.246419 17987 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:52.206387 17509 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:19:52.249836 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:52.246855 17623 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:52.229892 17086 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:52.247463 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:52.248504 12597 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:52.225343 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:52.255911 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 59266.8us
I20260709 13:19:52.256098 12703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 708) took 1134 ms. Trace:
I20260709 13:19:52.242247 17951 raft_consensus.cc:740] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:52.161506 17975 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:52.192983 17822 consensus_queue.cc:1048] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:19:52.263397 18005 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:52.256413 17509 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:52.236552 17731 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:52.264806 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:52.266310 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:52.305480 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:52.305896 12632 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:19:52.307448 12673 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:52.268029 17987 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:52.246474 17907 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:52.269147 12628 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:19:52.269218 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:52.282929 17843 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.298172 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:52.314713 12775 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:52.316069 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:52.309964 17128 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:52.291455 12672 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:52.231088 17961 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:52.264484 17871 raft_consensus.cc:697] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:52.266527 12780 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:19:52.324492 12629 raft_consensus.cc:2410] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:19:52.326849 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:52.327944 12776 raft_consensus.cc:2393] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 12: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:52.329123 12673 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:52.261219 17537 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:52.311867 12631 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:52.316987 12781 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:52.337884 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:52.338514 12780 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:19:52.339474 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:52.340497 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:52.340554 12779 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:19:52.341043 12776 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:52.298316 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:52.302027 18005 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:52.249553 17801 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.315160 17734 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:52.319844 17891 consensus_meta.cc:287] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.546s	user 0.009s	sys 0.003s
I20260709 13:19:52.346397 18005 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:52.353888 17891 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.355311 12701 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 19: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:52.356928 12777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 2535) took 1068 ms. Trace:
I20260709 13:19:52.357466 12777 rpcz_store.cc:276] 0709 13:19:51.288040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.894664 (+606624us) service_pool.cc:224] Handling call
0709 13:19:51.903954 (+  9290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:51.907600 (+  3646us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:51.907631 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:51.908055 (+   424us) log.cc:844] Serialized 26 byte log entry
0709 13:19:52.182092 (+274037us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:52.182132 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.182200 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:52.356600 (+174400us) raft_consensus.cc:1685] finished
0709 13:19:52.356633 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:52.356843 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3422,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":10918,"thread_start_us":247001,"threads_started":1,"wal-append.queue_time_us":378482}
I20260709 13:19:52.309365 17987 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:52.336105 17943 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:52.310995 17831 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:52.319929 12672 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:52.346408 12781 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:19:52.349383 17871 consensus_queue.cc:237] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.330703 17921 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:52.385816 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:52.386727 13378 raft_consensus.cc:2393] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 9: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
W20260709 13:19:52.390172 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 70325us
W20260709 13:19:52.397825 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:52.363893 12776 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:19:52.385871 17963 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:52.430419 17963 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:52.432700 17963 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:52.435496 17128 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:52.423009 17987 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:52.450644 17843 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:52.451341 17987 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.452344 17942 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:52.452894 12709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 2333) took 1592 ms. Trace:
I20260709 13:19:52.346944 17951 consensus_queue.cc:260] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.453231 12709 rpcz_store.cc:276] 0709 13:19:50.860119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.862367 (+  2248us) service_pool.cc:224] Handling call
0709 13:19:50.882091 (+ 19724us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:51.546745 (+664654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.452620 (+905875us) spinlock_profiling.cc:243] Waited 663 ms on lock 0x7b5400404554. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:52.452833 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1524313344}
I20260709 13:19:52.465111 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:52.465788 12777 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:19:52.400411 17831 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:52.478955 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:52.479779 13378 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:52.480831 12746 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:52.460204 17801 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:52.486904 12597 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:52.490365 12597 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:52.435480 17464 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:52.488822 17907 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:52.461006 17987 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:52.493916 12597 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:52.494839 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:52.467201 17963 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:52.497990 12597 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:52.458734 17509 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:52.467892 17943 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:52.494434 17907 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:52.500818 17907 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:52.501104 18010 log.cc:927] Time spent T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 13:19:52.502199 17894 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:52.504021 17940 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:52.506340 18012 log.cc:927] Time spent T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:19:52.506670 17839 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:52.507090 17839 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:52.507481 17839 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:52.510099 12783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2897) took 1572 ms. Trace:
I20260709 13:19:52.510499 12783 rpcz_store.cc:276] 0709 13:19:50.937724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.937845 (+   121us) service_pool.cc:224] Handling call
0709 13:19:50.938165 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:50.938519 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:50.938548 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:50.938999 (+   451us) log.cc:844] Serialized 26 byte log entry
0709 13:19:52.064138 (+1125139us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b18004a28a0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf520d1 00007f4c6d36da0f
0709 13:19:52.230073 (+165935us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:52.230114 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.230255 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:52.509726 (+279471us) raft_consensus.cc:1685] finished
0709 13:19:52.509753 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:52.510030 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22440,"raft.queue_time_us":27407,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":8563,"spinlock_wait_cycles":4243328,"thread_start_us":165750,"threads_started":1,"wal-append.queue_time_us":1346345}
I20260709 13:19:52.512482 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:52.513094 12783 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:52.513162 12708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 715) took 1220 ms. Trace:
I20260709 13:19:52.493041 17901 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:52.513674 12708 rpcz_store.cc:276] 0709 13:19:51.292336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.140384 (+848048us) service_pool.cc:224] Handling call
0709 13:19:52.140739 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:52.162405 (+ 21666us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:52.162442 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:52.162946 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 13:19:52.330114 (+167168us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:52.384952 (+ 54838us) log.cc:844] Serialized 14 byte log entry
0709 13:19:52.385348 (+   396us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.387172 (+  1824us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:52.511024 (+123852us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b18000158e0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c53f3 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:19:52.512807 (+  1783us) raft_consensus.cc:1685] finished
0709 13:19:52.512924 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:52.513094 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36829,"prepare.queue_time_us":54572,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":3641,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":96930,"spinlock_wait_cycles":5243648,"thread_start_us":166749,"threads_started":1,"wal-append.queue_time_us":326002}
I20260709 13:19:52.487741 17831 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:52.520232 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:52.520972 12777 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:52.520404 12783 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:19:52.496086 17801 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:52.536636 12776 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:19:52.532300 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:52.489408 17953 consensus_queue.cc:1059] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:19:52.538542 12776 raft_consensus.cc:1275] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:19:52.540165 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:52.518795 17464 raft_consensus.cc:697] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:52.553735 12672 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:52.533502 17509 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:52.534685 17894 raft_consensus.cc:697] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:52.560773 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:52.519304 17943 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.550350 17864 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:52.519443 17963 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.554720 17543 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:52.578024 12777 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:52.580370 12778 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:19:52.556277 17871 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:52.582196 17992 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:52.563805 17964 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:52.563143 17975 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:52.592291 12626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1897) took 1360 ms. Trace:
I20260709 13:19:52.575203 17930 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:52.571444 17940 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.594843 12626 rpcz_store.cc:276] 0709 13:19:51.231538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.231673 (+   135us) service_pool.cc:224] Handling call
0709 13:19:51.231935 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:52.162081 (+930146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.591925 (+429844us) spinlock_profiling.cc:243] Waited 910 ms on lock 0x7b54000deb54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:52.592189 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2093906816}
I20260709 13:19:52.582605 17961 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:52.590972 17992 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.611238 17992 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:52.582878 17864 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:52.614138 17864 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:52.615417 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:52.284242 12703 rpcz_store.cc:276] 0709 13:19:51.121818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.080061 (+958243us) service_pool.cc:224] Handling call
0709 13:19:52.080429 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:52.080630 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:19:52.080925 (+   295us) log.cc:844] Serialized 14 byte log entry
0709 13:19:52.255781 (+174856us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:52.255835 (+    54us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:52.255875 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.255937 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:52.256043 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19781,"prepare.queue_time_us":6448,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":39,"raft.queue_time_us":25819,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":7222,"thread_start_us":154615,"threads_started":1}
I20260709 13:19:52.623669 17848 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:52.624498 17848 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.629479 17848 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:52.636461 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:52.639240 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:52.641266 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:52.642243 12709 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:52.643379 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:52.571177 17509 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:52.646003 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:52.602843 17871 raft_consensus.cc:697] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:52.649544 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:52.649791 17866 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.654635 12633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 947) took 1163 ms. Trace:
I20260709 13:19:52.655524 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:52.655107 12633 rpcz_store.cc:276] 0709 13:19:51.491560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.493977 (+  2417us) service_pool.cc:224] Handling call
0709 13:19:51.494233 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:51.719781 (+225548us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.654339 (+934558us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b54003458d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:52.654554 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":513356032}
I20260709 13:19:52.656649 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:52.657491 12626 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 14: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:52.650067 17966 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:52.661808 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:52.662635 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:52.662518 12703 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:52.659564 12598 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:52.667332 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:52.661114 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:52.667886 12598 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:52.668733 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:52.604168 17940 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.660480 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:52.670264 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:52.672335 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:52.674257 12634 raft_consensus.cc:2393] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 13: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:52.674865 12598 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:52.672626 12709 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:52.648110 17930 raft_consensus.cc:697] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:52.702234 12747 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:52.675124 12706 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:52.704715 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:52.660317 17623 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:52.673659 17992 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:52.712582 18019 log.cc:927] Time spent T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:19:52.712323 17992 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.666667 17961 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.668174 17848 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.714905 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:52.672659 17987 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:52.676589 17964 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:52.708640 17086 raft_consensus.cc:2764] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:19:52.705819 17946 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:52.667706 17966 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.646457 17464 consensus_queue.cc:237] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.735878 18028 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:52.736264 12747 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:52.736512 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:52.663791 17866 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.710151 17953 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:52.668809 17920 consensus_queue.cc:1059] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 13:19:52.648394 17894 consensus_queue.cc:237] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:52.726333 17864 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:52.758924 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 51068.2us
I20260709 13:19:52.760876 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:52.761097 12747 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:52.743788 17086 raft_consensus.cc:2764] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:19:52.768677 17086 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:52.769153 17086 raft_consensus.cc:697] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:52.769996 17086 consensus_queue.cc:237] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:52.763345 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:52.732955 17961 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.707950 17623 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:52.770485 17999 log.cc:927] Time spent T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.977s	user 0.007s	sys 0.000s
I20260709 13:19:52.763592 17920 raft_consensus.cc:3055] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:52.745365 17953 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:52.734732 17848 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.742473 17930 consensus_queue.cc:237] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.780052 17509 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:52.761906 17864 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.783241 17961 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:52.741516 17946 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.785806 18026 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:19:52.786862 18023 log.cc:927] Time spent T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:19:52.832041 17946 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:52.832686 12782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 2532) took 1652 ms. Trace:
I20260709 13:19:52.833073 12782 rpcz_store.cc:276] 0709 13:19:51.180011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.539543 (+359532us) service_pool.cc:224] Handling call
0709 13:19:51.539852 (+   309us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:51.540210 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:51.540238 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:51.540791 (+   553us) log.cc:844] Serialized 52 byte log entry
0709 13:19:51.561392 (+ 20601us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:51.561450 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:51.561545 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:52.769885 (+1208340us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:52.832378 (+ 62493us) raft_consensus.cc:1685] finished
0709 13:19:52.832399 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:52.832623 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":103,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"spinlock_wait_cycles":26915072,"thread_start_us":20188,"threads_started":1,"wal-append.queue_time_us":178327}
I20260709 13:19:52.787195 17943 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:52.795143 17623 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:52.789500 17907 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:52.836252 17907 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:52.801724 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 469581us
W20260709 13:19:52.816432 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 56215.1us
I20260709 13:19:52.837800 17966 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:52.839538 17848 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:52.840123 18026 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:52.840605 18026 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:52.802672 17920 raft_consensus.cc:740] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:52.844053 17831 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.746505 17871 consensus_queue.cc:237] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:52.821233 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 60204.1us
I20260709 13:19:52.767241 17537 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.847352 17537 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:52.774137 17963 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:52.843930 17920 consensus_queue.cc:260] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.14, Last appended by leader: 14, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:52.847754 17901 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:52.830966 17864 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:52.858098 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:52.858655 12631 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:52.816740 17953 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:52.819135 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:52.851560 12673 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:52.866446 12598 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:52.848970 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:52.855790 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:52.867754 12776 raft_consensus.cc:2393] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 14: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:52.868860 12598 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:52.856601 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:52.870402 12627 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:52.864876 12633 raft_consensus.cc:1275] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:19:52.872543 12633 pending_rounds.cc:85] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 8
I20260709 13:19:52.872857 12633 pending_rounds.cc:107] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 14.9
W20260709 13:19:52.873251 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:52.874152 12633 raft_consensus.cc:2889] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:52.876833 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:52.877146 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:52.877671 12708 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:52.886034 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:52.888067 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:52.886672 13379 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:19:52.889008 12779 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:52.822883 17840 consensus_queue.cc:1048] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:19:52.889554 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:52.897650 12598 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:52.898352 12778 raft_consensus.cc:2393] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 14: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:52.890799 12627 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:19:52.900099 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:52.900774 12627 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:52.895829 12708 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:19:52.822522 17987 raft_consensus.cc:697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:52.892436 12633 raft_consensus.cc:1275] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:19:52.901916 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:52.902427 12673 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:52.905311 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:52.907730 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:52.908167 12598 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:52.908907 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:52.905403 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:52.901891 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:52.909497 12627 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:52.909937 12708 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:19:52.916605 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:52.912434 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:52.917735 12708 raft_consensus.cc:2393] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 16: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:52.914070 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:52.882400 17864 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:52.903321 18027 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:52.920131 17864 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:52.920573 17864 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:52.897442 17920 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:52.923183 12701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 2344) took 1658 ms. Trace:
I20260709 13:19:52.923547 12701 rpcz_store.cc:276] 0709 13:19:51.265671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.586526 (+320855us) service_pool.cc:224] Handling call
0709 13:19:52.922994 (+1336468us) spinlock_profiling.cc:243] Waited 764 ms on lock 0x7b54001cd4d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:52.923126 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1758185344}
I20260709 13:19:52.924597 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:52.930707 12625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56832 (request call id 3208) took 1257 ms. Trace:
I20260709 13:19:52.931137 12625 rpcz_store.cc:276] 0709 13:19:51.673294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.762430 (+ 89136us) service_pool.cc:224] Handling call
0709 13:19:52.930507 (+1168077us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b540027c254. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:52.930627 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":277234944}
I20260709 13:19:52.932367 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:52.890072 17942 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.935400 17964 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:52.936160 12626 raft_consensus.cc:1240] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:52.852037 18005 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:52.938361 17940 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:52.939304 18005 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:52.939636 12703 raft_consensus.cc:2393] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 16: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:52.881747 17623 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:19:52.940647 12625 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:52.899895 17840 consensus_queue.cc:1243] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [LEADER]: Peer 639ab2db265f4747bb7afb93e833f530 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:19:52.910372 18032 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:52.899264 17963 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:52.942436 17963 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:52.942267 18032 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:52.942939 17963 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:52.943403 12777 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:19:52.945986 17831 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:52.946805 12632 raft_consensus.cc:1240] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 13. Current term is 14. Ops: []
W20260709 13:19:52.918385 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:52.918787 12633 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:52.935107 17942 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:52.919284 12747 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:52.948762 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:52.896827 18031 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:52.964296 12631 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
W20260709 13:19:52.965301 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 46126.6us
W20260709 13:19:52.930770 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:52.969013 12707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 2328) took 2109 ms. Trace:
I20260709 13:19:52.914925 17543 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:52.973431 12707 rpcz_store.cc:276] 0709 13:19:50.858576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.860269 (+  1693us) service_pool.cc:224] Handling call
0709 13:19:50.860528 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:51.518472 (+657944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:52.962109 (+1443637us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b540034f154. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:52.962345 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1511553920}
I20260709 13:19:52.914925 17843 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:52.979358 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:52.887595 17839 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:52.972996 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:52.915692 17943 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:19:52.925251 18026 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:52.949128 17871 consensus_queue.cc:1048] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:19:52.994644 12627 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
W20260709 13:19:53.031829 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 58965.2us
I20260709 13:19:53.013195 12634 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:52.990798 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:53.031275 17848 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:52.916954 18030 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:53.036723 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:52.970032 17086 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:53.035825 12747 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:53.067088 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:53.046851 12672 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:53.079869 12776 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:53.080060 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:53.084748 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:53.085248 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:53.086344 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:53.086408 12700 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
W20260709 13:19:53.091387 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 184287us
I20260709 13:19:53.088074 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:53.088399 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 52670us
I20260709 13:19:52.980664 17843 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:53.098361 12747 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:53.099679 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:53.100224 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 53481us
I20260709 13:19:53.100935 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:53.098155 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:53.104240 12632 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:19:53.106801 12747 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:53.055198 17942 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:53.077948 17848 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.108896 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:53.117848 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:53.110008 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:53.118561 12632 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:53.111756 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:53.111069 12598 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:53.112299 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 138422us
I20260709 13:19:53.122136 13349 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:53.051198 17537 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:53.114368 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:53.055135 17891 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:53.116499 12625 raft_consensus.cc:1275] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:19:53.043358 17839 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:53.116742 12673 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:53.104059 12627 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:53.123962 12626 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 19: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:53.122790 12598 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:53.079514 18030 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:53.126797 13349 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:53.061148 18027 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:53.128012 12598 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:53.128620 18045 consensus_queue.cc:1059] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:19:53.130039 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 57624.4us
I20260709 13:19:53.049793 17922 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:19:53.222268 17946 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:53.222743 17537 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:53.223340 17922 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:53.108479 17907 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:53.219491 12598 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:53.220059 17839 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:53.083746 17864 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:53.226574 17864 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:53.114913 17961 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.227335 17961 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:53.227995 13348 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:53.230154 17623 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:53.231662 17922 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:53.232066 17922 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:53.232429 17922 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:53.049234 17953 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:53.235765 17953 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:53.220256 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 103577us
I20260709 13:19:53.102849 17969 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:53.221089 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.243089 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:53.106611 18039 raft_consensus.cc:2764] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:19:53.229295 17961 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:53.218489 17966 consensus_queue.cc:1059] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:19:53.244778 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:53.127110 17840 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:53.244781 17966 raft_consensus.cc:3055] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:53.246057 17840 raft_consensus.cc:697] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:53.246203 17966 raft_consensus.cc:740] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:53.246944 17840 consensus_queue.cc:237] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:53.247045 17966 consensus_queue.cc:260] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:53.248592 12701 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:53.248310 17966 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:53.249475 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:53.124935 17848 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:53.252741 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:53.253847 12673 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:53.254844 18039 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:53.253312 12701 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:53.255321 18039 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.255757 18039 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:53.041500 17987 consensus_queue.cc:237] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:53.265368 12673 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:53.266045 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:53.266459 12673 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:53.129594 18031 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:53.255870 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:53.130863 18030 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:53.274260 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:53.274850 13379 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:53.107792 17963 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:53.219354 17086 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:53.276180 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 169429us
I20260709 13:19:53.219234 17942 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:53.282392 12673 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:53.288723 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:53.292723 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:53.107402 17843 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:53.270529 17824 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:53.299875 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:19:53.301057 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 174390us
I20260709 13:19:53.275503 18041 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.283879 17865 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:53.301827 12747 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:53.317579 18026 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:53.318325 18026 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:53.318692 18026 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.319016 18026 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:53.323047 17824 raft_consensus.cc:697] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:53.324102 17824 consensus_queue.cc:237] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:53.326938 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:53.327771 12700 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 19: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:53.328606 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:53.329172 12700 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:53.332403 17866 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:53.334118 18040 log.cc:927] Time spent T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:19:53.334842 17865 raft_consensus.cc:697] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:53.298563 17943 consensus_queue.cc:1048] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:19:53.335983 17865 consensus_queue.cc:237] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:53.295325 17086 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:53.339572 17969 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:53.340683 17086 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:53.341701 18032 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:53.344219 12625 raft_consensus.cc:1275] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:19:53.298077 17891 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:53.345800 17891 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.346271 17891 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:53.348577 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:53.350723 13379 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:19:53.333961 17882 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:53.301093 17946 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:53.270453 17922 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:53.355173 17942 consensus_queue.cc:1048] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:19:53.297650 17961 consensus_queue.cc:1059] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:19:53.301458 17907 consensus_queue.cc:1059] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:19:53.360257 18026 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:53.382678 12700 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:53.353273 17882 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:19:53.393567 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 90357.4us
I20260709 13:19:53.394716 12700 raft_consensus.cc:1240] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:53.396222 13349 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:53.315951 18041 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:53.403051 18041 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:53.397996 17882 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:53.407419 17922 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:53.410427 17963 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:53.410686 12630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1887) took 2899 ms. Trace:
I20260709 13:19:53.354089 18039 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:53.411123 12630 rpcz_store.cc:276] 0709 13:19:50.511078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.515624 (+  4546us) service_pool.cc:224] Handling call
0709 13:19:50.515909 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:52.151380 (+1635471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.410412 (+1259032us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b540004a1d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:53.410596 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3740773760}
I20260709 13:19:53.411891 12701 raft_consensus.cc:2368] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15.
I20260709 13:19:53.413055 18039 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:53.414738 12701 raft_consensus.cc:1240] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:53.399070 17946 raft_consensus.cc:3055] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:53.418051 12701 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:53.418221 17946 raft_consensus.cc:740] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:53.419157 12632 raft_consensus.cc:3055] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:53.419301 12701 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:53.419662 12632 raft_consensus.cc:740] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:53.420503 12632 consensus_queue.cc:260] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:53.420735 17848 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:53.421931 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:53.422560 12625 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:19:53.421484 12632 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:53.429009 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:53.431932 13381 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:53.429992 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:53.430850 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:53.433986 13378 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:19:53.438021 12630 raft_consensus.cc:1240] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:53.448187 12709 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:53.448156 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 2008) took 2024 ms. Trace:
I20260709 13:19:53.449658 12702 rpcz_store.cc:276] 0709 13:19:51.423390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.859211 (+435821us) service_pool.cc:224] Handling call
0709 13:19:53.447861 (+1588650us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54002481d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:53.448069 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370193152}
I20260709 13:19:53.457598 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:19:53.457198 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 155432us
W20260709 13:19:53.458822 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:53.459903 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 63810.9us
I20260709 13:19:53.460856 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:53.460812 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:53.492162 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.494992 12700 pending_rounds.cc:85] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 7
I20260709 13:19:53.495499 12700 pending_rounds.cc:107] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:19:53.495857 12700 raft_consensus.cc:2889] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:53.511909 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.516736 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:53.422940 17848 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:53.522101 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.523957 12702 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:19:53.525935 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:53.526664 12708 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:53.419205 17946 consensus_queue.cc:260] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:53.528503 17946 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:53.530545 17992 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:53.518824 18041 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:53.533878 17662 raft_consensus.cc:3055] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:53.534312 17662 raft_consensus.cc:740] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:53.535375 13382 raft_consensus.cc:2393] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 17: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
W20260709 13:19:53.536862 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 52546us
I20260709 13:19:53.537192 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:53.538036 13349 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:53.539953 13378 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:19:53.541981 12775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2914) took 2087 ms. Trace:
I20260709 13:19:53.542538 12775 rpcz_store.cc:276] 0709 13:19:51.454294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.934513 (+480219us) service_pool.cc:224] Handling call
0709 13:19:51.934778 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:52.321076 (+386298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.541631 (+1220555us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b54001ebc54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:53.541895 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":873268096}
I20260709 13:19:53.537870 12630 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:53.547602 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:53.548521 12630 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:53.550922 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:53.547811 12673 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:53.535019 18026 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:53.576593 17882 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:53.460880 17863 consensus_queue.cc:1059] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.121s
I20260709 13:19:53.579031 18047 consensus_queue.cc:1059] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:53.580658 18047 raft_consensus.cc:3055] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:53.581161 18047 raft_consensus.cc:740] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:19:53.581995 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 44039.6us
I20260709 13:19:53.582582 17922 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:53.583372 17882 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:53.583688 18050 log.cc:927] Time spent T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
I20260709 13:19:53.638033 13349 leader_election.cc:400] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 10. Current term is 11.
I20260709 13:19:53.638408 13349 leader_election.cc:403] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:53.638649 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:53.639345 13349 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:53.639842 12630 raft_consensus.cc:2368] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 13. Current term is 14.
I20260709 13:19:53.592996 17891 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:53.602015 17951 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:53.606685 12629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56832 (request call id 3225) took 1378 ms. Trace:
W20260709 13:19:53.665810 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.609745 17942 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:53.611424 17860 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:53.619771 12783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 2563) took 1296 ms. Trace:
I20260709 13:19:53.619880 12625 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:19:53.641775 18051 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:19:53.631242 17890 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:53.591635 18030 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:53.640561 13381 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:19:53.666051 12629 rpcz_store.cc:276] 0709 13:19:52.228719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.228842 (+   123us) service_pool.cc:224] Handling call
0709 13:19:53.606476 (+1377634us) spinlock_profiling.cc:243] Waited 69.1 ms on lock 0x7b54002fe9d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:53.606618 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":158875392}
I20260709 13:19:53.667389 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:53.668022 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:53.669229 12780 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:53.676564 12775 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:19:53.632287 12632 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:19:53.677163 12632 pending_rounds.cc:85] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 8
I20260709 13:19:53.676862 12747 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:53.669636 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:53.677428 12632 pending_rounds.cc:107] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:19:53.677736 12632 raft_consensus.cc:2889] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:53.677800 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:53.678364 12774 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:53.671216 12783 rpcz_store.cc:276] 0709 13:19:52.322963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.523361 (+200398us) service_pool.cc:224] Handling call
0709 13:19:52.523674 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:52.595429 (+ 71755us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:52.595496 (+    67us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:52.595550 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:52.595589 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.619474 (+1023885us) spinlock_profiling.cc:243] Waited 71.5 ms on lock 0x7b5400089b54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:53.619527 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:53.619678 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":164441344}
I20260709 13:19:53.673108 12777 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:19:53.680056 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:53.669387 17942 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.640676 18049 consensus_queue.cc:1059] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:19:53.670811 17843 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:53.682277 18055 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:53.694785 18055 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:53.697007 18055 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:53.666167 17922 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:53.681604 18047 consensus_queue.cc:260] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:53.702809 18047 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:53.706606 17947 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:53.708237 18049 raft_consensus.cc:3055] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:53.708703 18049 raft_consensus.cc:740] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:53.668084 17963 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.683104 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:53.686499 12627 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:53.737454 12774 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 17.
W20260709 13:19:53.740826 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 57794.6us
I20260709 13:19:53.741756 12627 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:53.741591 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:53.682333 17942 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:53.747323 18054 log.cc:927] Time spent T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.357s	user 0.004s	sys 0.000s
I20260709 13:19:53.745106 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:53.682101 17911 raft_consensus.cc:2764] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 13:19:53.709702 18049 consensus_queue.cc:260] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:53.688228 17843 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:53.687896 18030 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:53.771919 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.752071 12746 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:53.772424 12709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 2361) took 1128 ms. Trace:
I20260709 13:19:53.779078 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:53.779668 12774 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:53.778534 12709 rpcz_store.cc:276] 0709 13:19:52.643926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.406146 (+762220us) service_pool.cc:224] Handling call
0709 13:19:53.448017 (+ 41871us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.448886 (+   869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:53.772216 (+323330us) spinlock_profiling.cc:243] Waited 41.5 ms on lock 0x7b54002481d0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:53.772366 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95491968}
I20260709 13:19:53.680814 17890 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:53.788406 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:53.784353 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:53.784538 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:53.789654 12780 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:53.785846 12775 raft_consensus.cc:1275] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:53.779881 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:53.786208 12746 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:53.787093 13381 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:53.749058 18055 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:53.710549 17946 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:53.785709 12598 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:53.788368 13376 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
W20260709 13:19:53.841706 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 56139.8us
I20260709 13:19:53.843104 12598 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:53.791308 12747 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:53.844305 12598 leader_election.cc:400] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15.
I20260709 13:19:53.844617 12598 leader_election.cc:403] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:19:53.792513 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.743678 17975 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:53.847846 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:53.795389 13385 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:19:53.796482 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:53.797900 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:53.849771 12777 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:53.792874 17907 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:53.786907 17848 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:53.798905 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:53.782786 17882 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.742756 17963 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:53.855873 17963 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:53.781126 18026 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:53.802017 12629 raft_consensus.cc:1240] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:19:53.803457 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:53.797897 18049 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:53.858440 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:53.859220 12700 raft_consensus.cc:2393] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 15: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:53.860155 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:53.860682 12700 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:53.806636 18035 log.cc:927] Time spent T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.757s	user 0.008s	sys 0.000s
I20260709 13:19:53.809082 18051 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 10. Current term is 11.
I20260709 13:19:53.745782 17947 consensus_queue.cc:1059] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:19:53.810976 13384 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:53.812919 18021 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:53.791882 18042 log.cc:927] Time spent T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.523s	user 0.000s	sys 0.015s
W20260709 13:19:53.818779 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.820226 17734 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:53.820394 12709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:19:53.821187 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:53.821938 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:53.827046 17891 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:53.828501 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:53.829576 17662 consensus_queue.cc:260] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:53.831267 12775 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:19:53.831912 18046 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:53.890544 18046 raft_consensus.cc:697] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:53.862831 17890 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:53.891485 18046 consensus_queue.cc:237] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:53.834075 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 47941.6us
I20260709 13:19:53.893049 12746 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:53.837255 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:53.838541 17961 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:53.895509 17961 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:53.895931 17961 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:53.900540 18026 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:53.901095 18026 raft_consensus.cc:697] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:53.839120 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:53.834182 17942 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:53.901964 18026 consensus_queue.cc:237] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:53.903517 17942 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:53.787461 12774 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:19:53.794262 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:53.848297 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:53.859004 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:53.804435 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:53.905383 13376 raft_consensus.cc:2393] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 14: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
W20260709 13:19:53.853626 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 62334.5us
I20260709 13:19:53.866207 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:53.906286 13381 raft_consensus.cc:2393] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 16: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:53.906509 12747 leader_election.cc:400] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 13. Current term is 14.
I20260709 13:19:53.906848 12747 leader_election.cc:403] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:19:53.908263 12707 raft_consensus.cc:1240] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 10. Current term is 12. Ops: []
I20260709 13:19:53.875332 12703 raft_consensus.cc:2393] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 13: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:53.848611 18030 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:53.908818 18030 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:53.909219 18030 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:53.901983 12781 raft_consensus.cc:2393] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 17: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:53.911221 12775 raft_consensus.cc:1275] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:19:53.911612 12775 pending_rounds.cc:85] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 8
I20260709 13:19:53.911782 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:53.911259 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:53.912691 12781 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 19: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:53.912904 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:53.913252 13348 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:53.913416 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:53.913779 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:53.878836 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:53.914086 17947 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:53.914198 17963 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:53.914657 17947 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.915102 17947 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:53.914861 17963 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:53.915362 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:53.915937 12782 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:53.917133 17963 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:53.913992 12781 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:53.880899 17734 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:53.913439 12774 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:53.918743 12702 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:53.853143 17966 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:53.866900 17882 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:53.871764 12700 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
I20260709 13:19:53.848342 17975 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:53.867048 17848 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:53.866485 17946 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:53.889004 13379 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11.
I20260709 13:19:53.848498 17987 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15.
I20260709 13:19:53.834764 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:53.874312 12709 raft_consensus.cc:606] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:19:53.880167 17662 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:53.914671 12780 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:53.942502 12779 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:19:53.913847 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:53.880460 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:53.881682 13385 raft_consensus.cc:3055] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:53.875391 12673 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:53.944527 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 69227.6us
I20260709 13:19:53.911823 12775 pending_rounds.cc:107] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:19:53.945420 12775 raft_consensus.cc:2889] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:53.947841 12774 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:19:53.948377 12782 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:19:53.943879 13385 raft_consensus.cc:740] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:53.921537 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:53.950752 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.922497 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:53.923261 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:53.924470 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:53.952332 12707 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:53.925890 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:53.926270 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:53.927294 12777 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:53.931694 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:53.932761 17975 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:53.932964 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:53.933513 13348 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:53.874027 18028 consensus_queue.cc:1059] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 13:19:53.956185 12747 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:53.941982 17734 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:53.958490 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:53.961750 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:53.965978 12747 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:53.971060 12780 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:53.971931 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:53.968580 18047 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:53.975180 18047 raft_consensus.cc:697] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:53.975982 17824 consensus_queue.cc:1059] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:53.976092 18047 consensus_queue.cc:237] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:53.952858 18059 log.cc:927] Time spent T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.076s	user 0.000s	sys 0.002s
I20260709 13:19:53.977789 17891 raft_consensus.cc:3055] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:53.978135 17891 raft_consensus.cc:740] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:53.978909 17891 consensus_queue.cc:260] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:53.980011 17891 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:53.981027 13383 raft_consensus.cc:2410] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:19:53.981715 17963 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:53.982471 17961 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:53.985909 18028 raft_consensus.cc:3055] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:53.986382 18028 raft_consensus.cc:740] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:53.987525 18028 consensus_queue.cc:260] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:53.988973 18028 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:53.990455 17848 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:53.991057 18006 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:53.942193 17946 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:53.966756 12673 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:53.877071 18021 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:53.963060 13385 consensus_queue.cc:260] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:53.998855 12634 raft_consensus.cc:2393] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 15: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:53.999172 13385 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:54.001026 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:54.001190 12704 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:19:53.950459 17966 raft_consensus.cc:697] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:53.958876 17907 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:54.002303 17907 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:54.002568 17966 consensus_queue.cc:237] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:54.004509 17907 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:54.005532 12706 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:19:53.998018 18055 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:53.958302 17951 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 13. Current term is 14.
I20260709 13:19:54.007009 17861 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.007679 17861 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:54.010553 18021 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:54.013693 17864 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:54.014246 17864 raft_consensus.cc:697] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:54.015105 17864 consensus_queue.cc:237] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:54.020013 12780 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:19:54.022670 17882 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:54.024739 13385 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:19:54.025091 13385 pending_rounds.cc:85] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: Aborting all ops after (but not including) 10
I20260709 13:19:54.025295 13385 pending_rounds.cc:107] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:19:54.025713 17848 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:53.998031 17946 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:54.027472 12780 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:54.028738 12780 raft_consensus.cc:1275] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:19:54.025497 13385 raft_consensus.cc:2889] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:54.034181 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:54.034936 12632 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:53.954306 17859 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:54.036121 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:54.036816 12630 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:54.039463 12702 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
I20260709 13:19:54.039721 12707 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:19:54.041468 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:54.046535 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 233556us
I20260709 13:19:54.029161 18030 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:54.052596 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:54.056057 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:54.056020 12780 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:19:54.053323 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:54.058527 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.062731 13376 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:19:54.044332 17537 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:19:54.076819 13381 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:19:54.078519 13385 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:54.083086 12776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 699) took 1197 ms. Trace:
I20260709 13:19:54.086289 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:54.090277 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:54.039441 17848 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.049084 17882 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:54.083518 12776 rpcz_store.cc:276] 0709 13:19:52.885333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.885486 (+   153us) service_pool.cc:224] Handling call
0709 13:19:52.885815 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:53.084474 (+198659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:54.082862 (+998388us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b54000b04d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:54.083026 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":445700864}
I20260709 13:19:54.105103 12629 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:54.103051 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:54.106880 12780 raft_consensus.cc:2368] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13.
I20260709 13:19:54.108891 12779 raft_consensus.cc:1240] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 14. Current term is 15. Ops: []
W20260709 13:19:54.110918 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.110984 12779 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:19:54.111492 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.051612 17907 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:54.052202 17859 raft_consensus.cc:697] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:19:54.119999 12704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38496 (request call id 707) took 2991 ms (client timeout 2529 ms). Trace:
I20260709 13:19:54.125779 17860 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:54.127184 17860 consensus_queue.cc:1048] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:54.128453 17860 consensus_queue.cc:1059] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:54.129715 17930 raft_consensus.cc:3055] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:54.130079 17930 raft_consensus.cc:740] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:54.130825 17930 consensus_queue.cc:260] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:54.135313 12629 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:19:54.137364 17843 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:54.139963 13380 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:19:54.142879 17930 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:54.146749 17843 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:54.146960 17947 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:54.148053 12672 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:54.149215 12773 tablet_service.cc:1558] Processing DeleteTablet for tablet 6609d5f93ab048558ea46e511ef3b2ec with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 4 (current committed config index is 5)) from {username='slave'} at 127.0.0.1:50664
I20260709 13:19:54.149646 18006 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:54.150869 12672 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:54.120414 12704 rpcz_store.cc:269] 0709 13:19:51.121315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.488251 (+366936us) service_pool.cc:224] Handling call
0709 13:19:54.112464 (+2624213us) spinlock_profiling.cc:243] Waited 2.46 s on lock 0x7b540014bed4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:54.112556 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5660658048}
I20260709 13:19:54.151697 12672 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:54.153111 12776 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:54.153270 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:54.155623 12783 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:19:54.144518 17946 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:54.149296 17947 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:54.141386 17961 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:54.166445 12781 raft_consensus.cc:2393] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 15: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:54.170070 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:54.165992 17961 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.171525 17961 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:54.172137 17946 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:54.157570 17734 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:54.173605 17961 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:54.174059 17734 raft_consensus.cc:697] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:54.158614 12672 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:54.168164 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:54.177734 12632 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:19:54.179046 12673 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:54.156594 17951 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:54.113318 17848 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:54.179332 12632 raft_consensus.cc:1275] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:54.159744 17907 raft_consensus.cc:697] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:54.165015 17891 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:54.193608 12630 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
W20260709 13:19:54.211251 12529 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54814 (request call id 221) took 38608 ms (client timeout 15000 ms). Trace:
I20260709 13:19:54.175009 17734 consensus_queue.cc:237] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:54.187340 18021 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:54.185439 18006 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:54.185406 12672 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
W20260709 13:19:54.225622 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 424490us
W20260709 13:19:54.230618 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 46272.4us
I20260709 13:19:54.241050 12626 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:54.246934 17861 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:54.211695 12529 rpcz_store.cc:269] 0709 13:19:15.615527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.630052 (+ 14525us) service_pool.cc:224] Handling call
0709 13:19:54.204141 (+38574089us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:46.822674 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:46.915215 (+ 92541us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:46.915289 (+    74us) write_op.cc:435] Acquired schema lock
0709 13:19:46.915311 (+    22us) tablet.cc:662] Decoding operations
0709 13:19:46.916732 (+  1421us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:46.916858 (+   126us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:46.916896 (+    38us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:19:46.939481 (+ 22585us) tablet.cc:701] Row locks acquired
0709 13:19:46.939541 (+    60us) write_op.cc:260] PREPARE: finished
0709 13:19:46.939772 (+   231us) write_op.cc:270] Start()
0709 13:19:46.939959 (+   187us) write_op.cc:276] Timestamp: P: 1783603186939688 usec, L: 0
0709 13:19:46.940006 (+    47us) op_driver.cc:348] REPLICATION: starting
0709 13:19:46.940764 (+   758us) log.cc:844] Serialized 3209 byte log entry
0709 13:19:47.886051 (+945287us) op_driver.cc:464] REPLICATION: finished
0709 13:19:48.534068 (+648017us) write_op.cc:301] APPLY: starting
0709 13:19:48.545381 (+ 11313us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:48.639262 (+ 93881us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:48.639458 (+   196us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:48.679470 (+ 40012us) spinlock_profiling.cc:243] Waited 906 us on lock 0x7b180013b700. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c5522 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:19:48.719034 (+ 39564us) spinlock_profiling.cc:243] Waited 23 ms on lock 0x7b180013b700. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c5557 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:19:48.884974 (+165940us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:49.124481 (+239507us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=16,mrs_lookups=0
0709 13:19:49.161003 (+ 36522us) write_op.cc:312] APPLY: finished
0709 13:19:49.290400 (+129397us) log.cc:844] Serialized 81 byte log entry
0709 13:19:49.611986 (+321586us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:49.638667 (+ 26681us) write_op.cc:454] Released schema lock
0709 13:19:49.672483 (+ 33816us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:49.824664 (+152181us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:49.829217 (+  4553us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":8341,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":555789,"apply.run_cpu_time_us":26277,"apply.run_wall_time_us":1253117,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37599,"num_ops":8,"prepare.queue_time_us":177914,"prepare.run_cpu_time_us":26696,"prepare.run_wall_time_us":426243,"raft.queue_time_us":457886,"raft.run_cpu_time_us":53111,"raft.run_wall_time_us":540000,"replication_time_us":906083,"spinlock_wait_cycles":55049984,"thread_start_us":1234874,"threads_started":5,"wal-append.queue_time_us":882992,"wal-append.run_cpu_time_us":37012,"wal-append.run_wall_time_us":2365414}]]}
I20260709 13:19:54.241044 17907 consensus_queue.cc:237] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:54.260488 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:54.261300 12626 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:54.192994 17951 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:54.241927 17891 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:54.245231 17963 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:54.258581 18006 raft_consensus.cc:697] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:54.271004 18030 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:54.271204 18006 consensus_queue.cc:237] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:54.265103 18021 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.272367 18021 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:54.263612 17848 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:54.276201 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:54.287492 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.202925 17859 consensus_queue.cc:237] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:54.295163 12706 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:54.298085 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:54.298383 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:54.298954 13380 raft_consensus.cc:2393] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 14: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:54.301334 12700 raft_consensus.cc:1240] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:54.304358 12700 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:19:54.307235 12700 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:19:54.310201 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:54.313942 12700 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:19:54.293947 12673 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:54.269848 17891 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:54.323421 17539 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:19:54.330004 17947 consensus_queue.cc:1059] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:54.350013 17947 raft_consensus.cc:3055] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:54.350440 17947 raft_consensus.cc:740] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:54.351529 17947 consensus_queue.cc:260] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:54.353039 17947 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:54.356945 17951 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:54.360015 17975 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:54.361465 12778 raft_consensus.cc:2393] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 18: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:54.363274 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:54.364172 12704 raft_consensus.cc:2368] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15.
I20260709 13:19:54.367034 12628 raft_consensus.cc:1240] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 18. Current term is 19. Ops: []
W20260709 13:19:54.370164 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.395849 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:54.395016 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:54.400727 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:19:54.401252 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.401764 13383 raft_consensus.cc:2410] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 15 because replica has last-logged OpId of term: 13 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 13.
I20260709 13:19:54.404732 12598 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:54.406656 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:54.407550 13385 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 9.
I20260709 13:19:54.414534 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:54.415459 13381 raft_consensus.cc:2376] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 11. Re-sending same reply.
I20260709 13:19:54.290220 17824 consensus_queue.cc:1048] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.101s
I20260709 13:19:54.290050 17822 consensus_queue.cc:1048] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.101s
I20260709 13:19:54.345801 17955 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:54.289844 18005 consensus_queue.cc:1059] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.101s
I20260709 13:19:54.336089 12530 catalog_manager.cc:5697] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 7 to 9, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 9 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:54.362576 17891 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:54.421583 17537 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.440469 17979 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:54.484658 18021 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:54.477734 17955 raft_consensus.cc:697] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:54.446798 17543 consensus_queue.cc:1059] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:19:54.507372 18005 raft_consensus.cc:3055] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:54.497393 18049 consensus_queue.cc:1059] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 13:19:54.496503 17537 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:54.515201 17975 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:54.499238 18028 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
W20260709 13:19:54.577732 12625 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.030s	sys 0.023s
W20260709 13:19:54.578984 12625 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.030s	sys 0.023s
W20260709 13:19:54.576206 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:54.552270 12747 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.006s	sys 0.011s
W20260709 13:19:54.581722 12747 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.007s	sys 0.011s
I20260709 13:19:54.514869 17979 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:19:54.583340 17975 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.548432 18005 raft_consensus.cc:740] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:54.611419 17975 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:54.612076 18005 consensus_queue.cc:260] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:54.587944 12633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 3266) took 1151 ms. Trace:
I20260709 13:19:54.613732 12633 rpcz_store.cc:276] 0709 13:19:53.436530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.436707 (+   177us) service_pool.cc:224] Handling call
0709 13:19:53.437056 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:53.450595 (+ 13539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:53.450618 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:53.457223 (+  6605us) log.cc:844] Serialized 26 byte log entry
0709 13:19:53.892354 (+435131us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b18004ade20. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf520d1 00007f4c6d36da0f
0709 13:19:54.365796 (+473442us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:54.365838 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:54.365914 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:54.587606 (+221692us) raft_consensus.cc:1685] finished
0709 13:19:54.587629 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:54.587884 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13065,"raft.queue_time_us":13189,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":7665,"spinlock_wait_cycles":27762432,"thread_start_us":473265,"threads_started":1,"wal-append.queue_time_us":1059480}
I20260709 13:19:54.613936 17543 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.588063 17955 consensus_queue.cc:237] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:54.616803 17543 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:54.527961 18031 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 13:19:54.590979 12597 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:54.578552 18047 raft_consensus.cc:3055] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:54.597230 12628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:19:54.599462 18028 raft_consensus.cc:3055] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:54.620478 12628 raft_consensus.cc:606] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:54.601421 17979 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.621186 12628 raft_consensus.cc:614] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:54.621371 17979 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:54.623389 17979 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:54.624027 17979 raft_consensus.cc:2764] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:19:54.609896 17537 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:54.620654 18028 raft_consensus.cc:740] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:54.620426 12705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 711) took 3490 ms. Trace:
W20260709 13:19:54.552213 17966 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.007s	sys 0.011s
W20260709 13:19:54.693823 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 245013us
I20260709 13:19:54.673628 17946 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:54.777366 17975 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:54.777091 17946 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:54.702643 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 276108us
I20260709 13:19:54.778509 17543 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:54.779700 17946 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:54.763969 13385 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 14. Preceding OpId from leader: term: 14 index: 14. (term mismatch)
I20260709 13:19:54.654860 17891 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:54.741497 12773 thread.cc:641] tablet-delete [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.014s	sys 0.003s
W20260709 13:19:54.782342 12773 thread.cc:608] tablet-delete [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.014s	sys 0.004s
W20260709 13:19:54.774061 17865 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.005s	sys 0.013s
W20260709 13:19:54.785378 17882 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.006s	sys 0.007s
W20260709 13:19:54.786062 17882 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.007s	sys 0.007s
I20260709 13:19:54.786790 17882 raft_consensus.cc:697] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:54.667538 18046 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:54.756081 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 430674us
W20260709 13:19:54.741706 12748 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.001s	sys 0.015s
W20260709 13:19:54.841297 12748 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.002s	sys 0.015s
W20260709 13:19:54.769219 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 149237us
I20260709 13:19:54.695801 18073 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:54.781342 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:54.781867 13385 pending_rounds.cc:85] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd: Aborting all ops after (but not including) 13
I20260709 13:19:54.846536 13381 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:19:54.846596 13385 pending_rounds.cc:107] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd: Aborting uncommitted NO_OP operation due to leader change: 13.14
I20260709 13:19:54.757925 12747 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:54.784843 18005 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:54.785527 17865 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.005s	sys 0.014s
I20260709 13:19:54.774577 18047 raft_consensus.cc:740] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:19:54.698734 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.787488 13380 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:19:54.787578 12702 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:54.789950 17891 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:54.855952 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 98147.7us
I20260709 13:19:54.712682 12627 raft_consensus.cc:1240] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:54.856781 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:54.772053 12672 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:54.625219 12705 rpcz_store.cc:276] 0709 13:19:51.131050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:52.112496 (+981446us) service_pool.cc:224] Handling call
0709 13:19:52.112802 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:52.113088 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:19:52.218302 (+105214us) log.cc:844] Serialized 14 byte log entry
0709 13:19:53.866413 (+1648111us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b18002fa700. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf5223a 00007f4c6d3bca2a
0709 13:19:54.362478 (+496065us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:54.392430 (+ 29952us) log.cc:844] Serialized 26 byte log entry
0709 13:19:54.392629 (+   199us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:54.392946 (+   317us) log.cc:844] Serialized 14 byte log entry
0709 13:19:54.393396 (+   450us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:54.393510 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:54.608127 (+214617us) spinlock_profiling.cc:243] Waited 21.4 ms on lock 0x7b18002fa700. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c5522 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:19:54.620106 (+ 11979us) raft_consensus.cc:1685] finished
0709 13:19:54.620137 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:54.620350 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20275,"prepare.queue_time_us":38657,"prepare.run_cpu_time_us":120,"prepare.run_wall_time_us":2922,"raft.queue_time_us":7139,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":29439,"spinlock_wait_cycles":75661568,"thread_start_us":579628,"threads_started":2,"wal-append.queue_time_us":2319916}
I20260709 13:19:54.840425 18049 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:54.844873 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.847247 13385 raft_consensus.cc:2889] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:54.787441 17966 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.008s	sys 0.012s
I20260709 13:19:54.851794 12673 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:54.852073 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:54.849692 12597 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:54.693064 17864 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.000s	sys 0.054s
I20260709 13:19:54.768184 17537 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:54.793782 17947 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:54.804479 17907 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:54.867790 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 95825.4us
I20260709 13:19:54.869705 12673 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:54.870504 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:54.870658 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:54.871942 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:54.852370 18073 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:54.887483 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:54.888892 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:54.888206 12629 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:54.890050 12633 raft_consensus.cc:2393] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 15: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:54.888962 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:54.891877 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:54.625414 18028 consensus_queue.cc:260] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:54.866715 17864 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.000s	sys 0.057s
W20260709 13:19:54.893507 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 44042.1us
I20260709 13:19:54.858762 18077 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:54.847682 18046 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:54.888940 12706 raft_consensus.cc:2393] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 14: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:54.909209 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:54.896708 12747 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:54.860879 18079 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:54.884481 18006 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:54.897918 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:54.868454 18049 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:54.890939 17734 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:54.858824 17859 consensus_queue.cc:1048] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:19:54.865366 17946 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:54.902612 12597 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:54.888266 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:19:54.904424 18092 log.cc:927] Time spent T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.115s	user 0.000s	sys 0.002s
I20260709 13:19:54.904799 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:54.871237 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:54.906848 17865 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:54.896032 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:54.911098 12776 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:19:54.920300 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:54.926528 17975 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:54.883718 17907 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:54.919677 17882 consensus_queue.cc:237] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:54.959182 17907 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:54.959798 18090 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:54.960261 18090 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:54.960340 17734 raft_consensus.cc:697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:54.960690 18090 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:54.961823 17987 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:54.899036 17947 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:54.862499 18051 consensus_queue.cc:1059] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 13:19:54.904102 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:54.934185 12633 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:19:54.934033 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 50131.4us
I20260709 13:19:54.937083 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:54.902669 18087 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:54.907545 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:54.946136 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 43655.8us
I20260709 13:19:54.974499 12597 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:54.903503 18026 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:54.892014 17537 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:54.951390 12778 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:54.948494 17865 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:54.951701 17890 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 13:19:54.961295 17734 consensus_queue.cc:237] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:54.900854 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:54.930351 18091 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:54.964520 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 98888.1us
I20260709 13:19:54.964840 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:54.981415 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:54.986563 12632 raft_consensus.cc:2410] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
W20260709 13:19:54.983091 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 47344us
I20260709 13:19:54.987854 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:54.989390 12782 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:19:54.992812 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:54.993985 12779 raft_consensus.cc:2393] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 15: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:54.995329 12776 raft_consensus.cc:1275] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:54.998747 12783 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 17: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:54.968851 17907 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:55.000844 12672 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:54.933876 18079 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:54.962570 17987 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:54.984647 17865 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:54.950008 18045 consensus_queue.cc:1059] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
W20260709 13:19:55.009543 18078 log.cc:927] Time spent T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:19:55.029150 12598 leader_election.cc:400] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15.
I20260709 13:19:55.029662 12598 leader_election.cc:403] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:19:55.010426 17955 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:55.011145 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:55.011938 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:55.014314 17537 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:55.019086 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:55.023607 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:55.032598 13380 raft_consensus.cc:2410] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:19:54.997205 17966 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:55.033800 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:55.026398 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:54.986716 18087 raft_consensus.cc:697] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:55.027958 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:55.028230 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:55.037616 12779 raft_consensus.cc:2410] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
W20260709 13:19:55.034226 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 59795.1us
I20260709 13:19:55.039427 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:55.040176 12781 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:55.043412 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:55.044162 12708 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:55.049116 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:55.050712 12703 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:19:55.050971 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:55.051623 12700 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:19:55.005247 17891 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:55.053252 12778 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:19:54.985078 18075 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.043s
W20260709 13:19:55.055332 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 158679us
I20260709 13:19:55.102195 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:54.982482 18026 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:55.007409 18006 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:19:55.133957 12781 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:19:55.016144 17930 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:19:55.006172 17907 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:55.099463 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:55.145665 12778 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:55.044703 17662 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15.
I20260709 13:19:54.997786 18091 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:55.053959 17537 consensus_queue.cc:1059] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
W20260709 13:19:55.153815 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 97884us
I20260709 13:19:55.143806 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:55.162041 12672 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:55.133633 18075 consensus_queue.cc:1243] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Peer 3d539f8d95d343d8b1696803f18b30d2 log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:19:55.165200 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:55.166286 12597 leader_election.cc:400] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13.
W20260709 13:19:55.145864 18098 log.cc:927] Time spent T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:19:55.059316 17966 consensus_queue.cc:1059] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.101s
I20260709 13:19:55.035601 18087 consensus_queue.cc:237] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:55.169943 12747 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:55.169457 12597 leader_election.cc:403] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:19:55.190430 12778 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:19:55.156039 18082 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:55.191737 18073 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:55.193454 18096 log.cc:927] Time spent T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
I20260709 13:19:55.176424 18077 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:19:55.165550 18021 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.223547 18026 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:55.224157 18047 consensus_queue.cc:260] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:55.225667 18100 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:55.179843 17979 consensus_queue.cc:1059] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:19:55.227136 12516 catalog_manager.cc:5028] TS 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): tablet 6609d5f93ab048558ea46e511ef3b2ec (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:19:55.166708 17882 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:55.229926 18101 raft_consensus.cc:3055] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:55.230587 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:55.232322 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 72066.9us
I20260709 13:19:55.175132 17946 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:55.175034 17539 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:55.223549 18103 log.cc:927] Time spent T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:19:55.176183 17891 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
W20260709 13:19:55.255394 18095 log.cc:927] Time spent T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.304s	user 0.000s	sys 0.001s
W20260709 13:19:55.264684 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 96101.8us
I20260709 13:19:55.253086 18108 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:55.218886 17859 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:19:55.289225 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:55.256374 18031 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13.
I20260709 13:19:55.257727 18049 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:55.253610 17947 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:19:55.290186 12700 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:55.260948 18101 raft_consensus.cc:740] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:55.263337 18100 raft_consensus.cc:697] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:55.290006 18047 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:55.318198 12672 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:55.311280 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 141433us
I20260709 13:19:55.282626 18045 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:55.263978 12516 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 49462.4us
W20260709 13:19:55.314983 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 53337us
I20260709 13:19:55.326668 12597 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:55.322394 12747 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:55.256578 18021 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:55.336532 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:55.337046 13348 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:55.338186 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:55.285030 17930 consensus_queue.cc:1048] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:19:55.312443 17537 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.338863 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:55.323539 18110 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:55.338899 12633 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:19:55.315613 18112 raft_consensus.cc:3055] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:55.340746 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:55.344772 18109 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:55.315820 18079 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:55.334561 18077 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:55.344339 18049 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:19:55.350682 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:55.292186 18108 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:55.343163 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:55.317912 17543 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
W20260709 13:19:55.350986 18105 log.cc:927] Time spent T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
W20260709 13:19:55.392150 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 49064.6us
I20260709 13:19:55.393654 12747 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:55.395334 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:55.369144 18104 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:55.397871 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:55.359030 18116 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:55.372176 18112 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:55.398640 18073 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:55.331988 18045 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:55.358314 18091 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:55.395090 18120 log.cc:927] Time spent T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:19:55.396721 17882 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:55.385730 18115 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.404690 12747 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:55.401437 18077 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:55.420759 17865 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:55.423250 12701 raft_consensus.cc:2368] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 9. Current term is 10.
I20260709 13:19:55.421801 18045 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:55.423890 18112 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:55.404990 18026 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:55.425323 17865 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:55.407737 17882 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:55.426146 17882 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:55.426478 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:55.427466 18073 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:55.428038 18073 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:55.428536 18073 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:55.372313 17537 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:55.432976 17537 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:55.409614 18100 consensus_queue.cc:237] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:55.435338 18108 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:55.372665 18101 consensus_queue.cc:260] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:55.438880 18101 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:55.377420 18110 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:55.418146 18115 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:55.408560 18104 raft_consensus.cc:697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:55.446096 18104 consensus_queue.cc:237] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:55.453648 18115 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:55.453758 18110 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:55.455960 12779 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:19:55.456801 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:55.457203 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:55.458123 12779 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 8.
I20260709 13:19:55.458588 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:55.459621 13385 raft_consensus.cc:2393] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 15: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:55.461045 12746 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:55.461530 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:55.463799 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:55.464511 13380 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 9.
I20260709 13:19:55.466804 13383 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:19:55.462174 12779 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:19:55.471565 12746 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:55.473163 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:55.474565 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:55.474732 12746 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:55.475236 12782 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:19:55.479014 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:55.479828 12782 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 17.
I20260709 13:19:55.478264 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:55.483500 13348 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:55.487860 12625 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:19:55.487887 13348 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:55.474129 18080 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:19:55.489996 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:55.490927 18123 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:55.491703 18123 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:55.492069 18123 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:55.492410 18123 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:55.459712 18115 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:55.488037 18090 consensus_meta.cc:287] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.527s	user 0.014s	sys 0.000s
I20260709 13:19:55.473989 18077 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:55.480875 18109 raft_consensus.cc:697] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:55.481094 18082 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:19:55.513641 18109 consensus_queue.cc:237] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:55.514856 18046 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:55.498025 18049 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:55.504083 18077 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:55.493863 18026 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 13:19:55.516724 18077 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:55.502125 18115 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:55.510617 18005 consensus_meta.cc:287] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.657s	user 0.012s	sys 0.000s
I20260709 13:19:55.516070 18049 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:55.517777 18049 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:55.519054 18115 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:55.520449 12777 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: [13.13-13.13]
I20260709 13:19:55.521942 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:55.522687 12628 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:55.523337 17907 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:55.524811 18132 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:55.527700 18028 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:55.536270 18110 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.536864 18110 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:55.537910 12707 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:19:55.512434 17539 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:55.501351 17537 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:55.541759 12747 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:55.542774 12779 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:19:55.546088 17882 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:55.546645 18110 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:55.547214 17987 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:55.512902 18073 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:55.549109 13381 raft_consensus.cc:2410] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:19:55.555377 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:55.565362 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:55.566789 17882 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:55.567104 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:55.569142 12708 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:19:55.570662 12708 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 13:19:55.562407 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 235819us
I20260709 13:19:55.562865 12700 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:19:55.572778 18101 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:55.576704 18101 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:55.577416 18082 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:55.577905 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:55.577945 18082 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:55.578573 18082 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:55.578684 18101 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:55.579104 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:55.579954 18115 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:55.580579 18115 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:55.580612 12747 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:55.582155 18100 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:55.582206 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:55.583542 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:55.584307 18100 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:55.585500 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:55.586350 18100 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:55.586561 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:55.587464 12700 raft_consensus.cc:2393] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 14: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:55.587610 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:55.588812 12702 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:19:55.589195 12702 pending_rounds.cc:85] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 11
I20260709 13:19:55.589525 12702 pending_rounds.cc:107] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:19:55.589883 12702 raft_consensus.cc:2889] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:55.591948 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:55.592556 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:55.592615 12633 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:19:55.597023 12625 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:55.544349 17537 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:55.601429 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:55.563566 17543 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:55.504812 18090 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:55.648628 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:55.600193 18021 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:55.649000 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 47654.1us
I20260709 13:19:55.649703 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:55.612118 18045 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.627933 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:55.654695 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:55.654995 12625 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 17.
I20260709 13:19:55.655370 12774 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
W20260709 13:19:55.656030 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 179225us
I20260709 13:19:55.598343 18101 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.612627 17537 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:55.553480 18123 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:55.659085 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:55.667583 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:55.668723 12774 raft_consensus.cc:2410] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 15 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:19:55.652072 18100 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:55.657075 13348 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:55.661162 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:55.614013 18047 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.613880 18080 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:55.601014 18046 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:55.612135 18132 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.647356 18130 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.673525 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:55.607599 18115 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:19:55.675359 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:55.692718 13376 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:55.680883 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:55.679562 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:55.686396 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:55.694123 12774 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:19:55.689486 18130 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:55.684765 18080 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:55.612434 17907 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:19:55.694155 13382 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:55.694316 13378 raft_consensus.cc:2410] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:19:55.695289 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:55.714115 12774 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:19:55.697034 12597 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:55.653488 18075 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:19:55.653970 17543 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:55.651690 17987 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:19:55.675366 18109 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:55.715724 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:55.607292 17882 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.102s
I20260709 13:19:55.662087 18045 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:55.718300 12777 raft_consensus.cc:1275] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:19:55.721693 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:55.666888 18101 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:55.726867 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:55.727939 12781 raft_consensus.cc:2410] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:19:55.708033 18128 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:55.671976 18077 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:55.686228 18047 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:55.720811 18104 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:55.734709 12627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:19:55.731319 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:55.746289 12627 raft_consensus.cc:606] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:55.719133 18110 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:55.726544 17946 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.749751 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:55.735598 18108 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:55.729971 18129 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:55.727075 18046 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:19:55.691530 18132 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:55.758358 12633 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:19:55.743007 18140 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.745909 18104 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:55.745281 18144 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.764865 12747 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:55.733736 18109 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:55.744859 18100 raft_consensus.cc:697] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:55.773237 18123 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:55.754959 18133 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:55.776836 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:55.755589 18108 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:55.762008 18141 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.770910 18104 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:55.756121 17946 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:55.744736 18128 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:55.784993 18115 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:55.752082 18110 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:55.854635 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:55.839566 18108 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:55.855768 12633 raft_consensus.cc:2393] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:55.771140 18140 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:55.865041 18140 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:55.783948 18144 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:55.764427 18129 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:55.861213 18128 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:55.788446 18145 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:55.866729 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:55.882434 13376 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: [13.13-13.13]
I20260709 13:19:55.884450 12633 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:55.838037 17987 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:55.887347 18150 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:55.887935 18150 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:55.889139 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:55.891069 12597 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:55.894451 13349 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:55.893498 17987 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:55.899057 18152 consensus_queue.cc:1059] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:55.897212 17979 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:55.902150 18152 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:55.903759 17979 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:55.903976 18152 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:55.904266 17979 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:55.904546 18152 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:55.905151 18080 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:55.842415 18133 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:55.884850 18145 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:55.849232 18141 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:55.882370 18100 consensus_queue.cc:237] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:55.883208 18115 raft_consensus.cc:697] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:55.936786 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:55.891292 18047 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:55.920305 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:56.021162 12781 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:55.914316 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:56.028357 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:56.030020 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:55.991894 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:55.904003 18142 log.cc:927] Time spent T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:19:56.033953 12707 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:19:56.044407 12707 raft_consensus.cc:606] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:56.044054 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:56.042789 12779 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
W20260709 13:19:56.015223 18139 log.cc:927] Time spent T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.408s	user 0.000s	sys 0.002s
W20260709 13:19:55.928442 18126 log.cc:927] Time spent T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.515s	user 0.002s	sys 0.000s
I20260709 13:19:55.929781 18099 raft_consensus.cc:3055] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:56.054950 12779 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17. Ops: [14.11-16.12]
I20260709 13:19:55.933812 18101 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:55.940341 18073 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:56.018285 18030 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.045989 18112 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:55.935338 18045 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:56.068791 12701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38496 (request call id 729) took 1050 ms. Trace:
I20260709 13:19:56.069175 13381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51762 (request call id 885) took 1036 ms. Trace:
I20260709 13:19:56.069253 12701 rpcz_store.cc:276] 0709 13:19:55.018758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.018865 (+   107us) service_pool.cc:224] Handling call
0709 13:19:56.068587 (+1049722us) spinlock_profiling.cc:243] Waited 391 ms on lock 0x7b540014bed4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:56.068729 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":899163904}
I20260709 13:19:56.069891 13381 rpcz_store.cc:276] 0709 13:19:55.033250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.033505 (+   255us) service_pool.cc:224] Handling call
0709 13:19:56.065529 (+1032024us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b54005f0354. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:56.069094 (+  3565us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1182707072}
I20260709 13:19:55.934927 18109 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:55.933032 18077 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:55.909845 18051 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.058458 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:55.927945 17537 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:55.915584 17543 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:56.060591 12746 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:55.929831 18132 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:55.931257 18110 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:56.016582 18130 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:56.063179 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:56.015309 18145 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:55.904798 18082 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:56.052963 18088 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:55.929176 18152 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:56.099731 18152 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:55.923239 17946 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:56.068838 18030 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:56.102995 18145 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:56.103991 18030 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:56.105715 18145 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:56.113088 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 218703us
I20260709 13:19:56.078043 18129 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:56.079495 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:56.080011 18051 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:55.938033 18049 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:56.075882 13384 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:56.057132 18099 raft_consensus.cc:740] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:56.125972 13377 raft_consensus.cc:2393] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 15: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:56.131261 13383 raft_consensus.cc:1240] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:56.128858 17861 raft_consensus.cc:993] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530: : Instructing follower 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:19:56.114204 18150 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:56.086690 18143 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:56.157419 12746 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:56.162215 13385 raft_consensus.cc:1240] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10. Ops: []
W20260709 13:19:56.210631 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.215727 17543 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:56.216492 17543 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:56.219590 17543 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:56.224633 18107 log.cc:927] Time spent T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.900s	user 0.000s	sys 0.002s
W20260709 13:19:56.233991 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 76650.8us
I20260709 13:19:56.168045 18104 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:56.170835 18141 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:56.175791 13349 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:56.179339 18051 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:56.183579 18128 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.126899 18115 consensus_queue.cc:237] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:56.248906 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 73235.8us
I20260709 13:19:56.202666 18133 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:56.207244 18049 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
W20260709 13:19:56.134152 18142 log.cc:927] Time spent T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:19:56.216710 18099 consensus_queue.cc:260] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:56.233283 17987 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:56.246202 12780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2969) took 1279 ms. Trace:
I20260709 13:19:56.175596 17979 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:56.241673 18080 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:56.259660 12780 rpcz_store.cc:276] 0709 13:19:54.966831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.967181 (+   350us) service_pool.cc:224] Handling call
0709 13:19:54.987031 (+ 19850us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:54.987411 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:54.987449 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:55.001204 (+ 13755us) log.cc:844] Serialized 26 byte log entry
0709 13:19:55.174133 (+172929us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:55.174182 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:55.174272 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:56.223820 (+1049548us) spinlock_profiling.cc:243] Waited 4.23 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:56.243463 (+ 19643us) raft_consensus.cc:1685] finished
0709 13:19:56.243488 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:56.246111 (+  2623us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4506,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":5323,"spinlock_wait_cycles":9727744,"thread_start_us":126305,"threads_started":1,"wal-append.queue_time_us":246172}
I20260709 13:19:56.265038 13349 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:56.244166 18154 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:19:56.282686 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.255730 18051 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:56.297856 17537 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:56.263336 18080 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:56.300648 18080 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:56.302593 18152 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:56.303018 18152 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:56.303373 18152 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:56.303179 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:56.303963 13384 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:56.302067 17979 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:56.309990 17662 raft_consensus.cc:3055] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:56.310762 17662 raft_consensus.cc:740] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:19:56.311007 18138 log.cc:927] Time spent T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.567s	user 0.002s	sys 0.000s
I20260709 13:19:56.314622 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:56.314985 13349 leader_election.cc:400] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 9. Current term is 10.
I20260709 13:19:56.315289 13349 leader_election.cc:403] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:19:56.317239 13384 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:56.311925 17662 consensus_queue.cc:260] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:56.263770 18099 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:56.254957 18108 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:56.255052 18128 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:56.299053 18051 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:56.309616 18154 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:56.321146 18049 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 9. Current term is 10.
I20260709 13:19:56.326267 18154 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:56.318646 17537 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:19:56.325350 18051 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:56.330240 17662 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:56.337553 18082 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:56.337965 18090 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:56.340305 18128 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:56.342725 18152 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:56.345779 18152 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:56.346323 18104 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:56.350911 18108 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:56.355013 18080 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:56.369859 18154 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:56.374491 18080 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:56.377640 18154 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:56.413004 12597 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:56.251382 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:56.426826 12774 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.007s	sys 0.003s
I20260709 13:19:56.250522 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:56.427474 12774 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.007s	sys 0.003s
I20260709 13:19:56.253113 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:56.253791 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:56.253602 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:56.436754 12627 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:19:56.464529 12776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2973) took 1442 ms. Trace:
I20260709 13:19:56.437801 12630 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:19:56.465960 12778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2977) took 1440 ms. Trace:
I20260709 13:19:56.470966 18090 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:56.473083 18112 consensus_queue.cc:1059] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:56.476186 12630 pending_rounds.cc:85] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 11
I20260709 13:19:56.433465 12709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 2393) took 1141 ms. Trace:
I20260709 13:19:56.434798 12625 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:19:56.439721 12775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 2970) took 1472 ms. Trace:
I20260709 13:19:56.514598 18051 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:56.479494 18090 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:56.510031 12630 pending_rounds.cc:107] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:19:56.441947 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:56.523908 12630 raft_consensus.cc:2889] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:56.528169 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:56.528177 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:56.538616 18154 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.538807 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:56.539817 12701 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:19:56.539664 18154 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:56.540319 17843 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.541126 17843 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:56.542258 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 48037.3us
I20260709 13:19:56.542980 13382 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:56.544018 13385 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:56.544145 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:56.544970 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:56.547001 18110 consensus_queue.cc:1048] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:56.547801 18100 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:56.548697 18100 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:56.550556 18128 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:56.551337 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:56.551534 18128 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:56.552194 13377 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:56.557490 18112 raft_consensus.cc:3055] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:56.469264 17979 consensus_queue.cc:1059] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:19:56.542449 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:56.558073 17861 raft_consensus.cc:1081] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Signalling peer 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:19:56.474697 18104 consensus_queue.cc:1059] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:19:56.564630 17734 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.530202 18090 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:56.572158 13383 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 15: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:56.578536 17843 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:56.578744 12629 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:19:56.578888 12783 raft_consensus.cc:1240] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:56.550591 17849 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.579598 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:56.485174 12709 rpcz_store.cc:276] 0709 13:19:55.292218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.292367 (+   149us) service_pool.cc:224] Handling call
0709 13:19:55.292775 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:55.293050 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:55.310431 (+ 17381us) log.cc:844] Serialized 14 byte log entry
0709 13:19:55.387474 (+ 77043us) log.cc:844] Serialized 14 byte log entry
0709 13:19:55.526299 (+138825us) spinlock_profiling.cc:243] Waited 1.9 ms on lock 0x7b18003d3780. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c5522 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:19:56.432048 (+905749us) spinlock_profiling.cc:243] Waited 2.49 ms on lock 0x7b18003d3780. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a23fc0a 00007f4c7397f219 00007f4c6a2c7a40 00007f4c6a2c24e4 00007f4c6a2c2455 00007f4c6d3aa060 00007f4c6d3bc716 00007f4c6d3cd287 00007f4c6d3cd208
0709 13:19:56.432509 (+   461us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:56.432645 (+   136us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:56.432797 (+   152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:56.432912 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:56.433119 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2222,"prepare.queue_time_us":17800,"prepare.run_cpu_time_us":132,"prepare.run_wall_time_us":5978,"raft.queue_time_us":2982,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":2984,"spinlock_wait_cycles":10096768,"thread_start_us":76025,"threads_started":1,"wal-append.queue_time_us":99712,"wal-append.run_cpu_time_us":1751,"wal-append.run_wall_time_us":1096310}
I20260709 13:19:56.580395 17849 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:56.581413 17979 raft_consensus.cc:3055] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:56.581720 12780 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:19:56.581967 17979 raft_consensus.cc:740] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:56.476472 12776 rpcz_store.cc:276] 0709 13:19:55.021882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.029800 (+  7918us) service_pool.cc:224] Handling call
0709 13:19:55.042614 (+ 12814us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:56.228401 (+1185787us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:56.228435 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:56.228963 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 13:19:56.461819 (+232856us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:56.461940 (+   121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:56.463892 (+  1952us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54002cae54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:56.463948 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:56.464005 (+    57us) raft_consensus.cc:1685] finished
0709 13:19:56.464016 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:56.464315 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9271,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":2926,"spinlock_wait_cycles":2726082304,"thread_start_us":232018,"threads_started":1,"wal-append.queue_time_us":136363}
I20260709 13:19:56.583928 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:56.584728 12633 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:56.586807 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:56.587565 12781 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:56.589730 17734 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:56.592285 18028 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:56.592851 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:56.593657 12707 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:19:56.594583 18028 consensus_queue.cc:1059] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:56.595186 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:56.596390 12707 raft_consensus.cc:2393] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 15: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:56.596509 12701 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:56.570781 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:56.558066 18112 raft_consensus.cc:740] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:56.589376 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:56.604429 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:56.605495 12782 raft_consensus.cc:1240] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:56.605902 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:56.583472 17979 consensus_queue.cc:260] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:56.607592 17979 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:56.607668 18129 consensus_queue.cc:1175] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3d539f8d95d343d8b1696803f18b30d2" responder_term: 8 status { last_received { term: 8 index: 8 } last_committed_idx: 8 last_received_current_leader { term: 8 index: 8 } }
I20260709 13:19:56.604562 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:56.605516 12776 raft_consensus.cc:2393] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 15: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:56.597452 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:56.617988 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:56.618876 12777 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 15: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:56.618984 13381 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 15: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:56.622030 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:56.622963 12704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 2390) took 1347 ms. Trace:
I20260709 13:19:56.623538 18100 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:56.626223 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:56.598313 17849 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:56.640314 12703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 2392) took 1350 ms. Trace:
I20260709 13:19:56.653585 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:56.492604 12775 rpcz_store.cc:276] 0709 13:19:54.967071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.967257 (+   186us) service_pool.cc:224] Handling call
0709 13:19:55.788467 (+821210us) spinlock_profiling.cc:243] Waited 1.23 ms on lock 0x7b5800193e98. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28db71 00007f4c6a28da22 00007f4c731a6c09 00007f4c72300ee0 00007f4c723ca0ab 00007f4c723ca131 00007f4c7238bfc0 00007f4c72376929 00007f4c723763d2 00007f4c6b8edbd4
0709 13:19:55.788684 (+   217us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:55.789063 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:55.789097 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:55.789624 (+   527us) log.cc:844] Serialized 26 byte log entry
0709 13:19:56.014693 (+225069us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b18001274e0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf520d1 00007f4c6d36da0f
0709 13:19:56.257184 (+242491us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:56.264303 (+  7119us) log.cc:844] Serialized 14 byte log entry
0709 13:19:56.264544 (+   241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:56.264612 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:56.439141 (+174529us) raft_consensus.cc:1685] finished
0709 13:19:56.439181 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:56.439483 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1599,"prepare.queue_time_us":6970,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":1144,"raft.queue_time_us":7419,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":37832,"spinlock_wait_cycles":29043456,"thread_start_us":242221,"threads_started":1,"wal-append.queue_time_us":480096}
I20260709 13:19:56.623364 12704 rpcz_store.cc:276] 0709 13:19:55.275992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.276129 (+   137us) service_pool.cc:224] Handling call
0709 13:19:55.276544 (+   415us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:55.277038 (+   494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:55.277075 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:55.287531 (+ 10456us) log.cc:844] Serialized 52 byte log entry
0709 13:19:55.412931 (+125400us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:55.413276 (+   345us) log.cc:844] Serialized 14 byte log entry
0709 13:19:55.927859 (+514583us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:56.622354 (+694495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:56.622471 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:56.622529 (+    58us) raft_consensus.cc:1685] finished
0709 13:19:56.622617 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:56.622798 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1042344,"prepare.queue_time_us":209,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":1141439,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":2241,"spinlock_wait_cycles":3845504,"thread_start_us":201887,"threads_started":2,"wal-append.queue_time_us":116574}
I20260709 13:19:56.635411 18051 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:56.665887 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:56.673759 12783 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:19:56.677879 18140 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:56.678175 18154 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:56.673604 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:56.679404 12779 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:56.640869 18028 consensus_queue.cc:1059] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
W20260709 13:19:56.711943 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 61408.2us
I20260709 13:19:56.476255 12778 rpcz_store.cc:276] 0709 13:19:55.025026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.199758 (+174732us) service_pool.cc:224] Handling call
0709 13:19:55.200295 (+   537us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:55.394546 (+194251us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:55.485240 (+ 90694us) log.cc:844] Serialized 14 byte log entry
0709 13:19:55.759301 (+274061us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:55.759745 (+   444us) log.cc:844] Serialized 26 byte log entry
0709 13:19:55.760795 (+  1050us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:55.777102 (+ 16307us) log.cc:844] Serialized 14 byte log entry
0709 13:19:55.777356 (+   254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:55.777419 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:56.465388 (+687969us) raft_consensus.cc:1685] finished
0709 13:19:56.465432 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:56.465753 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12258,"prepare.queue_time_us":22532,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":3372,"raft.queue_time_us":75618,"raft.run_cpu_time_us":383,"raft.run_wall_time_us":20203,"thread_start_us":549135,"threads_started":4,"wal-append.queue_time_us":265273}
I20260709 13:19:56.678608 18140 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:56.721050 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:56.722424 12628 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 15: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:19:56.723165 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.727804 13381 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:56.653723 17930 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:56.734221 17930 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:56.739434 18140 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:56.744593 13384 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:56.651252 12703 rpcz_store.cc:276] 0709 13:19:55.289596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.289740 (+   144us) service_pool.cc:224] Handling call
0709 13:19:55.315911 (+ 26171us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:55.316336 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:55.316384 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:55.316977 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 13:19:56.430741 (+1113764us) spinlock_profiling.cc:243] Waited 4.09 ms on lock 0x7b1800288ca0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf520d1 00007f4c6d36da0f
0709 13:19:56.568462 (+137721us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:56.574923 (+  6461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:56.575120 (+   197us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:56.601609 (+ 26489us) raft_consensus.cc:1685] finished
0709 13:19:56.601633 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:56.640119 (+ 38486us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12797,"raft.queue_time_us":303,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":8110,"spinlock_wait_cycles":9406336,"thread_start_us":72101,"threads_started":1,"wal-append.queue_time_us":1159701}
I20260709 13:19:56.749323 18110 raft_consensus.cc:3055] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:56.757300 17930 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:56.785847 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:56.783547 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:56.772912 18110 raft_consensus.cc:740] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:56.801976 13381 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:19:56.799266 17930 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:56.806751 17662 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.807039 17930 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:56.807400 17662 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:56.809842 17987 consensus_queue.cc:1059] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:19:56.810401 13349 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:56.811299 17987 raft_consensus.cc:3055] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:19:56.811434 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 49317.9us
I20260709 13:19:56.811790 17987 raft_consensus.cc:740] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:56.812901 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:19:56.806427 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:56.802421 12707 raft_consensus.cc:3055] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:56.852792 12707 raft_consensus.cc:740] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:56.792516 18183 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:56.823442 12598 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:56.825546 12628 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 19.
W20260709 13:19:56.818275 18175 log.cc:927] Time spent T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:19:56.863476 17930 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:56.825759 18115 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.819684 17734 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:56.863555 17662 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:56.857868 12597 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:56.894197 12633 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 18.
I20260709 13:19:56.813956 18051 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:19:56.900883 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 62445.2us
W20260709 13:19:56.902460 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 90004.6us
I20260709 13:19:56.814205 18180 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:56.914543 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:56.918951 12707 consensus_queue.cc:260] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:56.920846 12707 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:56.916529 17930 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:56.922323 17930 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:56.925134 18183 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:56.925762 17662 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:56.926316 17662 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:56.926797 17662 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:56.927726 12626 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:56.931773 18128 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:56.927776 18183 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:56.813374 17987 consensus_queue.cc:260] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:56.908807 18051 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:56.907018 18100 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:56.925251 17930 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:56.825824 18133 consensus_queue.cc:1059] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.112s
W20260709 13:19:56.940949 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 152485us
I20260709 13:19:56.944964 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:56.951740 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 168859us
I20260709 13:19:56.940665 18051 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:56.949179 12746 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:56.945902 18128 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:56.951458 18115 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:56.962881 18157 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:56.963479 18157 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:56.963841 18157 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:56.964515 18115 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:56.966024 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:56.966114 18028 consensus_queue.cc:1059] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:56.967198 18028 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:56.977329 18100 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:56.980367 18100 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:56.960526 18128 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:56.994920 12628 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:56.967914 18028 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:56.969065 18090 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:56.998399 18090 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:56.994138 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 59356.3us
I20260709 13:19:57.002244 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:56.942714 18180 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:56.987064 12704 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:56.986379 17930 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:56.994787 18128 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:57.021885 18108 raft_consensus.cc:3055] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:57.022414 18108 raft_consensus.cc:740] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:57.023499 18108 consensus_queue.cc:260] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:57.024794 18108 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:57.021011 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 52642.4us
I20260709 13:19:57.026489 12672 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:57.026757 18030 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:57.021762 17966 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.027573 18030 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.027801 17966 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.033896 18133 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:57.021049 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 2060) took 1231 ms. Trace:
I20260709 13:19:57.034773 17966 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:57.014580 18161 log.cc:927] Time spent T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:19:57.036055 17930 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:57.036721 17849 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:57.036806 17930 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.037626 17849 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.040822 18100 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.041522 18100 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:57.042070 12708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 2062) took 1203 ms. Trace:
I20260709 13:19:57.042922 12709 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 18.
I20260709 13:19:57.043457 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:57.042579 12708 rpcz_store.cc:276] 0709 13:19:55.839060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.840706 (+  1646us) service_pool.cc:224] Handling call
0709 13:19:55.841083 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:56.573016 (+731933us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:56.598778 (+ 25762us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:56.711787 (+113009us) log.cc:844] Serialized 26 byte log entry
0709 13:19:56.937590 (+225803us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:56.937669 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:56.940150 (+  2481us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:57.041744 (+101594us) raft_consensus.cc:1685] finished
0709 13:19:57.041761 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:57.041910 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":604208,"raft.queue_time_us":713132,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":118,"thread_start_us":242791,"threads_started":2,"wal-append.queue_time_us":246470}
I20260709 13:19:57.044970 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:57.045822 12774 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:19:57.047112 12672 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:57.048346 18183 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:57.048662 18133 raft_consensus.cc:697] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:57.049014 18183 raft_consensus.cc:697] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:57.040948 13349 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:57.056766 12781 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:19:57.034863 12702 rpcz_store.cc:276] 0709 13:19:55.754203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.754400 (+   197us) service_pool.cc:224] Handling call
0709 13:19:55.766332 (+ 11932us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:56.603513 (+837181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:56.610831 (+  7318us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:56.620234 (+  9403us) log.cc:844] Serialized 26 byte log entry
0709 13:19:56.659048 (+ 38814us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:56.659081 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:56.665225 (+  6144us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:56.968751 (+303526us) raft_consensus.cc:1685] finished
0709 13:19:56.968769 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:56.985696 (+ 16927us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":677625,"raft.queue_time_us":787208,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":235,"thread_start_us":101359,"threads_started":2,"wal-append.queue_time_us":323267}
I20260709 13:19:57.050653 18183 consensus_queue.cc:237] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.059242 17987 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:57.059262 18030 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:57.060597 13380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 838) took 1388 ms. Trace:
I20260709 13:19:57.062969 18180 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:57.063746 12673 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:19:57.064867 18186 log.cc:927] Time spent T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
I20260709 13:19:57.044920 18112 consensus_queue.cc:260] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:57.065783 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 46637.2us
I20260709 13:19:57.050004 18133 consensus_queue.cc:237] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:57.066599 18112 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:57.066845 18099 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:57.067314 18099 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:57.067642 18099 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:57.065148 18180 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
W20260709 13:19:57.091502 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 44589.3us
I20260709 13:19:57.060971 13380 rpcz_store.cc:276] 0709 13:19:55.672210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.672793 (+   583us) service_pool.cc:224] Handling call
0709 13:19:55.673160 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:56.032506 (+359346us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:56.320694 (+288188us) log.cc:844] Serialized 14 byte log entry
0709 13:19:56.545004 (+224310us) log.cc:844] Serialized 14 byte log entry
0709 13:19:56.545264 (+   260us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:56.546142 (+   878us) log.cc:844] Serialized 26 byte log entry
0709 13:19:56.546340 (+   198us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:56.546383 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.060024 (+513641us) spinlock_profiling.cc:243] Waited 75.7 ms on lock 0x7b54003890d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:57.060069 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:57.060238 (+   169us) raft_consensus.cc:1685] finished
0709 13:19:57.060253 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:57.060402 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8492,"prepare.queue_time_us":310122,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":67,"raft.queue_time_us":120949,"raft.run_cpu_time_us":399,"raft.run_wall_time_us":401,"spinlock_wait_cycles":174036352,"thread_start_us":731157,"threads_started":3,"wal-append.queue_time_us":175311}
I20260709 13:19:57.093699 18187 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:57.080631 18030 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:57.090935 18182 consensus_queue.cc:1059] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:19:57.083511 18129 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:57.137909 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 44371.2us
I20260709 13:19:57.145159 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:57.157908 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:57.139878 17966 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:57.201807 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 44065.2us
I20260709 13:19:57.206581 12707 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 13. (term mismatch)
I20260709 13:19:57.206125 13378 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:19:57.203167 12634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:19:57.207055 12707 pending_rounds.cc:85] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 12
I20260709 13:19:57.215248 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:57.215389 12780 raft_consensus.cc:1240] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:57.215809 12779 raft_consensus.cc:1240] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:57.216163 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:57.228386 12707 pending_rounds.cc:107] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 13.13
I20260709 13:19:57.228830 12707 raft_consensus.cc:2889] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:57.222587 12634 raft_consensus.cc:606] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:57.229998 12634 raft_consensus.cc:614] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:57.214829 12778 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:57.219074 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:57.222632 12700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 2061) took 1237 ms. Trace:
I20260709 13:19:57.246497 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:19:57.267948 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:57.269080 12708 raft_consensus.cc:2410] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:19:57.267911 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:57.260769 18135 log.cc:927] Time spent T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 1.525s	user 0.010s	sys 0.000s
I20260709 13:19:57.238302 17662 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.264863 18104 raft_consensus.cc:3055] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:57.272044 18104 raft_consensus.cc:740] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:57.272521 17662 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:57.272838 17539 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.273032 18104 consensus_queue.cc:260] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.267758 12700 rpcz_store.cc:276] 0709 13:19:55.776562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.776819 (+   257us) service_pool.cc:224] Handling call
0709 13:19:55.839814 (+ 62995us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:19:56.613495 (+773681us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:56.613538 (+    43us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:19:56.614694 (+  1156us) log.cc:844] Serialized 78 byte log entry
0709 13:19:56.764561 (+149867us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:56.983936 (+219375us) log.cc:844] Serialized 14 byte log entry
0709 13:19:56.993066 (+  9130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.005435 (+ 12369us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:57.005500 (+    65us) raft_consensus.cc:1685] finished
0709 13:19:57.005519 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:57.013740 (+  8221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":690864,"prepare.queue_time_us":256335,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":724973,"raft.run_cpu_time_us":355,"raft.run_wall_time_us":7554,"thread_start_us":419165,"threads_started":3,"wal-append.queue_time_us":251461}
I20260709 13:19:57.232615 17947 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:57.274389 17947 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:57.210460 18030 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.275671 12706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 2388) took 2005 ms. Trace:
I20260709 13:19:57.276079 12706 rpcz_store.cc:276] 0709 13:19:55.270245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.270395 (+   150us) service_pool.cc:224] Handling call
0709 13:19:55.270799 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:56.657084 (+1386285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:56.660559 (+  3475us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:56.668846 (+  8287us) log.cc:844] Serialized 26 byte log entry
0709 13:19:56.714192 (+ 45346us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:56.740867 (+ 26675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.064081 (+323214us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:57.275210 (+211129us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b54001086d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:57.275267 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:57.275316 (+    49us) raft_consensus.cc:1685] finished
0709 13:19:57.275332 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:57.275509 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1057534,"raft.queue_time_us":1166392,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":145,"spinlock_wait_cycles":314520320,"thread_start_us":171797,"threads_started":2,"wal-append.queue_time_us":121972}
I20260709 13:19:57.277626 18030 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:57.277706 18129 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.278335 17947 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:57.280031 18129 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:57.280716 18182 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:57.275116 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:57.255540 12630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1945) took 1655 ms. Trace:
I20260709 13:19:57.232898 17966 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:57.281957 12630 rpcz_store.cc:276] 0709 13:19:55.600292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.600502 (+   210us) service_pool.cc:224] Handling call
0709 13:19:55.613886 (+ 13384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:56.527931 (+914045us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.255104 (+727173us) spinlock_profiling.cc:243] Waited 823 ms on lock 0x7b540004a1d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:19:57.255440 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1892494592}
I20260709 13:19:57.284082 17849 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:57.284250 17966 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:57.286612 18028 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:57.233539 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.227421 17930 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:57.274122 18104 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:57.296397 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:57.297873 13384 raft_consensus.cc:2393] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 10: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
W20260709 13:19:57.240952 12598 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.570s (SENT) [suppressed 6 similar messages]
I20260709 13:19:57.298555 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:57.298894 12704 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:19:57.303824 12633 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:57.287200 18026 raft_consensus.cc:3055] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:57.304109 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:57.307933 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:57.303642 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:57.213541 18051 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:57.313939 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:57.314596 12706 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:19:57.307144 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:57.318089 12630 raft_consensus.cc:1240] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:57.318931 12704 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:19:57.320925 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.273411 17539 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:57.306696 18026 raft_consensus.cc:740] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:57.302114 18180 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:19:57.310693 18100 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:57.318478 17979 consensus_meta.cc:287] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.710s	user 0.012s	sys 0.004s
I20260709 13:19:57.304618 18030 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 13:19:57.332219 18080 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:57.332939 18080 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.335223 18080 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:57.339506 18128 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:57.340714 12625 raft_consensus.cc:1240] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:57.346622 18026 consensus_queue.cc:260] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:57.348337 18026 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:57.352842 18128 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:57.352634 18099 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:57.360153 12633 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 10.
I20260709 13:19:57.354979 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:57.366477 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:57.366896 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:57.369056 13380 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:19:57.382082 12780 raft_consensus.cc:1813] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Handling vote request from an unknown peer 19cd864c053744bebd96093a7982537e
I20260709 13:19:57.391602 12780 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 12: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:57.393047 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:57.395325 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 92178.4us
I20260709 13:19:57.367259 17734 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:57.399281 12627 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
I20260709 13:19:57.396936 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:19:57.409366 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.387265 12633 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820"
dest_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:56832
W20260709 13:19:57.411403 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 170532us
I20260709 13:19:57.411705 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:57.412528 13381 raft_consensus.cc:2393] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 15: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:57.417520 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:57.418300 18144 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:57.420326 17979 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.420868 17979 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.421811 12632 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:57.422986 17979 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:57.426637 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:57.428458 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:57.429944 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:57.430328 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:57.430852 12628 raft_consensus.cc:2393] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:57.431011 12634 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:57.431740 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:57.434015 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:57.435474 12705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 2053) took 1879 ms. Trace:
I20260709 13:19:57.435863 12705 rpcz_store.cc:276] 0709 13:19:55.555998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:55.556431 (+   433us) service_pool.cc:224] Handling call
0709 13:19:55.556766 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:56.786385 (+1229619us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:19:57.001171 (+214786us) log.cc:844] Serialized 14 byte log entry
0709 13:19:57.312974 (+311803us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:57.365573 (+ 52599us) log.cc:844] Serialized 26 byte log entry
0709 13:19:57.365710 (+   137us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:57.365740 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.366365 (+   625us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:57.435032 (+ 68667us) raft_consensus.cc:1685] finished
0709 13:19:57.435051 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:57.435301 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":950237,"prepare.queue_time_us":217185,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":998001,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":6395,"thread_start_us":587293,"threads_started":3,"wal-append.queue_time_us":381971}
I20260709 13:19:57.400731 18154 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:57.437916 18154 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:57.439101 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:57.441233 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:57.441614 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:57.442292 12632 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:19:57.443352 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:57.443681 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:57.443374 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:57.444332 12774 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:57.401088 18128 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:57.445647 12597 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:19:57.447537 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 181689us
I20260709 13:19:57.452101 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:57.452142 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:57.453593 18144 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.454012 12630 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:19:57.454099 12631 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:57.454308 18144 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:57.452216 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:57.456122 12746 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:57.456662 18144 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:57.457762 18140 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.458221 18140 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.459034 17859 raft_consensus.cc:3055] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:57.459383 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:57.459417 17859 raft_consensus.cc:740] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:57.461997 18115 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:57.462438 18115 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:57.462810 18115 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:57.462632 17859 consensus_queue.cc:260] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:57.462827 12626 raft_consensus.cc:1275] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:19:57.462016 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 47384.4us
I20260709 13:19:57.416513 18108 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:57.382879 18157 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:57.470285 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:57.447397 12633 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:57.467293 12597 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:57.466010 18112 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.474110 17859 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:57.474892 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:57.460521 18140 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:57.482108 12633 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:57.450552 18128 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:57.487632 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:57.494012 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:57.494586 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:57.495028 12629 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 15: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:57.490546 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:57.485526 17979 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.502316 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:57.502926 17979 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.503810 17539 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:57.501925 18112 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.509675 18075 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:57.509822 18030 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:57.510151 18075 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:57.510339 18030 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:57.510612 18075 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:57.510830 18030 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:57.511106 18112 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:57.496575 18128 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:57.464735 17947 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:57.516296 17947 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:57.502385 12631 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:57.517422 12628 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:57.483763 18144 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:19:57.533721 18082 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.888s	user 0.009s	sys 0.012s
I20260709 13:19:57.533779 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:57.534317 12633 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.482859 18133 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:57.517958 12626 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:19:57.537993 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:57.538998 12631 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
W20260709 13:19:57.539577 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.536759 12597 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:57.517804 17979 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:57.548869 12631 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:19:57.538674 18082 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.934s	user 0.010s	sys 0.012s
I20260709 13:19:57.549805 18129 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:57.552711 18201 raft_consensus.cc:993] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e: : Instructing follower 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:19:57.559216 12597 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:57.560410 12633 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:57.547006 18080 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.564792 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:57.556008 18129 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:57.586719 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 221960us
I20260709 13:19:57.605217 12597 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:57.570007 17979 raft_consensus.cc:1081] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Signalling peer 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:19:57.554885 18144 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:57.608129 12673 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:57.578506 18129 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:57.596485 18080 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.561980 18133 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.634635 18080 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:57.635347 18133 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:57.576606 18201 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:57.616461 18157 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:57.617264 18140 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.619230 13378 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:57.620906 18128 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:57.626350 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:57.641269 12631 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
W20260709 13:19:57.651679 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 71820.3us
I20260709 13:19:57.610646 18099 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:57.613766 18108 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:57.659310 12634 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
W20260709 13:19:57.658797 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 63228.6us
I20260709 13:19:57.663125 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:57.690336 12633 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:57.610605 17966 consensus_queue.cc:1059] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:19:57.675191 12597 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:57.696497 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:19:57.694900 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 88375us
I20260709 13:19:57.606787 17539 consensus_queue.cc:1059] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.083s
W20260709 13:19:57.763478 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 108078us
I20260709 13:19:57.759289 12631 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:57.768891 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.693243 18201 raft_consensus.cc:697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:57.658541 18140 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.691423 18108 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:57.746676 18080 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:57.770689 17849 consensus_queue.cc:1059] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:19:57.771173 18112 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:57.779558 18213 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:57.781061 18227 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.763504 18180 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
W20260709 13:19:57.782022 12627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39590 (request call id 983) took 1270 ms (client timeout 1615 ms). Trace:
I20260709 13:19:57.762959 18030 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.663446 18075 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:57.786512 18209 log.cc:927] Time spent T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:19:57.787006 18218 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.790568 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:19:57.795459 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.798929 18051 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:57.800420 18082 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:57.855365 18224 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:57.855918 18224 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.840773 18080 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.858131 18080 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:19:57.858434 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.858628 18080 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:57.859139 18080 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:57.804160 18129 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.804352 18154 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:57.861452 18129 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:57.806005 12632 raft_consensus.cc:1240] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 13. Ops: []
I20260709 13:19:57.806188 18144 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:57.806910 17539 consensus_queue.cc:1048] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:57.808187 17930 raft_consensus.cc:3055] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:57.866194 17930 raft_consensus.cc:740] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:57.867084 17930 consensus_queue.cc:260] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.868285 17930 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:57.813906 18128 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:57.816622 18140 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:57.831176 12633 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 20.
I20260709 13:19:57.690109 18026 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:57.870965 18026 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:57.873046 18026 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:57.807507 12673 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:57.836038 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:57.663601 18187 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.103s
I20260709 13:19:57.875253 18030 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:57.844202 17861 raft_consensus.cc:3055] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:57.876282 17861 raft_consensus.cc:740] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:57.841956 12631 raft_consensus.cc:2468] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:19:57.877127 17861 consensus_queue.cc:260] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:57.843377 12627 rpcz_store.cc:269] 0709 13:19:56.522361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:56.526554 (+  4193us) service_pool.cc:224] Handling call
0709 13:19:57.781835 (+1255281us) spinlock_profiling.cc:243] Waited 744 ms on lock 0x7b54000ddc54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:57.781942 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1710393600}
I20260709 13:19:57.800909 18133 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:57.878386 17861 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:57.878780 18133 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:57.880967 18133 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:57.855831 18082 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:57.837896 18108 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:57.858945 18227 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:57.897012 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 57680us
W20260709 13:19:57.899315 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 91872.3us
I20260709 13:19:57.861316 18080 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:57.906493 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:57.915616 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.916018 13381 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:19:57.854035 18213 raft_consensus.cc:697] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:57.918035 18213 consensus_queue.cc:237] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:57.922968 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:57.885718 18218 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:57.927956 18227 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:57.928845 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:57.929512 13378 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:19:57.934309 18201 consensus_queue.cc:237] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.935315 18082 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:57.911803 18180 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:57.945924 12626 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:19:57.945916 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:57.955607 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:57.965296 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:57.967566 13349 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:57.949386 18180 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:57.970330 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:57.980224 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.980724 12631 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:57.983456 13381 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:19:57.984805 13376 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:19:57.961546 18227 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:57.961841 18224 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:58.016016 18230 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:58.016366 18224 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:58.016975 18224 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:58.017665 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:57.978091 18180 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:57.986514 18140 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:58.013605 18049 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:58.013378 18227 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:57.964609 18082 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:58.014747 12746 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:58.014693 13349 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:57.954604 18187 consensus_queue.cc:1059] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:19:57.966528 18080 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:57.982004 18154 raft_consensus.cc:2804] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:57.986833 18030 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.017529 18230 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:58.034679 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:58.041320 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:58.008710 18100 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:58.043107 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 58201.9us
I20260709 13:19:57.984395 18218 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:57.978916 18183 consensus_queue.cc:1059] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:19:58.047979 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:58.048612 12779 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:36488
I20260709 13:19:58.048688 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:58.045146 18082 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:58.044868 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:58.046681 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:58.057963 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:58.032240 18133 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:58.045917 12777 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
W20260709 13:19:58.072388 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 49565.8us
I20260709 13:19:58.076961 12780 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:58.049456 12775 raft_consensus.cc:1240] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:58.057648 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:58.076519 12778 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:58.034171 18049 raft_consensus.cc:697] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:58.083228 18075 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:58.083451 18181 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:58.084115 18181 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:58.049360 18110 consensus_queue.cc:260] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:58.051512 18115 consensus_meta.cc:287] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.588s	user 0.011s	sys 0.002s
I20260709 13:19:58.085791 18110 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:58.085861 18115 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:58.087106 12776 raft_consensus.cc:2368] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16.
I20260709 13:19:58.087960 18115 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:58.053663 17947 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:58.088657 12597 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:58.089159 18049 consensus_queue.cc:237] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:58.089740 12597 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:58.082739 12746 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:58.090966 18082 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:57.989167 18144 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.082s
I20260709 13:19:58.091537 12774 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:19:58.107890 17947 raft_consensus.cc:697] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:58.108454 18051 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:58.109383 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:58.118191 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:58.119784 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:58.031441 18080 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:58.143437 18080 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:58.068879 18230 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:58.083273 18128 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:58.146148 18128 raft_consensus.cc:697] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:58.147217 18128 consensus_queue.cc:237] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:58.121587 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:58.153942 13376 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:58.123157 12777 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:58.050872 18224 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:58.027868 18140 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:58.117663 18154 raft_consensus.cc:697] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:58.081199 18133 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:58.125890 18051 raft_consensus.cc:697] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 15 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:19:58.093468 18115 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:19:58.068562 18108 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:58.127130 17947 consensus_queue.cc:237] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:58.086701 18181 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:58.128970 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:58.170292 12775 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:58.170897 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 45447.4us
W20260709 13:19:58.140977 18232 log.cc:927] Time spent T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:19:58.059731 18227 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:58.121968 17861 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 13:19:58.127255 13381 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 18: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:58.055596 18030 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:58.051618 18100 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:58.189231 12627 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:58.167647 12597 leader_election.cc:400] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16.
W20260709 13:19:58.190135 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 66198.8us
I20260709 13:19:58.163530 17543 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.174199 18115 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:58.178215 18051 consensus_queue.cc:237] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:58.213831 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:58.183902 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 63073.6us
I20260709 13:19:58.161646 18133 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:58.181723 18202 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:58.191637 17539 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:58.203750 12597 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:58.204327 17543 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:58.206610 18030 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:58.208503 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.241487 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:58.241775 18108 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:58.209482 18100 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:58.242497 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:58.243094 13381 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:19:58.245667 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:58.246522 18100 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:58.246950 18100 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:58.247434 18100 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:58.248512 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:58.249143 12774 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:19:58.211110 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.181488 18112 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:58.250203 12597 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:58.250682 18182 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:58.251271 18182 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:58.251499 17859 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.252249 18108 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:58.252337 17859 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:58.252808 18108 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:58.155664 18082 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:58.240588 17543 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:58.253394 18082 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:58.253487 18108 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:58.253880 18082 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:58.254002 17539 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:58.255615 17966 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:58.256155 17966 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:58.256621 17966 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:58.252473 12597 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:58.257875 17543 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:58.258266 17543 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:58.258407 18182 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:58.258600 17543 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:58.247047 18133 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:58.217381 12625 raft_consensus.cc:1240] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:58.261688 18133 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:58.263202 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:58.263820 13381 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:58.214558 12705 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:19:58.265817 18180 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:58.251459 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:58.209724 12783 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:19:58.267192 12776 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:58.279160 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:58.251619 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 134188us
I20260709 13:19:58.297099 13376 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:19:58.298022 12780 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:19:58.214228 18230 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:58.302098 12777 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:58.300318 12597 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:58.317282 12597 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:58.321130 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.324672 12746 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:58.322161 12705 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
W20260709 13:19:58.329077 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 57507.4us
I20260709 13:19:58.343648 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.323585 18030 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:58.331925 18129 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:58.334451 18133 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:58.337736 18201 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:19:58.364812 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.369901 18115 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:58.355446 18227 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:19:58.375160 18227 consensus_queue.cc:1243] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:19:58.377095 12783 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:19:58.377648 18133 raft_consensus.cc:697] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:58.380949 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:19:58.382375 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.382247 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.381363 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.383080 12776 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:58.397250 18177 raft_consensus.cc:3055] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:58.355882 17987 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.383164 12706 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:19:58.407634 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.410787 12707 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:19:58.382062 18030 raft_consensus.cc:697] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:58.324862 18154 consensus_queue.cc:237] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:58.413455 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.413461 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:58.417201 13376 raft_consensus.cc:2393] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 14: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:58.401954 18177 raft_consensus.cc:740] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:58.418381 12779 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:58.419292 12779 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:58.422842 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:58.430608 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 202124us
I20260709 13:19:58.434151 12747 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:19:58.453061 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.418792 17539 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:58.449970 12629 raft_consensus.cc:1240] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:19:58.437915 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:58.468708 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.417634 18181 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.472426 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:58.474453 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:58.472847 12625 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
W20260709 13:19:58.456591 18237 log.cc:927] Time spent T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:19:58.477813 12634 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:19:58.475996 12631 raft_consensus.cc:1240] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 13. Ops: []
I20260709 13:19:58.410291 17987 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:58.482347 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.412585 18133 consensus_queue.cc:237] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:58.435735 18201 raft_consensus.cc:697] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:58.511777 12627 raft_consensus.cc:1275] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:19:58.475907 13381 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:19:58.426735 18110 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:58.508025 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 87603.7us
W20260709 13:19:58.517838 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:58.510564 18231 log.cc:927] Time spent T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.448s	user 0.000s	sys 0.003s
I20260709 13:19:58.523141 12782 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:19:58.511027 18227 raft_consensus.cc:2764] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:19:58.525283 12630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 3324) took 1008 ms. Trace:
I20260709 13:19:58.523690 12747 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:58.513335 12597 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:58.534675 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:58.535408 12776 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
W20260709 13:19:58.536468 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.536652 18112 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:58.537266 18112 raft_consensus.cc:697] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:58.537856 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:58.538468 18238 consensus_queue.cc:1059] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:58.538893 12747 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:58.538237 18112 consensus_queue.cc:237] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:58.528952 12630 rpcz_store.cc:276] 0709 13:19:57.511928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:57.564037 (+ 52109us) service_pool.cc:224] Handling call
0709 13:19:57.578668 (+ 14631us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:57.604501 (+ 25833us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:57.604545 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:57.657003 (+ 52458us) log.cc:844] Serialized 26 byte log entry
0709 13:19:57.951040 (+294037us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:57.951099 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:57.951185 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:58.356868 (+405683us) spinlock_profiling.cc:243] Waited 6.98 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:19:58.520147 (+163279us) raft_consensus.cc:1685] finished
0709 13:19:58.520177 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:58.520397 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41892,"raft.queue_time_us":7346,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":7803,"spinlock_wait_cycles":16051584,"thread_start_us":183548,"threads_started":1,"wal-append.queue_time_us":290242}
I20260709 13:19:58.543390 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:58.541528 18080 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:58.512219 18181 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:58.542539 18243 log.cc:927] Time spent T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:19:58.521050 18110 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:58.514142 18177 consensus_queue.cc:260] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:58.542639 18026 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:58.582885 12705 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:19:58.582139 18255 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.549636 18075 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:19:58.589839 12629 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:19:58.592545 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:58.559468 18241 log.cc:927] Time spent T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
W20260709 13:19:58.550709 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.549424 18080 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:19:58.624290 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.581418 18140 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:58.552685 17987 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:58.582451 18144 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:58.587400 18201 consensus_queue.cc:237] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:58.647769 12673 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:58.641572 12779 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:58.648705 12673 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:58.542045 18229 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:19:58.530810 18030 consensus_queue.cc:237] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:58.639401 12775 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:19:58.590080 18026 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:58.591229 18100 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:58.642619 18080 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:58.659379 18239 log.cc:927] Time spent T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
W20260709 13:19:58.660125 18254 log.cc:927] Time spent T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 13:19:58.659935 18259 consensus_queue.cc:1059] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:19:58.660503 17930 raft_consensus.cc:3055] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:19:58.660943 18258 log.cc:927] Time spent T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:19:58.705857 17930 raft_consensus.cc:740] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:58.707068 17930 consensus_queue.cc:260] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:58.708465 17930 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:58.712462 18227 raft_consensus.cc:3055] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:19:58.714560 18245 log.cc:927] Time spent T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:19:58.717727 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.718963 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:58.719355 18244 log.cc:927] Time spent T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 13:19:58.719942 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:58.721014 18242 log.cc:927] Time spent T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 13:19:58.721879 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.662954 17861 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:58.725881 17861 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:58.726446 17861 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:58.726658 17966 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:58.728205 12776 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:58.731773 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:58.732509 18180 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:58.734297 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:58.734547 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:58.735126 13381 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:58.656512 18026 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:58.735953 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.665759 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:58.737910 18100 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:58.738163 12706 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:58.651211 17539 consensus_queue.cc:1059] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:19:58.669389 17543 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:58.741168 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:58.742533 17543 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:58.721027 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.670519 12746 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:19:58.744505 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 74058.2us
I20260709 13:19:58.674520 18226 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.746063 18226 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:58.748070 18226 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:58.674270 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:58.672015 18179 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:58.751223 18179 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:58.751678 18179 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:58.678567 12700 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:19:58.697175 12628 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:58.653846 18177 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:58.754011 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:58.625174 18255 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:58.754701 18110 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:58.672969 18251 log.cc:927] Time spent T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:19:58.756847 18255 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:58.734310 18181 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:58.758482 18259 raft_consensus.cc:3055] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:58.758951 18259 raft_consensus.cc:740] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:19:58.759095 18233 log.cc:927] Time spent T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.705s	user 0.001s	sys 0.002s
I20260709 13:19:58.759888 18259 consensus_queue.cc:260] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:58.721336 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:58.758517 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:58.713150 18227 raft_consensus.cc:740] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:58.764715 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:58.765918 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 117258us
I20260709 13:19:58.766678 12633 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 10.
I20260709 13:19:58.734906 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:58.744567 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.754734 13376 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:19:58.770304 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:58.771783 18108 consensus_meta.cc:287] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.518s	user 0.010s	sys 0.002s
I20260709 13:19:58.772727 12781 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:58.772423 18108 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:58.774299 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:58.761395 18259 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:58.785326 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:58.787783 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.787797 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:58.789279 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:58.790365 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:58.792013 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:58.793856 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:58.794198 18248 log.cc:927] Time spent T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.323s	user 0.002s	sys 0.000s
W20260709 13:19:58.788462 18234 log.cc:927] Time spent T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.723s	user 0.001s	sys 0.002s
I20260709 13:19:58.794994 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.796018 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.796509 12707 raft_consensus.cc:2376] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16. Re-sending same reply.
I20260709 13:19:58.797909 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.798432 18213 raft_consensus.cc:3055] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:58.798799 18213 raft_consensus.cc:740] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:58.799068 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:58.799580 18213 consensus_queue.cc:260] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:58.801427 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:58.802551 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:58.803073 12704 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:58.803848 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:58.805063 18227 consensus_queue.cc:260] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:58.806248 12781 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:19:58.806348 17539 consensus_queue.cc:1048] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:58.806811 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:58.808781 12673 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:58.811295 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.812348 18208 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:58.791349 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.797279 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.816108 12700 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
W20260709 13:19:58.817863 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 279023us
I20260709 13:19:58.819564 12747 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:58.812834 18208 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:58.821242 13381 raft_consensus.cc:1240] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:58.777386 18224 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:58.823479 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:58.824934 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:58.828313 12633 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:58.826390 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:58.830013 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.830462 12625 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:58.837103 12704 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:19:58.846467 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:58.846745 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.825829 18229 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:58.822947 18208 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:58.851950 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:58.874511 12775 raft_consensus.cc:2393] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 11: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:58.852865 13380 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: []
W20260709 13:19:58.855218 18268 log.cc:927] Time spent T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.158s	user 0.001s	sys 0.002s
I20260709 13:19:58.855473 12706 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:19:58.840543 18224 raft_consensus.cc:697] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:19:58.895398 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 56020.4us
I20260709 13:19:58.859311 17861 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:58.864832 17859 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:58.851195 18229 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:58.869954 18180 consensus_queue.cc:1059] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:58.871348 12779 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:58.825996 17966 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:58.827515 18112 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.893517 12707 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:19:58.897150 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.898921 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.898654 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:58.912446 12702 raft_consensus.cc:2393] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 18: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:58.894130 12633 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:19:58.903164 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:58.907060 12672 connection.cc:256] RPC call timeout handler was delayed by -1.9494183889999022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.195s for a response
W20260709 13:19:58.913375 12672 connection.cc:256] RPC call timeout handler was delayed by -1.8668318980001004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1686s for a response
W20260709 13:19:58.913868 12672 connection.cc:256] RPC call timeout handler was delayed by -1.5716318980000779s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2014s for a response
I20260709 13:19:58.907696 18259 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:58.908293 17859 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:19:58.829476 18183 consensus_queue.cc:1059] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:19:58.924153 17859 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:58.916904 18108 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:58.926465 17859 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:58.928802 18270 log.cc:927] Time spent T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:19:58.930402 18049 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:58.917382 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.914568 18112 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:58.919241 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:58.920182 18224 consensus_queue.cc:237] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:58.934466 12708 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:19:58.904541 18255 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.921352 12775 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:58.934834 12705 raft_consensus.cc:2410] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:19:58.905431 18229 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:58.938742 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.939499 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:58.937040 12747 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:58.961951 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:58.933763 18082 consensus_meta.cc:287] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.679s	user 0.008s	sys 0.004s
W20260709 13:19:58.961946 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 108841us
I20260709 13:19:58.966400 12703 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:19:58.938966 18259 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:58.970089 18213 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:19:58.971681 18259 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:58.938428 18128 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.972656 18128 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:58.973716 18263 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:58.959065 18255 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:58.975396 18128 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:58.976495 18255 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:58.930857 18049 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:19:58.982959 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 44343.9us
I20260709 13:19:58.984445 12775 raft_consensus.cc:1240] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:58.987944 12747 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:58.958972 18100 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:58.994051 13349 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:59.003001 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:58.992045 18115 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:58.987653 18049 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:59.010842 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 74193.3us
I20260709 13:19:58.968355 18082 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:58.992410 18129 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:59.027457 12775 raft_consensus.cc:1240] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 11. Current term is 13. Ops: []
I20260709 13:19:59.009092 18154 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:59.009716 17861 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:59.007824 18202 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:19:58.991189 18179 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:59.031553 18129 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:59.037794 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 58901.6us
I20260709 13:19:59.033835 18154 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:59.039739 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:59.042517 18273 log.cc:927] Time spent T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:19:59.045855 12778 raft_consensus.cc:2393] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 11: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:58.996088 18100 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:59.081009 18100 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:59.013348 18259 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.081810 18259 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:59.082243 18080 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:59.083988 18259 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:19:59.048590 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.084308 18080 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:59.026839 12672 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:59.051909 18202 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:59.074766 18154 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:59.094234 18112 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:59.095091 12630 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:19:59.095275 12701 raft_consensus.cc:2393] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 16: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:59.095276 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:59.096239 18179 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:59.097124 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:59.098258 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:59.098846 12774 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:59.099761 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:59.100397 12774 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:19:59.101347 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:59.096851 18229 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:59.101905 12774 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:59.102732 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:59.103574 18229 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:59.103933 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:59.104781 12777 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:19:59.111516 18208 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:59.112654 12700 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 17. Current term is 19. Ops: []
I20260709 13:19:59.113031 12774 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:19:59.113762 12781 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:59.115293 12673 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:59.115862 18110 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:59.116955 17734 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:59.117460 18110 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:19:59.118599 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 106008us
I20260709 13:19:59.119134 12780 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:59.096874 17849 raft_consensus.cc:3055] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:59.054498 18215 raft_consensus.cc:3055] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:59.119827 18215 raft_consensus.cc:740] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:59.119685 12598 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:59.120780 18215 consensus_queue.cc:260] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:59.060046 18082 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:59.054970 18269 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:59.064400 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:19:59.122299 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 57989.7us
I20260709 13:19:59.122725 18269 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:59.123128 18269 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:59.123461 18269 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:59.061269 12627 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:59.071921 12747 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:19:59.049142 18026 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:59.068910 12597 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:59.124601 12598 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:59.121255 12780 raft_consensus.cc:1240] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:59.129949 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:59.130753 13378 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:59.145547 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 76780us
I20260709 13:19:59.096959 18179 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:59.152901 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:19:59.161988 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 90123us
I20260709 13:19:59.164155 13380 raft_consensus.cc:1240] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:59.167604 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:59.168725 13384 raft_consensus.cc:2410] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 19 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:19:59.161252 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:59.170130 13381 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:19:59.172693 12777 raft_consensus.cc:1240] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:19:59.159940 12597 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:59.174970 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:59.173400 12783 raft_consensus.cc:1240] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:59.160673 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:59.177393 12634 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:59.178354 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:59.178910 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:19:59.179111 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 59855us
I20260709 13:19:59.180907 12672 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:59.181746 17539 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:59.182898 18082 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:59.183032 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:59.184778 18082 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:19:59.185189 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.163362 18128 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:59.185900 18128 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:59.186272 18128 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:59.189371 12672 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:59.190943 12672 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:19:59.191231 18177 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:59.192775 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:59.194828 18208 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:59.195410 18108 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:59.195719 17861 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:59.195922 18108 raft_consensus.cc:697] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:59.175563 12631 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:59.200428 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:59.202199 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.203869 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:59.205456 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:59.207229 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:59.178982 13385 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:19:59.210044 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:59.210613 12631 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:19:59.211511 18049 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:59.213376 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.214208 12703 raft_consensus.cc:2410] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:19:59.215274 18049 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:59.215991 18154 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:59.219529 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:59.220341 12780 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:59.220528 18129 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:59.119692 17849 raft_consensus.cc:740] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:59.222892 12634 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:19:59.223440 17849 consensus_queue.cc:260] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:59.224622 17849 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:59.225747 18128 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:59.227701 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:59.228574 12701 raft_consensus.cc:2410] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:19:59.196882 18108 consensus_queue.cc:237] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:59.229996 18026 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:59.152462 18255 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.230866 18255 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:59.227737 18128 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:59.232959 18255 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:59.145197 18110 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:59.234784 12709 raft_consensus.cc:2368] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13.
I20260709 13:19:59.151775 18226 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.084s
I20260709 13:19:59.145080 18115 raft_consensus.cc:2764] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:19:59.151986 17966 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:59.237007 12774 raft_consensus.cc:1240] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 19. Current term is 20. Ops: []
W20260709 13:19:59.241676 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.153380 17734 raft_consensus.cc:697] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:59.242074 18154 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:59.243073 17734 consensus_queue.cc:237] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:59.243686 18208 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.247102 18208 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:59.250735 18208 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:59.254416 12778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36488 (request call id 2639) took 1022 ms. Trace:
I20260709 13:19:59.254894 12778 rpcz_store.cc:276] 0709 13:19:58.232266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:58.241232 (+  8966us) service_pool.cc:224] Handling call
0709 13:19:59.254165 (+1012933us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b54000cba54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:19:59.254346 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1847795584}
I20260709 13:19:59.254025 12597 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:19:59.266269 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 123739us
W20260709 13:19:59.272883 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:59.272323 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 98197.9us
I20260709 13:19:59.276161 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.272192 13378 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:19:59.285919 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.286108 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.286888 12706 raft_consensus.cc:2410] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:19:59.287859 12746 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:59.286811 12702 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 17.
I20260709 13:19:59.289033 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:59.290239 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:59.291882 12598 leader_election.cc:336] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.570s (SENT)
I20260709 13:19:59.292333 12598 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:59.296360 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.298970 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.301631 12783 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:19:59.287602 12597 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:59.302049 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.303203 12708 raft_consensus.cc:2410] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:19:59.261967 18215 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:59.305356 13378 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:59.306196 12597 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:59.307104 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:59.308970 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.309077 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:59.314136 12706 raft_consensus.cc:2376] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16. Re-sending same reply.
I20260709 13:19:59.309980 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:19:59.323621 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.327047 13385 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:19:59.286770 18100 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:59.337400 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:59.338618 12702 raft_consensus.cc:2410] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:19:59.327826 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:59.281168 12780 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:19:59.289700 18112 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:59.329887 12597 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:19:59.342157 12628 raft_consensus.cc:1240] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:59.306210 18224 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:59.385023 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:59.311450 18128 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:59.390494 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:59.387436 17987 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:59.383767 18082 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:19:59.393074 12776 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 13:19:59.409258 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 117386us
W20260709 13:19:59.450513 12598 leader_election.cc:336] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.623s (SENT)
I20260709 13:19:59.450994 12598 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:59.394130 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:59.344894 18201 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:59.454962 18201 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:59.455437 18201 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:59.415767 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:59.344038 18226 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 13:19:59.387745 18180 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:19:59.346598 18115 consensus_queue.cc:1059] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 13:19:59.411409 18202 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:59.308355 18255 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:59.441764 13384 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:19:59.456844 12780 raft_consensus.cc:2368] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 10. Current term is 11.
W20260709 13:19:59.475747 13349 leader_election.cc:336] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.532s (SENT)
I20260709 13:19:59.479624 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:59.484687 12779 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:59.447960 18112 raft_consensus.cc:697] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:59.445218 18128 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:59.510190 18286 log.cc:927] Time spent T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
W20260709 13:19:59.512115 18285 log.cc:927] Time spent T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
W20260709 13:19:59.513267 18288 log.cc:927] Time spent T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:19:59.517211 12779 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9.
W20260709 13:19:59.553421 12598 leader_election.cc:336] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.785s (SENT)
W20260709 13:19:59.553668 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.525713 18180 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:59.526468 12517 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 4c13fa2aff324f65ad0bf6191ea68fab with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 639ab2db265f4747bb7afb93e833f530 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:19:59.534972 18292 log.cc:927] Time spent T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:19:59.542225 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:59.558195 12780 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:59.545795 18229 raft_consensus.cc:3055] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:59.478430 18110 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:19:59.546906 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:59.547052 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 71334.4us
W20260709 13:19:59.564627 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 217431us
I20260709 13:19:59.548844 18112 consensus_queue.cc:237] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:59.549881 18128 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:59.563079 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:59.568789 12779 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 15: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:59.564400 12701 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:59.566987 12598 leader_election.cc:336] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.504s (SENT)
I20260709 13:19:59.554502 17987 raft_consensus.cc:697] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:59.574188 12780 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 18.
W20260709 13:19:59.575975 12598 leader_election.cc:336] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.822s (SENT)
I20260709 13:19:59.562902 18229 raft_consensus.cc:740] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:19:59.592545 18229 consensus_queue.cc:260] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:59.577031 18110 consensus_queue.cc:1059] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:19:59.581405 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:59.594812 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:59.595352 12779 raft_consensus.cc:2376] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 18. Re-sending same reply.
I20260709 13:19:59.595443 12783 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:59.596189 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:59.597144 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:59.600301 12783 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 15: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:59.601444 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:59.602193 12783 raft_consensus.cc:2376] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15. Re-sending same reply.
I20260709 13:19:59.603318 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:59.604038 12783 raft_consensus.cc:2376] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14. Re-sending same reply.
I20260709 13:19:59.604914 18269 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:59.588250 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:59.707101 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:59.708983 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:59.589936 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:59.590708 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:59.591508 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:59.608086 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:59.713745 12775 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 12: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:59.610224 17987 consensus_queue.cc:237] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:19:59.584589 13348 leader_election.cc:336] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.615s (SENT)
I20260709 13:19:59.615170 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:59.621006 18224 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:59.623445 18202 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:19:59.632155 18110 consensus_queue.cc:1243] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer 3d539f8d95d343d8b1696803f18b30d2 log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:19:59.636920 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:59.638538 18179 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:19:59.639091 12598 leader_election.cc:336] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.867s (SENT)
I20260709 13:19:59.639472 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:59.682422 12673 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:19:59.640146 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 58716.2us
I20260709 13:19:59.723631 12598 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:59.650501 18201 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:59.652238 18227 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:59.657918 18180 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:59.684918 12627 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:19:59.663007 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.674072 18154 consensus_queue.cc:1059] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
I20260709 13:19:59.653519 18128 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:59.687444 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.745642 12704 raft_consensus.cc:2393] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 19: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:19:59.689024 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.690001 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.676429 12701 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:19:59.690779 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.691843 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.692893 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.748008 12703 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:19:59.679388 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:59.749007 12774 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:19:59.694954 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.589486 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:59.715029 12783 raft_consensus.cc:2368] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 10. Current term is 11.
I20260709 13:19:59.715510 12781 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:19:59.711766 12776 raft_consensus.cc:2376] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 9. Re-sending same reply.
I20260709 13:19:59.722052 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:59.641908 18229 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:59.726861 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 175720us
W20260709 13:19:59.762347 13348 leader_election.cc:336] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.760s (SENT)
W20260709 13:19:59.763119 13348 leader_election.cc:336] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.561s (SENT)
I20260709 13:19:59.682749 18129 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:19:59.733737 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 94685.5us
W20260709 13:19:59.765911 12530 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54800 (request call id 231) took 44075 ms (client timeout 15000 ms). Trace:
W20260709 13:19:59.734599 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 52278.8us
I20260709 13:19:59.737284 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:59.768292 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:59.768991 12775 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:59.739853 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:59.739705 12597 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:19:59.748469 12708 raft_consensus.cc:2393] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 16: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:59.750943 13381 raft_consensus.cc:1240] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:59.752451 12778 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:59.749956 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:59.768899 12673 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:59.773087 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:59.727525 18224 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:59.774257 12700 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:59.774660 12634 raft_consensus.cc:2393] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 19: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:19:59.774756 13349 leader_election.cc:336] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.884s (SENT)
I20260709 13:19:59.782104 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:59.783775 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:59.784026 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:59.786192 12597 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:19:59.787279 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.787961 12704 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
I20260709 13:19:59.790407 18269 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:19:59.749353 18049 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:19:59.749204 18215 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:59.792513 18215 raft_consensus.cc:697] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:59.793042 18110 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:59.793512 18110 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:59.793963 18110 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:59.794562 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:59.795557 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:59.795918 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:59.796689 12629 raft_consensus.cc:2393] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 11: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:59.796787 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:59.783600 18179 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:59.797559 12632 raft_consensus.cc:2393] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 16: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:59.798205 18179 raft_consensus.cc:697] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 18 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:19:59.798357 12633 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:19:59.798725 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:59.799402 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:59.799302 18179 consensus_queue.cc:237] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:19:59.797677 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:59.800695 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:59.788841 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.801435 12633 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 18: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:19:59.777173 17966 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.802079 17966 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:59.802433 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.803812 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.784773 13349 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:59.804673 17966 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:59.804823 12704 raft_consensus.cc:2410] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:19:59.799887 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:59.752413 18202 consensus_queue.cc:1059] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
W20260709 13:19:59.805933 13349 leader_election.cc:336] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.662s (SENT)
I20260709 13:19:59.807345 12628 raft_consensus.cc:2393] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 12: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:19:59.812621 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.805703 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:59.806668 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:59.839361 12631 raft_consensus.cc:2393] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 16: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:59.815999 12783 raft_consensus.cc:1275] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
W20260709 13:19:59.839512 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.833484 18154 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:59.833976 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.844964 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:59.835510 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.842485 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.835397 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.848609 12777 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:59.850247 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:59.851006 12777 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:59.839259 12625 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
I20260709 13:19:59.853341 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:59.854103 12783 raft_consensus.cc:2393] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 13: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:59.854632 12700 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 20.
I20260709 13:19:59.858161 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:59.858659 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:59.859243 12776 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:59.862344 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.863062 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:19:59.863883 13377 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:19:59.864923 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:59.868398 12783 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:59.874626 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:19:59.875520 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.877774 12706 raft_consensus.cc:2393] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 16: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:19:59.865680 12781 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 9.
I20260709 13:19:59.880592 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:19:59.863498 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 57587.8us
W20260709 13:19:59.884433 13349 leader_election.cc:336] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.561s (SENT)
I20260709 13:19:59.864231 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:59.884985 13349 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:19:59.889822 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:59.890638 12781 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:19:59.895197 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:19:59.895921 12778 raft_consensus.cc:2393] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 14: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:19:59.896900 12705 raft_consensus.cc:2410] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:19:59.897975 18255 consensus_queue.cc:1059] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:59.899560 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:59.877188 18202 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.901234 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:59.902227 13381 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:59.902334 18295 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:59.903688 18297 raft_consensus.cc:3055] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:59.902247 18129 consensus_queue.cc:1048] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:19:59.904098 18297 raft_consensus.cc:740] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:59.905103 18297 consensus_queue.cc:260] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:59.905702 12702 raft_consensus.cc:2368] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13.
W20260709 13:19:59.907043 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.907358 18297 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:59.905467 18129 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:19:59.916059 18154 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:59.917779 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:19:59.902057 18202 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:19:59.766362 12530 rpcz_store.cc:269] 0709 13:19:15.674522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.674783 (+   261us) service_pool.cc:224] Handling call
0709 13:19:59.740870 (+44066087us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:54.763151 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:54.763696 (+   545us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:54.763727 (+    31us) write_op.cc:435] Acquired schema lock
0709 13:19:54.763740 (+    13us) tablet.cc:662] Decoding operations
0709 13:19:54.764099 (+   359us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:54.764165 (+    66us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:54.764196 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:19:54.764408 (+   212us) tablet.cc:701] Row locks acquired
0709 13:19:54.764429 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:19:54.764583 (+   154us) write_op.cc:270] Start()
0709 13:19:54.764688 (+   105us) write_op.cc:276] Timestamp: P: 1783603194764537 usec, L: 0
0709 13:19:54.764707 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:19:54.765373 (+   666us) log.cc:844] Serialized 533 byte log entry
0709 13:19:55.084826 (+319453us) op_driver.cc:464] REPLICATION: finished
0709 13:19:55.336498 (+251672us) write_op.cc:301] APPLY: starting
0709 13:19:55.344545 (+  8047us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:55.371517 (+ 26972us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:55.371667 (+   150us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:55.372466 (+   799us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:55.372628 (+   162us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:19:55.372673 (+    45us) write_op.cc:312] APPLY: finished
0709 13:19:55.372912 (+   239us) log.cc:844] Serialized 25 byte log entry
0709 13:19:55.373093 (+   181us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:55.373182 (+    89us) write_op.cc:454] Released schema lock
0709 13:19:55.373224 (+    42us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:55.392862 (+ 19638us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:55.394662 (+  1800us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":420614,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":232303,"apply.run_cpu_time_us":3773,"apply.run_wall_time_us":51601,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8311,"num_ops":1,"prepare.queue_time_us":213952,"prepare.run_cpu_time_us":97107,"prepare.run_wall_time_us":141603,"raft.queue_time_us":66988,"raft.run_cpu_time_us":39477,"raft.run_wall_time_us":243068,"replication_time_us":286181,"thread_start_us":487094,"threads_started":4,"wal-append.queue_time_us":173885,"wal-append.run_cpu_time_us":6354,"wal-append.run_wall_time_us":1434601}]]}
I20260709 13:19:59.920264 18224 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:19:59.923005 18224 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:59.880553 18080 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:59.847095 17966 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:59.928709 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:59.920264 18202 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:19:59.949571 12625 raft_consensus.cc:2393] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 16: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:19:59.918493 13349 leader_election.cc:336] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.760s (SENT)
I20260709 13:19:59.950527 13349 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:59.951815 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.934221 12776 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:19:59.935045 12777 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
I20260709 13:19:59.936422 13381 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:19:59.942579 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:19:59.953475 13384 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:59.958776 13348 leader_election.cc:336] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Timed out: RequestConsensusVote RPC to 127.11.63.67:41181 timed out after 1.727s (SENT)
I20260709 13:19:59.959120 13349 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:19:59.960213 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:19:59.970033 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:59.971979 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:19:59.975855 12630 raft_consensus.cc:1240] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:59.978533 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:59.979288 12630 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:19:59.982164 12630 raft_consensus.cc:1240] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:59.972086 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:19:59.973040 12775 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:19:59.994062 12629 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:19:59.995720 12707 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:19:59.793516 18215 consensus_queue.cc:237] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:00.004197 12625 raft_consensus.cc:2393] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 19: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:20:00.007498 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:19:59.958536 18269 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:20:00.019418 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 59253.6us
I20260709 13:20:00.026463 13349 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:00.022148 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:00.031505 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:00.032331 12706 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:20:00.035590 12631 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:19:59.967867 17966 raft_consensus.cc:697] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:19:59.993777 18049 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:00.035288 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:00.052667 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:00.002100 18298 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:00.065593 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:00.072656 12705 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:59.949252 18080 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:00.076270 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:00.078990 12779 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:20:00.082185 12707 raft_consensus.cc:3055] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:00.083063 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:00.083063 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:00.085793 13384 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:20:00.086774 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:00.089181 12707 raft_consensus.cc:740] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:00.110010 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:00.111411 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:00.114275 12628 raft_consensus.cc:2368] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15.
I20260709 13:20:00.115978 12631 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 17. Current term is 18. Ops: [15.11-17.12]
I20260709 13:20:00.121388 12631 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:00.075261 18051 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:20:00.113431 18294 log.cc:927] Time spent T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:20:00.107288 18269 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:20:00.135712 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 241626us
I20260709 13:20:00.140601 12707 consensus_queue.cc:260] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:00.142143 12707 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:00.124799 18229 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.154512 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:20:00.179783 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.179986 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:20:00.180331 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.180493 13348 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:00.127007 18201 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:00.184792 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:00.185959 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.187034 12747 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:00.181111 12597 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:00.193292 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:00.193989 12709 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:00.198062 18291 raft_consensus.cc:3055] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:00.198339 12628 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:20:00.198513 18291 raft_consensus.cc:740] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:00.199765 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:00.145427 18128 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:00.205463 12597 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:00.205461 13348 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:00.216194 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:00.219342 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:00.219961 12782 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:20:00.220016 12774 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:00.224033 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:00.228417 12707 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:20:00.230346 13348 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:00.206688 18297 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:20:00.229369 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 237906us
I20260709 13:20:00.233780 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:00.229472 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:00.221673 18201 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:00.236800 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:00.209450 18051 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:00.149314 17966 consensus_queue.cc:237] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:00.188520 18229 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:00.287271 12704 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 18 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:00.302174 12629 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
W20260709 13:20:00.302505 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 72241.2us
I20260709 13:20:00.304462 12709 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:20:00.306785 12625 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:00.302531 12629 pending_rounds.cc:85] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 10
I20260709 13:20:00.307809 12629 pending_rounds.cc:107] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:20:00.308063 12629 raft_consensus.cc:2889] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:00.209321 17947 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.079s
W20260709 13:20:00.288110 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 58716.7us
I20260709 13:20:00.235750 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:00.229211 18049 raft_consensus.cc:2764] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 13:20:00.295867 12778 raft_consensus.cc:2393] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 19: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:00.210240 18112 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:00.295456 12774 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:20:00.297555 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:20:00.300769 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.288679 18297 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:00.288442 18267 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:20:00.319428 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 132428us
I20260709 13:20:00.208643 18291 consensus_queue.cc:260] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:00.302095 18226 raft_consensus.cc:2764] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 13:20:00.324241 18179 consensus_queue.cc:1059] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:20:00.325433 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 272794us
I20260709 13:20:00.314011 18128 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:00.313992 18112 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:00.322278 18297 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:00.318069 18201 raft_consensus.cc:697] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:00.365038 13348 leader_election.cc:400] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 10. Current term is 11.
W20260709 13:20:00.366564 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.376245 13348 leader_election.cc:403] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:20:00.371832 13384 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:00.374385 12598 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:00.322036 18267 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:00.374603 12747 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:00.320165 18295 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:20:00.382092 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:00.386536 12747 leader_election.cc:400] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13.
I20260709 13:20:00.386875 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:00.480896 12747 leader_election.cc:403] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:00.481192 13378 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:20:00.355484 18154 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:00.364079 18080 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:00.376482 18291 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:20:00.485626 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 101801us
W20260709 13:20:00.480365 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:00.496979 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 143335us
I20260709 13:20:00.498744 12705 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
I20260709 13:20:00.481424 18051 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 10. Current term is 11.
I20260709 13:20:00.378641 18112 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:00.495837 12747 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:00.386649 18100 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:00.379380 18267 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:20:00.510660 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 136316us
I20260709 13:20:00.477732 18305 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:20:00.516827 18227 consensus_meta.cc:287] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.773s	user 0.001s	sys 0.010s
I20260709 13:20:00.521938 18201 consensus_queue.cc:237] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:00.497351 18325 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13.
I20260709 13:20:00.526561 13348 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:00.527194 18226 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:00.529182 18229 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:20:00.575316 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 48832us
I20260709 13:20:00.579107 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:00.579924 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:00.580652 12774 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 12: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:00.581617 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:00.582918 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:00.590982 12703 raft_consensus.cc:2410] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:20:00.541704 12705 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:20:00.542783 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:00.598726 12708 raft_consensus.cc:2376] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16. Re-sending same reply.
I20260709 13:20:00.568006 12598 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:00.569264 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:00.600769 12709 raft_consensus.cc:2393] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 19: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:00.570959 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:00.638723 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:00.639616 12703 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:20:00.648057 12776 raft_consensus.cc:3055] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:00.543371 18100 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:00.649731 12776 raft_consensus.cc:740] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:00.663414 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:00.664486 12701 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:20:00.670022 13385 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:00.677055 12707 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:20:00.684458 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:00.576426 18224 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.686889 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:00.574512 18328 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:00.698123 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:00.702430 12776 consensus_queue.cc:260] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:00.703720 12776 raft_consensus.cc:3060] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:00.645746 18305 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:00.669037 18100 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:00.681865 18179 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.711467 18179 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:00.712795 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 178268us
I20260709 13:20:00.713649 12598 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:00.720901 12747 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:00.722172 12747 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:00.724023 18325 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:00.724812 18325 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:20:00.735520 18301 log.cc:927] Time spent T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.546s	user 0.004s	sys 0.000s
I20260709 13:20:00.743333 12747 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:00.755756 12747 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:00.756743 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:00.757131 18325 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:00.757354 12705 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:20:00.757745 18325 raft_consensus.cc:697] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:00.757881 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:00.758340 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:00.759084 12705 raft_consensus.cc:2410] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:20:00.758997 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:00.758739 18325 consensus_queue.cc:237] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:00.759616 12708 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:20:00.759868 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:00.759980 18326 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:00.760468 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:00.760588 18326 raft_consensus.cc:697] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:00.760985 12708 raft_consensus.cc:2376] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15. Re-sending same reply.
I20260709 13:20:00.761742 18229 consensus_queue.cc:1059] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:00.762603 18229 consensus_queue.cc:1243] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 639ab2db265f4747bb7afb93e833f530 log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:20:00.763361 18227 raft_consensus.cc:3055] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:00.763924 18227 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:00.763302 18326 consensus_queue.cc:237] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:00.764670 18227 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:00.766937 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:00.768625 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:20:00.737288 18311 log.cc:927] Time spent T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.411s	user 0.000s	sys 0.001s
W20260709 13:20:00.738844 18302 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.412s	user 0.000s	sys 0.001s
I20260709 13:20:00.765304 18179 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:00.644805 18321 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:20:00.750823 18110 consensus_meta.cc:287] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.956s	user 0.003s	sys 0.008s
I20260709 13:20:00.692340 18224 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:00.753341 18112 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:00.769291 12708 raft_consensus.cc:2376] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16. Re-sending same reply.
W20260709 13:20:00.741330 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:00.803539 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:00.803990 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.804354 12705 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:20:00.772423 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:00.807072 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:00.776989 12597 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:20:00.780474 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:00.807698 12705 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:20:00.743259 18115 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:20:00.789525 12700 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.013s	sys 0.033s
W20260709 13:20:00.813653 12700 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.013s	sys 0.033s
I20260709 13:20:00.790926 12707 raft_consensus.cc:1240] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:20:00.793596 12627 raft_consensus.cc:3055] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:00.815896 12627 raft_consensus.cc:740] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:00.796465 13378 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:00.810531 12747 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:00.812243 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:00.815450 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:00.823773 12707 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:20:00.830183 12776 raft_consensus.cc:1275] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:20:00.830794 12776 pending_rounds.cc:85] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 11
I20260709 13:20:00.831085 12776 pending_rounds.cc:107] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:20:00.831434 12776 raft_consensus.cc:2889] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:00.802677 18227 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:00.840174 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:00.850117 12701 raft_consensus.cc:1275] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
W20260709 13:20:00.835752 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 489642us
I20260709 13:20:00.853334 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:00.854540 12747 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:00.857601 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:00.834685 18108 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:00.810773 18321 raft_consensus.cc:697] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:00.858774 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:00.860013 12747 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:00.812980 18330 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:00.801846 18110 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:00.834448 18328 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:20:00.873128 12700 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:20:00.875178 18322 raft_consensus.cc:3055] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:00.875725 18322 raft_consensus.cc:740] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:00.858469 17966 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:00.858331 18026 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:00.861897 18080 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:00.866552 18327 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:00.862304 18229 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:00.888219 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:00.870537 18108 raft_consensus.cc:697] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:00.860116 18297 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:00.906957 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:00.908667 12706 raft_consensus.cc:2393] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 19: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:00.903724 12700 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:20:00.876847 18322 consensus_queue.cc:260] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:00.911078 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:00.911725 12705 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:20:00.914054 18322 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:00.914248 12707 raft_consensus.cc:3055] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:00.914894 12707 raft_consensus.cc:740] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:00.915894 12707 consensus_queue.cc:260] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:00.917155 12707 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:00.918763 17734 consensus_queue.cc:1059] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:00.920423 18179 raft_consensus.cc:3055] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:00.921048 18179 raft_consensus.cc:740] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:00.922132 18179 consensus_queue.cc:260] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:00.924279 12672 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:00.925406 18334 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:00.926185 18321 consensus_queue.cc:237] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:00.927251 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:00.928544 18334 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:00.930168 12672 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:00.934053 18334 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:00.934659 12705 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:20:00.935663 12672 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:00.936468 18327 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:20:00.937485 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 71178.6us
I20260709 13:20:00.948472 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:20:00.941962 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 47275.6us
W20260709 13:20:00.942518 18299 log.cc:927] Time spent T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.865s	user 0.002s	sys 0.000s
I20260709 13:20:00.957180 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:00.960122 12672 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:00.961716 12702 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 15. Current term is 16. Ops: []
W20260709 13:20:00.961748 12634 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.015s	sys 0.026s
I20260709 13:20:00.962639 12747 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:00.903141 18180 consensus_queue.cc:1059] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:20:00.890782 17966 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:00.975386 18297 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:00.964964 18229 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:00.977471 18229 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:00.961315 18327 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:00.977520 18180 raft_consensus.cc:3055] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:00.980366 18180 raft_consensus.cc:740] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:20:00.966729 18343 log.cc:927] Time spent T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:20:00.976919 13376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 877) took 1521 ms. Trace:
I20260709 13:20:00.983692 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:00.970562 18154 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:00.972350 18100 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:00.973126 18110 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:00.973206 18108 consensus_queue.cc:237] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:00.974822 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:00.909528 18267 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:00.963084 12634 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.016s	sys 0.027s
I20260709 13:20:00.992405 12634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 3365) took 1019 ms. Trace:
I20260709 13:20:00.994846 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:00.995748 13381 raft_consensus.cc:2410] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:20:00.996445 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:00.977996 18112 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:00.967468 18337 log.cc:927] Time spent T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:20:00.992753 12634 rpcz_store.cc:276] 0709 13:19:59.973083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.974085 (+  1002us) service_pool.cc:224] Handling call
0709 13:20:00.008276 (+ 34191us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.008495 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:20:00.345867 (+337372us) log.cc:844] Serialized 14 byte log entry
0709 13:20:00.992078 (+646211us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:00.992129 (+    51us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:00.992171 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.992243 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:00.992351 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":276636,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":3688,"thread_start_us":968132,"threads_started":2}
I20260709 13:20:00.987149 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:00.982632 13376 rpcz_store.cc:276] 0709 13:19:59.455742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.455879 (+   137us) service_pool.cc:224] Handling call
0709 13:19:59.456255 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:59.478664 (+ 22409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:59.479326 (+   662us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:59.547638 (+ 68312us) log.cc:844] Serialized 26 byte log entry
0709 13:19:59.746861 (+199223us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:59.746905 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:59.746978 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:00.941955 (+1194977us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:20:00.976632 (+ 34677us) raft_consensus.cc:1685] finished
0709 13:20:00.976655 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:00.976855 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6052,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":18854,"spinlock_wait_cycles":839345024,"thread_start_us":198812,"threads_started":1,"wal-append.queue_time_us":419632}
I20260709 13:20:00.977952 18229 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:00.999251 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:20:01.000197 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 118470us
I20260709 13:20:00.979346 18179 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:00.976917 17966 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:00.986935 18026 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:01.002674 12672 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:00.979803 12707 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:20:00.988891 18154 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:01.006290 12598 leader_election.cc:336] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.518s (SENT)
I20260709 13:20:00.990721 18115 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:01.041066 12598 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:01.039562 12672 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:20:01.050874 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 51858.1us
I20260709 13:20:01.037863 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:20:01.053814 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 47557.2us
I20260709 13:20:01.069319 13381 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:20:01.069434 12598 leader_election.cc:336] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.607s (SENT)
I20260709 13:20:00.981278 18180 consensus_queue.cc:260] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:01.001631 18325 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:00.994865 18298 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:20:01.102291 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 216503us
I20260709 13:20:01.052491 17734 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:00.982597 18215 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:20:01.074069 12598 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:01.074283 12780 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 14. Current term is 15. Ops: [14.13-14.13]
W20260709 13:20:01.104185 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.035877 18339 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:01.062239 18297 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:01.036595 12707 pending_rounds.cc:85] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 10
I20260709 13:20:01.054639 18352 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:01.106539 12707 pending_rounds.cc:107] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:20:01.107775 12707 raft_consensus.cc:2889] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:20:01.106933 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 67534.2us
W20260709 13:20:01.107461 12598 leader_election.cc:336] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.827s (SENT)
I20260709 13:20:01.110513 12747 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:01.114179 12598 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:01.092252 18334 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:01.091208 18325 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:20:01.109344 18269 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.919s	user 0.007s	sys 0.023s
I20260709 13:20:01.113225 18224 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:20:01.146237 18338 log.cc:927] Time spent T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 13:20:01.146787 12531 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40528 (request call id 137) took 57361 ms (client timeout 15000 ms). Trace:
I20260709 13:20:01.150166 13381 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:20:01.116459 12672 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:01.109700 18201 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:01.112200 18180 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:01.147271 12531 rpcz_store.cc:269] 0709 13:19:03.635254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.635733 (+   479us) service_pool.cc:224] Handling call
0709 13:20:00.988635 (+57352902us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":797472,"reported_tablets":28}
I20260709 13:20:01.160822 18319 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:01.161724 18267 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:01.162803 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 47166us
I20260709 13:20:01.163892 18267 consensus_queue.cc:1059] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:20:01.166129 18342 log.cc:927] Time spent T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:20:01.151189 18297 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:01.166258 18267 raft_consensus.cc:3055] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:01.167040 18267 raft_consensus.cc:740] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:20:01.167436 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 60004.4us
W20260709 13:20:01.168129 12598 leader_election.cc:336] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.666s (SENT)
I20260709 13:20:01.168313 18267 consensus_queue.cc:260] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:01.149046 18115 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:01.170831 18267 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:20:01.175211 18335 log.cc:927] Time spent T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
W20260709 13:20:01.177917 18344 log.cc:927] Time spent T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:20:01.178112 13381 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:20:01.165225 12747 leader_election.cc:400] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15.
W20260709 13:20:01.179224 18304 log.cc:927] Time spent T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.891s	user 0.001s	sys 0.003s
I20260709 13:20:01.179454 12747 leader_election.cc:403] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:01.180581 12626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 1010) took 1468 ms. Trace:
I20260709 13:20:01.180924 12626 rpcz_store.cc:276] 0709 13:19:59.711866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.724234 (+ 12368us) service_pool.cc:224] Handling call
0709 13:19:59.724602 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:59.736527 (+ 11925us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:59.737967 (+  1440us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:59.749894 (+ 11927us) log.cc:844] Serialized 26 byte log entry
0709 13:20:00.130702 (+380808us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:00.130763 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.130852 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:01.178684 (+1047832us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:20:01.180306 (+  1622us) raft_consensus.cc:1685] finished
0709 13:20:01.180327 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:01.180522 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22156,"raft.queue_time_us":484,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":4676,"spinlock_wait_cycles":437159168,"thread_start_us":370528,"threads_started":1,"wal-append.queue_time_us":445487}
I20260709 13:20:01.182149 18115 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15.
I20260709 13:20:01.183429 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:01.184517 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:01.117895 18208 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:01.186311 12597 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:20:01.177281 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:01.190811 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:01.191510 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.192358 13376 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:20:01.126435 18325 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:01.195197 12747 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:01.196858 12747 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:01.198045 12747 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:01.200942 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:01.201606 18110 consensus_queue.cc:1059] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:20:01.201691 17734 consensus_queue.cc:1059] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:01.202287 18326 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:01.188699 18115 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:20:01.207388 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.204180 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:01.194372 18297 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:20:01.209233 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 207038us
I20260709 13:20:01.194339 18352 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.202656 18330 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:01.196556 18224 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:01.221199 12776 raft_consensus.cc:1240] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 19. Current term is 20. Ops: []
W20260709 13:20:01.121649 18269 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.939s	user 0.007s	sys 0.024s
I20260709 13:20:01.203091 18328 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:01.230523 13381 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:20:01.191807 18319 consensus_queue.cc:1059] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:20:01.236092 12779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 2671) took 1022 ms. Trace:
I20260709 13:20:01.238355 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:01.239151 12634 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:01.239513 12633 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:20:01.207046 18305 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:01.249305 12779 rpcz_store.cc:276] 0709 13:20:00.213883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.216917 (+  3034us) service_pool.cc:224] Handling call
0709 13:20:00.217204 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:00.326730 (+109526us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:00.326768 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:00.348922 (+ 22154us) log.cc:844] Serialized 26 byte log entry
0709 13:20:00.841411 (+492489us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:00.841449 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.841509 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:01.235895 (+394386us) raft_consensus.cc:1685] finished
0709 13:20:01.235915 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:01.236029 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98148,"raft.run_cpu_time_us":57,"raft.run_wall_time_us":58,"thread_start_us":554524,"threads_started":2,"wal-append.queue_time_us":611011}
I20260709 13:20:01.247023 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:01.252776 12776 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:01.250787 12629 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:20:01.254626 12774 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:20:01.208508 18080 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:20:01.208814 18339 consensus_queue.cc:1059] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:20:01.234753 18100 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:01.270386 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:01.236814 18269 raft_consensus.cc:697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:01.272631 18305 raft_consensus.cc:697] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:01.247193 18309 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:20:01.275979 18269 consensus_queue.cc:237] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:01.276203 18305 consensus_queue.cc:237] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:01.278031 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:01.278721 12709 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:20:01.273530 12774 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:20:01.273942 12703 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:01.274330 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:20:01.272416 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 65114.2us
I20260709 13:20:01.287217 12778 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:01.288331 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:01.270442 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:01.289058 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.288992 12633 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:20:01.292764 12779 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:01.295007 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 2097) took 1272 ms. Trace:
I20260709 13:20:01.295400 12702 rpcz_store.cc:276] 0709 13:20:00.022356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.022490 (+   134us) service_pool.cc:224] Handling call
0709 13:20:00.064820 (+ 42330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:00.963943 (+899123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.294774 (+330831us) spinlock_profiling.cc:243] Waited 897 ms on lock 0x7b54001cd9d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:20:01.294949 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2062195968}
I20260709 13:20:01.249997 18326 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:20:01.297801 12598 leader_election.cc:336] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.864s (SENT)
W20260709 13:20:01.311053 12598 leader_election.cc:336] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 2.036s (SENT)
I20260709 13:20:01.311633 12598 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:01.300488 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:01.314994 12707 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:01.303128 12776 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
W20260709 13:20:01.322453 13349 leader_election.cc:336] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 2.184s (SENT)
I20260709 13:20:01.304497 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:01.324966 12633 raft_consensus.cc:1240] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:01.328769 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:20:01.300060 18315 log.cc:927] Time spent T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.936s	user 0.003s	sys 0.000s
W20260709 13:20:01.299006 18347 log.cc:927] Time spent T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:20:01.348565 13378 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:20:01.332185 18312 log.cc:927] Time spent T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.854s	user 0.004s	sys 0.000s
I20260709 13:20:01.339671 13376 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:01.242632 18352 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:01.308740 18215 raft_consensus.cc:3055] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:20:01.351845 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 109353us
I20260709 13:20:01.297470 18026 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:20:01.348129 12598 leader_election.cc:336] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 2.094s (SENT)
I20260709 13:20:01.256116 18321 consensus_queue.cc:1048] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.073s
I20260709 13:20:01.325224 18179 raft_consensus.cc:3055] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:01.363727 12672 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:01.368849 12779 raft_consensus.cc:1275] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:20:01.355736 12598 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:01.358107 12707 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
W20260709 13:20:01.352541 18349 log.cc:927] Time spent T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:20:01.346354 18108 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:01.372838 12672 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:01.373471 18179 raft_consensus.cc:740] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:01.380122 18179 consensus_queue.cc:260] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:01.381742 18179 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:01.387715 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:01.354051 18215 raft_consensus.cc:740] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:01.355046 18361 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:20:01.400067 12598 leader_election.cc:336] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.766s (SENT)
I20260709 13:20:01.400730 12598 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:01.358546 18100 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:01.349398 18355 consensus_queue.cc:1059] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:20:01.383697 12627 consensus_queue.cc:260] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:01.405269 12627 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:01.406206 12778 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:20:01.358582 18129 consensus_queue.cc:1059] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 13:20:01.414587 12774 raft_consensus.cc:1240] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:20:01.355183 18297 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:20:01.387136 18322 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:20:01.417330 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:01.420346 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 115921us
I20260709 13:20:01.388232 18327 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:01.372092 18359 raft_consensus.cc:2764] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
W20260709 13:20:01.422714 13349 leader_election.cc:336] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.828s (SENT)
I20260709 13:20:01.422359 12598 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:01.422513 12632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 1014) took 1523 ms. Trace:
I20260709 13:20:01.387777 18310 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:01.423267 13349 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:01.426448 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:01.427412 13376 raft_consensus.cc:2368] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16.
I20260709 13:20:01.423074 12631 raft_consensus.cc:1240] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:20:01.431739 12703 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:20:01.432796 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:01.402205 18100 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:01.377182 18080 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:20:01.377700 18227 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:20:01.423426 12632 rpcz_store.cc:276] 0709 13:19:59.899190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:59.903263 (+  4073us) service_pool.cc:224] Handling call
0709 13:19:59.903599 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:59.920689 (+ 17090us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:59.920720 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:59.921232 (+   512us) log.cc:844] Serialized 26 byte log entry
0709 13:20:00.138375 (+217143us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:00.138411 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:00.274607 (+136196us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b54002db9d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:20:00.274674 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:01.242125 (+967451us) spinlock_profiling.cc:243] Waited 5.6 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:20:01.422293 (+180168us) raft_consensus.cc:1685] finished
0709 13:20:01.422314 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:01.422457 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":40257,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":25164,"spinlock_wait_cycles":51307136,"thread_start_us":216755,"threads_started":1,"wal-append.queue_time_us":398752}
I20260709 13:20:01.437031 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:01.448972 12778 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:20:01.453366 12598 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:01.454556 12598 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:01.403605 18154 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:01.419345 18362 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:01.436434 17947 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:01.444255 18129 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:01.445374 18322 raft_consensus.cc:697] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:01.450306 18100 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:01.471000 12627 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
W20260709 13:20:01.464450 18303 log.cc:927] Time spent T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 1.320s	user 0.007s	sys 0.000s
I20260709 13:20:01.405741 18361 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:01.477753 12627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56838 (request call id 1988) took 1328 ms (client timeout 1651 ms). Trace:
I20260709 13:20:01.443535 18269 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
W20260709 13:20:01.479691 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 101787us
I20260709 13:20:01.462798 18310 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:01.450676 18201 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:20:01.469012 18362 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:01.483958 12627 rpcz_store.cc:269] 0709 13:20:00.149261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.156012 (+  6751us) service_pool.cc:224] Handling call
0709 13:20:00.793409 (+637397us) spinlock_profiling.cc:243] Waited 47 ms on lock 0x7b54002731a0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d329dc0 00007f4c6d3b5e18 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:20:01.383345 (+589936us) spinlock_profiling.cc:243] Waited 21.2 ms on lock 0x7b54002731a0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d33006f 00007f4c6d3a8f60 00007f4c6d3aa7de 00007f4c6d3b627c 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2
0709 13:20:01.477689 (+ 94344us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156883584}
I20260709 13:20:01.444923 18229 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:01.482811 18256 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 13:20:01.484283 18129 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:01.451536 18215 consensus_queue.cc:260] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:01.526368 18215 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:01.494530 18336 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.495173 18362 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:01.519595 12747 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:01.519735 18352 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:01.519869 18229 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:01.481642 18376 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:20:01.510685 18310 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:20:01.515751 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 61253.1us
W20260709 13:20:01.558009 18300 log.cc:927] Time spent T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 1.272s	user 0.000s	sys 0.006s
I20260709 13:20:01.494755 18363 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.502039 18339 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:20:01.563206 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 73556.5us
I20260709 13:20:01.565208 12598 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:01.566073 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:20:01.563109 18308 log.cc:927] Time spent T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 1.240s	user 0.000s	sys 0.006s
I20260709 13:20:01.561780 18129 raft_consensus.cc:697] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:01.598117 18129 consensus_queue.cc:237] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:01.598627 18154 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:01.562065 18376 consensus_queue.cc:1243] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:20:01.559796 18326 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:01.604481 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:01.619407 12672 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:01.571774 18229 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:01.555238 18336 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:01.620678 12672 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:01.597010 12747 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:01.610797 18334 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.615242 18377 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:01.610611 18326 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:20:01.627731 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 46779.1us
I20260709 13:20:01.568444 18339 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:01.625790 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 195264us
I20260709 13:20:01.566373 18322 consensus_queue.cc:237] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:01.632463 12747 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:01.566992 18363 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:01.636682 18325 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:01.622465 18179 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:01.637214 18382 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:01.621443 18229 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:01.641026 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.659878 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:01.662997 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:01.665711 12703 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:20:01.625104 18310 raft_consensus.cc:697] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:01.651707 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:01.633126 18326 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:01.631160 18352 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:01.666597 18229 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:01.675185 18269 consensus_queue.cc:1059] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:20:01.677541 18269 raft_consensus.cc:3055] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:01.678045 18269 raft_consensus.cc:740] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:01.679193 18269 consensus_queue.cc:260] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:01.680879 18269 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:01.666290 18361 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:20:01.686165 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 121038us
I20260709 13:20:01.669327 18267 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:01.690949 12598 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:01.650918 18382 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:01.688167 18352 raft_consensus.cc:697] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:01.630926 18334 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:01.693612 18267 raft_consensus.cc:697] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:01.705986 18379 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:20:01.706952 18368 log.cc:927] Time spent T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 13:20:01.708039 18380 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.086s
I20260709 13:20:01.710518 12631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 1986) took 1586 ms. Trace:
I20260709 13:20:01.710979 12631 rpcz_store.cc:276] 0709 13:20:00.124348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.125687 (+  1339us) service_pool.cc:224] Handling call
0709 13:20:01.422830 (+1297143us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:01.430260 (+  7430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.710282 (+280022us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54002db9d0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:20:01.710457 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2982214528}
I20260709 13:20:01.714617 18310 consensus_queue.cc:237] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:01.722079 18374 log.cc:927] Time spent T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:20:01.722586 18339 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:01.723850 18336 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:01.723415 18385 consensus_queue.cc:1059] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:20:01.724658 18383 consensus_queue.cc:1059] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:20:01.725970 18383 raft_consensus.cc:3055] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:01.726302 18334 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:01.726367 18383 raft_consensus.cc:740] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:20:01.736917 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.698670 18382 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:01.694765 18112 consensus_queue.cc:1048] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
W20260709 13:20:01.782711 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 215036us
I20260709 13:20:01.728048 18363 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:01.787426 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 136422us
I20260709 13:20:01.788879 12672 leader_election.cc:400] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16.
I20260709 13:20:01.789366 12672 leader_election.cc:403] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:01.787652 12597 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:01.792440 12704 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:20:01.773023 18383 consensus_queue.cc:260] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:01.795842 18383 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:01.791282 18298 consensus_queue.cc:1048] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:20:01.794606 18387 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:20:01.809691 18352 consensus_queue.cc:237] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:01.813740 18373 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:01.814352 18373 raft_consensus.cc:697] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:01.815284 18373 consensus_queue.cc:237] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:01.819526 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 115215us
I20260709 13:20:01.824177 18394 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:01.825016 18297 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:01.827090 12598 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:01.829879 13376 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:01.819739 18298 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:01.849184 18298 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:01.851639 18298 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:01.852214 18326 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:01.856683 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:01.861294 12774 raft_consensus.cc:2393] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 16: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:01.822906 17987 raft_consensus.cc:3055] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:01.820510 18383 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:01.853981 18387 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:20:01.853724 18394 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:01.870985 17987 raft_consensus.cc:740] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:01.878804 17987 consensus_queue.cc:260] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:01.879944 17987 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:01.887768 18384 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:01.860868 18100 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:01.891247 18100 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:01.904501 13376 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:20:01.905522 18393 log.cc:927] Time spent T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:20:01.901232 18382 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:01.917092 18382 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:01.918920 18357 raft_consensus.cc:3055] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:01.919515 18357 raft_consensus.cc:740] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:01.918619 18362 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:01.920178 18326 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:01.923035 18362 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:01.924965 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:01.930382 13385 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:20:01.867071 18298 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:20:01.883745 18325 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:01.885128 18394 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:01.869505 18267 consensus_queue.cc:237] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:01.927037 18388 log.cc:927] Time spent T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
W20260709 13:20:01.935009 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:01.943220 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:20:01.950412 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 281264us
I20260709 13:20:01.952744 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:01.955504 12627 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:20:01.950044 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:01.954145 12778 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:20:01.953089 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:01.951233 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:01.962930 18229 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:01.965157 18229 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:01.958439 12629 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:20:01.952409 12673 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:01.968808 12775 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:20:01.969439 12629 pending_rounds.cc:85] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 10
I20260709 13:20:01.969856 12629 pending_rounds.cc:107] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:20:01.970162 12629 raft_consensus.cc:2889] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:01.988133 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:01.988981 12626 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:01.882826 18383 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:01.995451 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 206764us
I20260709 13:20:01.998279 12700 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:02.000098 12703 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:20:02.007409 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:02.009264 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:02.010238 12706 raft_consensus.cc:2410] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:20:02.010157 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:02.010794 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:02.011601 12705 raft_consensus.cc:2376] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 20. Re-sending same reply.
I20260709 13:20:02.013906 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:02.014793 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:02.016271 12703 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:02.016707 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:02.017431 12706 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:01.978063 18180 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:02.020524 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:01.985836 18326 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.027197 12700 raft_consensus.cc:1240] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:02.028131 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:02.028287 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:02.038513 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:20:02.039038 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 183143us
I20260709 13:20:02.040086 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:02.040961 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:01.978634 18226 consensus_queue.cc:1059] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:20:02.041538 12705 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:02.044322 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:01.996239 18328 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:01.990015 18229 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:20:01.987294 18154 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:20:02.057973 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:02.058801 12709 raft_consensus.cc:2376] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16. Re-sending same reply.
I20260709 13:20:02.060966 12703 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
W20260709 13:20:02.046029 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 93710.8us
I20260709 13:20:02.046327 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:01.986960 18373 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:20:02.062949 12707 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:02.018952 18363 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16.
I20260709 13:20:02.045979 12700 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:02.064014 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:02.146597 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:02.144042 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:02.147933 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:02.148614 12705 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:20:02.148626 12709 raft_consensus.cc:2376] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12. Re-sending same reply.
I20260709 13:20:02.151573 12706 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:20:02.156740 12626 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
I20260709 13:20:02.163770 12703 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:20:02.165130 12703 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:20:02.166391 12703 raft_consensus.cc:1275] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:20:02.171483 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:20:02.171757 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 109085us
I20260709 13:20:02.042991 18326 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:02.180419 12747 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:02.182022 12707 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:20:02.065163 17966 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:02.064642 18392 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:02.196942 12700 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:20:02.183873 12673 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:20:02.236845 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 53041.8us
W20260709 13:20:02.243162 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.060840 18328 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:02.236685 18392 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:02.234050 18227 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:20:02.364089 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:02.363966 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 126238us
I20260709 13:20:02.240833 18180 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:02.320292 18154 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:02.241664 17966 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:02.341917 18394 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:02.354769 18383 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:20:02.357856 18402 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
W20260709 13:20:02.346995 18400 log.cc:927] Time spent T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.302s	user 0.000s	sys 0.001s
I20260709 13:20:02.170141 18321 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:02.365012 12673 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:02.345968 18370 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:02.388618 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 148944us
I20260709 13:20:02.386590 18154 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:02.399925 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:20:02.401118 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.367532 18180 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:02.366454 18392 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:02.401782 13385 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:02.410902 18322 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:02.411342 12775 raft_consensus.cc:2393] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 17: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:02.416571 12781 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:20:02.418501 12777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 2681) took 1223 ms. Trace:
I20260709 13:20:02.389111 18340 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:02.455829 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:20:02.420125 18390 log.cc:927] Time spent T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.355s	user 0.000s	sys 0.001s
I20260709 13:20:02.398080 18321 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:02.427910 17966 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:02.394176 18394 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:02.385919 18227 raft_consensus.cc:697] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:02.430442 18154 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:02.460973 13384 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 13:20:02.488277 13384 pending_rounds.cc:85] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd: Aborting all ops after (but not including) 12
I20260709 13:20:02.488586 13384 pending_rounds.cc:107] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 13:20:02.488874 13384 raft_consensus.cc:2889] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:02.454619 12777 rpcz_store.cc:276] 0709 13:20:01.195025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.197830 (+  2805us) service_pool.cc:224] Handling call
0709 13:20:01.202818 (+  4988us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:01.869318 (+666500us) spinlock_profiling.cc:243] Waited 3.43 ms on lock 0x7b18003abdc0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6cf5a4bc 00007f4c6a2a848b 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c2455 00007f4c6d3aa060 00007f4c6d3aabfb 00007f4c6d3b564b
0709 13:20:01.869452 (+   134us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:01.869477 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:01.870061 (+   584us) log.cc:844] Serialized 26 byte log entry
0709 13:20:01.996035 (+125974us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:01.996092 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:01.996342 (+   250us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:02.418211 (+421869us) raft_consensus.cc:1685] finished
0709 13:20:02.418233 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:02.418443 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5997,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":9667,"spinlock_wait_cycles":7881984,"thread_start_us":137585,"threads_started":2,"wal-append.queue_time_us":178542}
I20260709 13:20:02.469142 12775 raft_consensus.cc:2393] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 15: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:02.472131 18316 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.480062 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:02.480448 12781 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:02.481421 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:02.482592 12625 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:20:02.494863 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:02.435276 18180 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:02.486088 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:02.498517 12628 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:02.482677 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:02.486796 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:02.491601 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:02.452872 18383 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.455940 18322 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:02.501260 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:02.499818 12630 raft_consensus.cc:2376] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20. Re-sending same reply.
I20260709 13:20:02.504061 12777 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:02.478085 18385 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.506683 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:02.510852 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:02.402958 18370 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:02.510896 17966 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 13:20:02.514935 18385 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:02.522480 18370 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:02.522915 18385 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:02.523710 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.524909 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:02.533023 18328 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:20:02.543079 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.516177 18383 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:02.546867 12632 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:02.548503 18383 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:02.516513 18267 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:20:02.484225 18340 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:02.552402 18340 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:02.555514 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:02.556167 12632 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:20:02.557433 12673 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:02.562013 12632 raft_consensus.cc:1240] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:02.563490 13380 raft_consensus.cc:3055] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:02.563925 13380 raft_consensus.cc:740] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:02.565550 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:02.566720 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:02.505344 18412 raft_consensus.cc:476] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Not starting pre-election -- already a leader
W20260709 13:20:02.570380 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 50032.1us
I20260709 13:20:02.513687 18322 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:02.564870 13380 consensus_queue.cc:260] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:02.574936 13349 leader_election.cc:336] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 2.110s (SENT)
I20260709 13:20:02.575136 13380 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:02.563055 18414 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:20:02.567889 18340 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:02.554772 18229 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:20:02.567910 18404 log.cc:927] Time spent T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:20:02.493818 18394 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:02.586428 18226 consensus_queue.cc:1059] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:02.560461 18326 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:02.497179 18316 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:02.583776 18340 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:20:02.593406 13349 leader_election.cc:336] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 2.201s (SENT)
I20260709 13:20:02.593868 13349 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:02.590022 18227 consensus_queue.cc:237] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:02.595579 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:02.603008 12632 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 19: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
W20260709 13:20:02.590678 18407 log.cc:927] Time spent T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:20:02.588590 18309 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:02.611285 18377 raft_consensus.cc:3055] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:02.595973 18340 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:02.579067 18370 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:20:02.613252 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:02.599913 18394 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:02.615252 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:02.613340 18377 raft_consensus.cc:740] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:02.643124 18392 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:02.614120 18309 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:02.644424 18180 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:02.669689 18180 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:02.685197 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:02.686046 12629 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:02.687687 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:02.688452 12632 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:02.693773 17947 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:02.699182 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:02.703478 12746 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:02.704303 18229 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:02.704699 18229 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:02.705062 18229 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:02.707091 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:02.707252 18080 consensus_queue.cc:1059] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:02.707793 12778 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 18.
I20260709 13:20:02.708280 18080 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:02.710000 18224 raft_consensus.cc:3055] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:02.710361 18224 raft_consensus.cc:740] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:02.725163 18224 consensus_queue.cc:260] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:02.726395 18224 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:02.726544 18229 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:02.728718 18229 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:02.754684 12632 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
W20260709 13:20:02.763388 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 57817.8us
I20260709 13:20:02.759939 12673 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:02.677809 18392 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:02.783476 13380 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:20:02.783527 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.784773 18180 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:20:02.786456 18410 log.cc:927] Time spent T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:20:02.785341 18229 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:20:02.798547 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 205152us
W20260709 13:20:02.801472 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.804445 12628 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:20:02.807006 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:02.807986 12632 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:20:02.790491 18100 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:02.816999 12673 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:20:02.817052 12597 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.68:36021 timed out after 1.759s (SENT) [suppressed 54 similar messages]
I20260709 13:20:02.813030 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:02.819612 12628 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:20:02.823846 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:02.792090 18226 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:02.827754 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:02.828373 12779 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:02.776655 18377 consensus_queue.cc:260] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:02.810029 18291 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.799160 18392 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.810791 18180 raft_consensus.cc:697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:02.825793 18339 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:02.847028 12779 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
I20260709 13:20:02.848664 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:02.849917 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:02.811192 18229 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:02.829793 18179 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:02.851749 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:02.873139 12779 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:20:02.874444 12782 raft_consensus.cc:1240] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:20:02.842118 18291 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:02.853323 18411 raft_consensus.cc:3055] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:02.878418 18411 raft_consensus.cc:740] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:02.853148 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:02.826561 18100 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:02.855870 18357 consensus_queue.cc:260] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:02.879638 18291 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:02.880085 18357 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:02.880709 18100 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:02.833253 18226 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:02.882962 18226 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:20:02.883448 18226 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:02.860366 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:02.850359 18377 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:02.884238 12775 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:02.886915 18180 consensus_queue.cc:237] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:02.842980 18392 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:02.889323 18392 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:02.863176 18336 raft_consensus.cc:3055] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:02.889783 18336 raft_consensus.cc:740] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:02.890556 18336 consensus_queue.cc:260] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:02.891734 18336 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:02.866127 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:02.833500 18325 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.894133 12780 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:20:02.894176 18325 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:02.894897 18229 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:02.896338 18325 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:02.835659 18316 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:02.835839 18380 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:02.833595 18321 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:02.887629 18421 log.cc:927] Time spent T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
I20260709 13:20:02.896459 18322 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:02.865901 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:02.913344 12775 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:20:02.915087 12775 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:20:02.916482 12775 raft_consensus.cc:1240] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16. Ops: []
W20260709 13:20:02.925948 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 55650.7us
I20260709 13:20:02.926352 12776 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:02.904659 17947 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:02.847506 18339 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:02.906549 12629 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:20:02.908851 12779 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:02.911284 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:02.936635 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:02.929179 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:20:02.930965 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 113956us
W20260709 13:20:02.938158 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:02.893299 18340 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:02.903918 18411 consensus_queue.cc:260] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:02.965865 18340 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:02.922235 18325 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:02.896339 18179 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:02.924839 18330 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:03.007136 13385 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:02.976186 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:03.014865 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:02.938807 18100 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:02.940114 18339 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:03.045149 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:02.960309 17966 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:02.911952 18380 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:03.078392 12629 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
W20260709 13:20:02.976599 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 123541us
W20260709 13:20:03.029057 18406 log.cc:927] Time spent T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:20:03.022786 17734 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:03.086611 17734 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:03.087705 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:03.053750 18100 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:03.088232 18224 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:03.044617 18330 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:03.089844 18229 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.090454 18229 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:03.033906 18316 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:03.091929 18316 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:03.092855 18229 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:20:03.094357 18430 log.cc:927] Time spent T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:20:03.024776 18383 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.052018 18411 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:03.088903 18224 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:02.983352 12673 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:03.057166 18394 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:03.121121 13349 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:03.122509 18226 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:03.117087 18425 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:03.118691 18383 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:03.126335 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 143088us
W20260709 13:20:03.117336 18440 log.cc:927] Time spent T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:20:03.135937 18422 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.411s	user 0.004s	sys 0.000s
I20260709 13:20:03.144193 18322 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:03.144548 18224 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:03.023353 18325 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:03.159655 13384 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:20:03.175870 13384 pending_rounds.cc:85] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd: Aborting all ops after (but not including) 11
I20260709 13:20:03.176240 13384 pending_rounds.cc:107] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:20:03.176568 13384 raft_consensus.cc:2889] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:03.160548 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:03.161491 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:03.131191 18385 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:03.052731 18378 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:20:03.167714 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:03.005126 18180 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:03.178836 13376 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:20:03.180065 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:20:03.180095 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 174603us
I20260709 13:20:03.183816 13383 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:03.167402 18387 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.165940 18442 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:20:03.080811 17966 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:03.183670 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 63042.7us
I20260709 13:20:03.191839 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:03.195248 13378 raft_consensus.cc:1240] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:20:03.080502 18413 consensus_queue.cc:1059] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.118s
W20260709 13:20:03.198360 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 169631us
I20260709 13:20:03.057482 18129 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.126s
I20260709 13:20:03.166880 18394 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:03.203373 18426 log.cc:927] Time spent T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
I20260709 13:20:03.205221 18383 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:03.192878 18387 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:03.186126 18385 raft_consensus.cc:697] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:03.210839 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:03.220527 13376 raft_consensus.cc:2393] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 16: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:20:03.220652 18385 consensus_queue.cc:237] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:03.221815 13380 raft_consensus.cc:2410] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 17 because replica has last-logged OpId of term: 16 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 13:20:03.225026 18380 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:03.211520 12776 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:20:03.193495 18180 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:03.227455 18180 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:03.219555 18325 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:03.227689 17966 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:03.141057 12525 catalog_manager.cc:5697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 10 to 15, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 15 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:03.231086 18394 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:03.233304 18387 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:03.235698 12776 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:03.209424 18413 consensus_queue.cc:1243] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer 3d539f8d95d343d8b1696803f18b30d2 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:20:03.240953 13383 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
I20260709 13:20:03.242336 13377 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:20:03.243285 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:20:03.243775 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:03.243923 13378 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:03.246735 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:03.252575 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:03.256487 18309 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:03.257912 13378 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:20:03.259840 18339 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:03.261159 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:03.258071 18419 log.cc:927] Time spent T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.640s	user 0.001s	sys 0.000s
I20260709 13:20:03.229457 12525 catalog_manager.cc:5697] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 12 to 13, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 13 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:03.264487 12525 catalog_manager.cc:5697] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:03.265731 18339 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:03.271091 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 75469.5us
I20260709 13:20:03.272540 13385 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:20:03.272857 12672 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:03.272248 18080 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:20:03.285386 12672 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:03.287086 12706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 780) took 1273 ms. Trace:
I20260709 13:20:03.288321 12597 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:03.287472 12706 rpcz_store.cc:276] 0709 13:20:02.013650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.152537 (+138887us) service_pool.cc:224] Handling call
0709 13:20:02.152931 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:02.171894 (+ 18963us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:02.171927 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:02.228608 (+ 56681us) log.cc:844] Serialized 26 byte log entry
0709 13:20:02.586947 (+358339us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:20:03.281915 (+694968us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b18002235a0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6cf5a4bc 00007f4c6a2a848b 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c2455 00007f4c70daa3c7 00007f4c6d3bc9a7 00007f4c6d3cc0bf
0709 13:20:03.282120 (+   205us) log.cc:844] Serialized 14 byte log entry
0709 13:20:03.285027 (+  2907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.285132 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:03.285179 (+    47us) raft_consensus.cc:1685] finished
0709 13:20:03.286911 (+  1732us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:03.287031 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21174,"prepare.queue_time_us":198506,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":19344,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":45682,"spinlock_wait_cycles":5023488,"thread_start_us":467188,"threads_started":2,"wal-append.queue_time_us":324486}
I20260709 13:20:03.291549 12672 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:03.287665 18339 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:03.289826 12525 catalog_manager.cc:5697] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 5 to 10, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 10 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:03.295665 18385 consensus_queue.cc:1059] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:03.296363 18357 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:03.294020 18336 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:03.297825 17987 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:03.298218 18336 raft_consensus.cc:697] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:03.298491 18322 raft_consensus.cc:3055] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:03.298907 18322 raft_consensus.cc:740] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:03.296924 12525 catalog_manager.cc:5697] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 9 to 16. New cstate: current_term: 16 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:03.299345 18336 consensus_queue.cc:237] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:03.299751 18322 consensus_queue.cc:260] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:03.294166 18180 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:03.301961 18180 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:03.302328 18180 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:03.303506 18100 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:03.304719 12779 raft_consensus.cc:1240] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:03.304636 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:03.305951 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:03.305886 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:03.306092 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:03.310859 12775 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:03.309875 12777 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:20:03.320523 12702 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:20:03.306644 12779 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:03.323105 18179 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:03.323871 12774 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 761) took 1367 ms. Trace:
I20260709 13:20:03.324112 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:03.324669 18316 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:03.307459 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:03.324288 12774 rpcz_store.cc:276] 0709 13:20:01.956303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:01.975163 (+ 18860us) service_pool.cc:224] Handling call
0709 13:20:01.975493 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:01.975919 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:01.975958 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:01.976494 (+   536us) log.cc:844] Serialized 26 byte log entry
0709 13:20:03.260597 (+1284103us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b18003b8ba0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf520d1 00007f4c6d36da0f
0709 13:20:03.290188 (+ 29591us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:03.290244 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.290321 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:03.323555 (+ 33234us) raft_consensus.cc:1685] finished
0709 13:20:03.323576 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:03.323807 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14900,"raft.queue_time_us":3150,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":16321,"spinlock_wait_cycles":5221248,"thread_start_us":23833,"threads_started":1,"wal-append.queue_time_us":1335471}
I20260709 13:20:03.340309 18363 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:03.341027 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 48906.3us
I20260709 13:20:03.349366 12672 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:03.351886 18112 consensus_queue.cc:1059] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:20:03.353845 18112 consensus_queue.cc:1243] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Peer 3d539f8d95d343d8b1696803f18b30d2 log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:20:03.354324 18305 raft_consensus.cc:3055] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:03.354702 18305 raft_consensus.cc:740] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:03.355523 17734 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:03.355463 18305 consensus_queue.cc:260] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:03.356173 18453 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:03.356799 18201 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:20:03.358317 18371 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:03.356813 18305 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:03.351490 18180 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:03.363050 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:03.300794 12525 catalog_manager.cc:5697] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:03.369491 12781 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:20:03.371178 12781 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:20:03.350850 18357 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:03.365933 12775 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:20:03.343546 18385 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:03.365756 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:03.373107 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:03.378521 12776 raft_consensus.cc:2393] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 18: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:03.345732 18339 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.382468 12627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:03.382297 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:03.383985 12627 raft_consensus.cc:606] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:03.362884 18352 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.374890 18100 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:03.382031 18371 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:03.394687 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:03.389451 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:03.347133 18363 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:03.398355 18100 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:03.352524 18405 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:20:03.383244 18385 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:03.394338 18325 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.402070 18180 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:03.402776 18385 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:03.369683 12779 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:20:03.375259 17734 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:03.406198 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:03.402062 18325 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:03.408744 12708 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:03.394027 12525 catalog_manager.cc:5697] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:03.409986 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:03.411254 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:03.412632 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:03.413285 12707 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 18.
I20260709 13:20:03.409332 18325 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:03.414391 18363 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:03.396999 18371 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:03.416973 18371 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:03.417793 18325 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.418394 18325 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:03.418874 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
W20260709 13:20:03.417809 12598 leader_election.cc:336] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.512s (SENT)
I20260709 13:20:03.423034 18370 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:03.423141 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:03.423820 18370 raft_consensus.cc:697] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:03.424887 12704 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:20:03.425177 18371 consensus_queue.cc:1059] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:03.425899 18371 consensus_queue.cc:1243] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer 3d539f8d95d343d8b1696803f18b30d2 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:20:03.426841 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:03.427487 12703 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:20:03.426337 18309 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:03.428426 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:03.429031 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:03.430354 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:03.433614 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:03.432374 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:03.434268 18325 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:20:03.437333 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:03.436605 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:03.437795 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:03.439288 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:03.437822 12703 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:20:03.424851 18370 consensus_queue.cc:237] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:03.441939 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:20:03.449296 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:03.441920 12598 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:03.449708 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:03.451572 12781 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:03.450515 12778 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:20:03.445080 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:03.451761 12776 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:20:03.457463 18363 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:20:03.447026 18443 log.cc:927] Time spent T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.298s	user 0.003s	sys 0.000s
I20260709 13:20:03.395442 18339 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:03.459192 18316 consensus_queue.cc:1059] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:03.460297 18316 raft_consensus.cc:3055] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:03.460456 18339 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:03.460659 18316 raft_consensus.cc:740] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:03.460697 12704 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:20:03.462075 18256 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:03.462205 12598 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:03.463258 18371 raft_consensus.cc:2764] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:20:03.463781 12701 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 13:20:03.465871 12598 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:03.405478 18352 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:03.381975 18357 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:03.491088 12781 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
W20260709 13:20:03.555806 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 84229.5us
W20260709 13:20:03.555166 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 55847.4us
I20260709 13:20:03.560286 12672 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:03.561431 12672 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:03.628463 18439 raft_consensus.cc:2804] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:03.446805 12525 catalog_manager.cc:5697] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 7 to 12, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 12 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:03.630553 12525 catalog_manager.cc:5697] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:03.638206 12525 catalog_manager.cc:5697] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 14 to 17, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 17 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:03.645687 18309 consensus_queue.cc:1048] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:03.647519 12525 catalog_manager.cc:5697] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:03.649503 12525 catalog_manager.cc:5697] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:03.653735 18371 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:03.662763 18371 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:03.555366 18454 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:03.666358 12708 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:03.666014 18454 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:03.668802 18454 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:03.665473 18371 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:03.670281 12706 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
W20260709 13:20:03.670559 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 111724us
I20260709 13:20:03.672559 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:03.673062 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:03.673676 18405 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:03.673835 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:03.674230 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:03.674445 18405 raft_consensus.cc:697] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:03.674800 12633 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:20:03.675678 18405 consensus_queue.cc:237] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:03.676599 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:03.677752 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 114005us
I20260709 13:20:03.677961 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:03.678645 12672 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:03.678789 12634 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:20:03.680084 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:03.680162 12672 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:03.557626 18305 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:03.656981 18357 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:03.557083 18316 consensus_queue.cc:260] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:03.660360 18352 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:03.661777 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 2127) took 1605 ms. Trace:
I20260709 13:20:03.769068 12702 rpcz_store.cc:276] 0709 13:20:02.056270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:02.179084 (+122814us) service_pool.cc:224] Handling call
0709 13:20:02.232587 (+ 53503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:03.320950 (+1088363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:03.661427 (+340477us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54001b1554. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:20:03.661698 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2501270784}
I20260709 13:20:03.709611 12626 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:20:03.710824 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:03.712464 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:03.771281 12630 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:20:03.771431 12625 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
W20260709 13:20:03.733335 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:03.744115 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:03.744465 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:03.772754 12780 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:20:03.773339 12628 raft_consensus.cc:2410] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:20:03.744180 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:03.774758 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:03.779047 12708 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:03.776154 18180 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:03.788555 18336 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:03.786664 18439 raft_consensus.cc:697] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:03.768491 18385 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:20:03.796051 18316 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:03.797287 18180 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:03.800107 18305 raft_consensus.cc:697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:03.811198 12598 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:03.804044 18336 raft_consensus.cc:697] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:03.808385 18180 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:03.815868 18436 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:03.805495 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 226600us
I20260709 13:20:03.822561 18305 consensus_queue.cc:237] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:03.819077 18362 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:03.821116 18436 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:03.832367 18362 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:03.835417 18436 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:03.838480 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:03.840178 12747 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:03.819257 18439 consensus_queue.cc:237] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:03.844208 18352 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:03.831130 18336 consensus_queue.cc:237] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:03.841477 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:03.860175 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:03.861130 18454 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:03.861639 18454 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:03.862079 18454 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:03.872516 18454 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:03.873421 18362 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:03.812840 12525 catalog_manager.cc:5697] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 11 to 14, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 14 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:03.881047 12525 catalog_manager.cc:5697] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 7 to 10, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 10 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:03.910719 12525 catalog_manager.cc:5697] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
W20260709 13:20:03.944242 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:03.903208 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:03.890388 18454 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:03.959811 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:03.961485 12780 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:20:03.964068 12780 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:20:03.949745 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 138722us
I20260709 13:20:03.967752 12598 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:03.969063 12598 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:03.951467 18305 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:03.970829 18371 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:03.984249 18201 consensus_queue.cc:1059] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:03.975764 18373 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:03.987483 18373 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:03.987978 18373 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:03.991220 18401 raft_consensus.cc:3055] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:03.991772 18401 raft_consensus.cc:740] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:03.999279 18330 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:03.971355 18305 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:04.018868 18439 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:04.019629 18439 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:04.022475 18100 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:04.025013 18330 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:04.025671 18401 consensus_queue.cc:260] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:04.026865 18401 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:04.014462 18305 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:04.052238 18080 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:04.053064 18080 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:04.055903 18080 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:04.056933 18180 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:04.059142 18439 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:03.913758 12525 catalog_manager.cc:5697] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 5 to 8, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 8 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:04.062466 18201 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:04.063205 18201 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:04.063851 12525 catalog_manager.cc:5697] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 5 to 13. New cstate: current_term: 13 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:04.066071 18201 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:04.074469 12672 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:04.076387 18357 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.076969 18357 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:04.077358 12709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:20:04.078063 12709 raft_consensus.cc:606] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:04.078474 12709 raft_consensus.cc:614] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:04.079270 18357 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:04.079473 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:04.079938 18357 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:04.080381 18357 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:04.080762 18357 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:04.080945 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:04.081753 12704 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:04.082044 18385 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:04.082443 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:04.082782 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:04.082604 18385 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:04.083186 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:04.083575 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:04.083958 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:04.084627 18201 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.085086 12709 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:20:04.079312 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:04.082700 17734 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.086382 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:04.086793 17734 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:04.087592 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:04.088703 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:04.089301 12709 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:20:04.085157 18201 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:04.089533 17734 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:04.095429 12704 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:04.006160 18454 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.094452 18201 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:04.090969 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:04.090598 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:04.102305 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:04.103086 12708 raft_consensus.cc:2376] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16. Re-sending same reply.
I20260709 13:20:04.103983 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:04.098341 18454 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:04.105728 18439 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:20:04.102389 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 95202.3us
I20260709 13:20:04.107936 18297 consensus_queue.cc:1059] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:04.108880 18297 raft_consensus.cc:3055] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:04.109306 18297 raft_consensus.cc:740] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:04.109959 18454 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:04.089311 12702 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:20:04.110198 18297 consensus_queue.cc:260] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:04.111148 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:04.111522 18297 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:04.112479 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:04.113072 12702 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
W20260709 13:20:04.113821 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 785445us
I20260709 13:20:04.104009 17734 consensus_queue.cc:1059] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
I20260709 13:20:04.116931 18392 raft_consensus.cc:3055] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:04.111186 18322 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:20:04.105738 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 797376us
I20260709 13:20:04.120467 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:20:04.121534 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:04.114368 12709 raft_consensus.cc:2376] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19. Re-sending same reply.
I20260709 13:20:04.127991 12704 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:20:04.133477 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:04.134230 13378 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:20:04.137104 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:04.134271 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:04.131249 12747 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:04.141744 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:04.144060 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:04.146989 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:20:04.147598 12703 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.048s	sys 0.067s
I20260709 13:20:04.147809 12774 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
W20260709 13:20:04.148018 12703 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.048s	sys 0.067s
I20260709 13:20:04.147432 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:04.148559 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:04.149176 12702 raft_consensus.cc:2393] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 14: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:04.146836 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:04.148104 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:04.146888 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:04.085019 18385 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:04.154670 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:04.149271 13376 raft_consensus.cc:2393] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 16: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:20:04.156311 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:04.156425 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:04.157933 12747 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:04.162887 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:04.166568 13385 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
I20260709 13:20:04.166726 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:04.167845 12783 raft_consensus.cc:2410] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 8.
I20260709 13:20:04.126896 18454 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:04.126118 18392 raft_consensus.cc:740] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:04.171012 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:04.141634 18427 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.141181 17966 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:04.173732 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:04.163952 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:04.185598 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:04.186489 12631 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:04.187160 13376 raft_consensus.cc:2410] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 18 because replica has last-logged OpId of term: 16 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 13:20:04.172717 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:04.183334 13382 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:20:04.147234 18370 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:04.190881 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:04.141181 18328 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:04.162490 18330 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:04.191746 12598 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:04.163820 18100 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:04.150080 18414 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:20:04.199728 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:04.200839 12704 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
I20260709 13:20:04.164726 18371 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:04.196422 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:04.202502 13385 raft_consensus.cc:2393] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 11: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:20:04.130013 18357 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:04.175551 18362 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:04.193753 18330 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:04.154287 18450 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:20:04.222627 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:04.224118 13378 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 18: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:20:04.229234 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:04.195879 18100 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:04.232378 12708 raft_consensus.cc:2393] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 13: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:04.233016 12629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:04.233258 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:04.236351 12672 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:04.243643 12597 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:04.244879 12597 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:04.182680 18454 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:04.207484 18362 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:04.259614 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:04.267494 12627 raft_consensus.cc:3060] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:04.185633 18373 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:04.268818 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:04.210842 18330 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:04.225298 18371 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:04.198868 18328 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:04.238497 18100 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:04.237154 18385 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:04.251324 18316 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:04.263285 18439 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:04.282382 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:04.278729 13349 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:20:04.327678 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 49026.2us
I20260709 13:20:04.328421 13349 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:04.199702 18370 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:04.292932 12631 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
I20260709 13:20:04.297519 12705 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:20:04.319254 12627 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:20:04.251732 18310 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:04.192317 18427 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:04.237941 18080 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:04.263675 18309 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:04.330296 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:04.332141 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:04.321622 12516 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 981fb2f435e24d06a81fc02cf4591cc5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 522fd2ba0e9f402691b5ef29af8579cd is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:04.271736 18362 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:04.222105 17966 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:04.234112 17734 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.089s
W20260709 13:20:04.351291 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 159663us
I20260709 13:20:04.335022 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:04.275681 18371 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:04.345765 12627 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:20:04.350283 12631 raft_consensus.cc:1240] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:04.356779 12625 raft_consensus.cc:1240] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:04.356966 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:04.263491 18436 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 13:20:04.231280 18392 consensus_queue.cc:260] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:04.354893 12516 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 84057.1us
I20260709 13:20:04.326629 18439 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:04.344565 18226 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:04.338486 18201 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:04.190759 12525 catalog_manager.cc:5697] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 8 to 9, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 9 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:04.375857 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:04.376255 12631 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:04.371420 18439 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:04.357852 18371 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:04.397153 12625 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:04.401688 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:04.360653 18436 consensus_queue.cc:1243] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer 522fd2ba0e9f402691b5ef29af8579cd log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:20:04.415735 12781 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:20:04.346434 18305 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:04.425364 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:04.429301 12705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 2493) took 1006 ms. Trace:
I20260709 13:20:04.429991 12705 rpcz_store.cc:276] 0709 13:20:03.422715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.422835 (+   120us) service_pool.cc:224] Handling call
0709 13:20:04.429043 (+1006208us) spinlock_profiling.cc:243] Waited 873 ms on lock 0x7b54001d8654. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:20:04.429213 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2008988544}
I20260709 13:20:04.434201 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:04.435011 12631 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:20:04.381233 18180 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:04.402444 18438 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:20:04.425668 18403 log.cc:927] Time spent T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 2.269s	user 0.016s	sys 0.026s
I20260709 13:20:04.399250 18080 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:04.405848 18392 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:04.450181 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:04.452191 12629 raft_consensus.cc:606] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:04.452198 12700 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 16. Current term is 17. Ops: []
W20260709 13:20:04.460042 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:04.461079 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:04.454421 18357 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:04.457311 18468 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.191s	user 0.000s	sys 0.000s
I20260709 13:20:04.460048 12629 raft_consensus.cc:614] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:04.466598 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:04.393824 18201 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:04.493085 18201 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:04.493822 12703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 2498) took 1069 ms. Trace:
I20260709 13:20:04.494233 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:04.494190 12703 rpcz_store.cc:276] 0709 13:20:03.424610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.454001 (+ 29391us) service_pool.cc:224] Handling call
0709 13:20:03.454439 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:03.454758 (+   319us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:03.454799 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:03.455326 (+   527us) log.cc:844] Serialized 26 byte log entry
0709 13:20:04.148589 (+693263us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:04.148632 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:04.148698 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:04.493635 (+344937us) raft_consensus.cc:1685] finished
0709 13:20:04.493656 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:04.493768 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":510880,"threads_started":1,"wal-append.queue_time_us":636635}
I20260709 13:20:04.497643 18080 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:04.499823 18080 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:20:04.503557 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:04.510051 18476 log.cc:927] Time spent T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 13:20:04.511546 13377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 909) took 1226 ms. Trace:
I20260709 13:20:04.515506 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:04.523233 12704 raft_consensus.cc:2410] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:20:04.528029 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:04.528925 12630 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:20:04.512024 13377 rpcz_store.cc:276] 0709 13:20:03.285523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.285738 (+   215us) service_pool.cc:224] Handling call
0709 13:20:03.287170 (+  1432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:04.004991 (+717821us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:04.005026 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:04.005524 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 13:20:04.184746 (+179222us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:04.184799 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:04.184864 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:04.511279 (+326415us) raft_consensus.cc:1685] finished
0709 13:20:04.511308 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:04.511474 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":717445,"raft.queue_time_us":817456,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"thread_start_us":178921,"threads_started":1,"wal-append.queue_time_us":211146}
I20260709 13:20:04.531984 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:04.517467 12779 raft_consensus.cc:1275] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:20:04.485455 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:04.485670 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:04.489229 18373 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:04.545837 18226 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:04.546527 18226 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:04.490701 18310 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:04.549463 18310 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:04.551584 18328 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:04.541523 12778 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:04.552353 18108 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:04.553058 18108 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:04.556473 18108 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:04.558540 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:04.551586 18310 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:04.563351 12778 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:20:04.563798 12625 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:20:04.564421 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:04.565222 12778 raft_consensus.cc:2393] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 14: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:04.566231 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:04.568817 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:04.569945 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:04.571128 12625 raft_consensus.cc:2410] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 8.
W20260709 13:20:04.574996 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:04.593379 12778 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:20:04.593835 12634 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:20:04.545892 18474 log.cc:927] Time spent T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.268s	user 0.000s	sys 0.001s
W20260709 13:20:04.649878 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 104993us
I20260709 13:20:04.651911 12672 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:20:04.751927 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 100119us
I20260709 13:20:04.591653 18373 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:04.760602 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:04.769892 12781 raft_consensus.cc:1275] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:20:04.599038 18454 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:20:04.782197 18479 log.cc:927] Time spent T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.377s	user 0.001s	sys 0.000s
W20260709 13:20:04.781407 12774 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.011s	sys 0.021s
W20260709 13:20:04.786612 12774 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.011s	sys 0.021s
I20260709 13:20:04.764380 18180 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:04.795620 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:20:04.796823 13383 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.006s	sys 0.004s
W20260709 13:20:04.797230 13383 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.006s	sys 0.004s
W20260709 13:20:04.771970 18336 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.003s	sys 0.008s
W20260709 13:20:04.787945 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 369587us
I20260709 13:20:04.763125 18201 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:04.790758 13380 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.051s	sys 0.089s
W20260709 13:20:04.823590 13380 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.051s	sys 0.089s
I20260709 13:20:04.814965 18411 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.774050 18480 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:04.774175 18373 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:04.794133 18370 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:04.820566 18427 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:04.810812 18336 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.003s	sys 0.008s
W20260709 13:20:04.828097 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 474038us
I20260709 13:20:04.835253 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:04.761320 18385 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:20:04.840876 12781 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
W20260709 13:20:04.842538 12747 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.017s	sys 0.083s
W20260709 13:20:04.842955 12747 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.017s	sys 0.083s
W20260709 13:20:04.843724 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 685806us
W20260709 13:20:04.845158 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 491198us
I20260709 13:20:04.846033 13349 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:04.773919 18100 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:04.859469 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:04.859916 13349 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:04.862459 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:04.826175 18428 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:04.826944 18180 raft_consensus.cc:697] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:04.902429 12707 raft_consensus.cc:1240] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 13:20:04.903430 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 43631.7us
I20260709 13:20:04.904503 13349 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:04.899127 12673 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:20:04.906625 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:04.908794 12673 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:20:04.867199 12628 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.004s	sys 0.008s
W20260709 13:20:04.913185 12628 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.004s	sys 0.008s
I20260709 13:20:04.919914 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:20:04.920325 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 75293.3us
I20260709 13:20:04.921011 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:04.918318 12778 raft_consensus.cc:2368] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16.
I20260709 13:20:04.924719 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:04.927084 12673 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:04.937994 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:04.942510 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:04.862711 17966 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:04.944312 12780 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.803s	user 0.009s	sys 0.013s
I20260709 13:20:04.946640 12746 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:20:04.952646 12780 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.811s	user 0.009s	sys 0.013s
I20260709 13:20:04.946914 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:04.832973 18201 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:04.839183 18480 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:04.837061 18411 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:04.841768 18439 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:04.918316 18357 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:04.963201 12706 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:20:04.964738 12703 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:20:04.598788 12525 catalog_manager.cc:5697] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 11 to 14, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 14 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:20:05.030478 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 65210.8us
I20260709 13:20:04.961709 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:05.089210 13385 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:20:04.925371 18435 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:20:05.008530 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:04.978873 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:05.095161 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:04.943171 18370 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.099s
I20260709 13:20:04.954460 18227 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.063153 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:05.058509 18357 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:05.058806 18392 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:05.073869 18180 consensus_queue.cc:237] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:05.080372 18201 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:05.081740 18371 consensus_meta.cc:287] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.656s	user 0.010s	sys 0.001s
I20260709 13:20:05.083276 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:05.090266 13377 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 15. (term mismatch)
I20260709 13:20:05.146647 13377 pending_rounds.cc:85] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd: Aborting all ops after (but not including) 14
I20260709 13:20:05.146936 13377 pending_rounds.cc:107] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd: Aborting uncommitted NO_OP operation due to leader change: 16.15
I20260709 13:20:05.147236 13377 raft_consensus.cc:2889] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:04.951311 18454 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:05.089390 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:20:05.153213 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 63959.6us
I20260709 13:20:05.156378 12597 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:05.030416 18108 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:04.967140 18267 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:04.958861 18436 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:05.191931 12525 catalog_manager.cc:5697] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 7 to 15, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 15 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:05.192837 18436 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:05.193712 12525 catalog_manager.cc:5697] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:05.195327 18436 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:05.195904 18436 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:05.195605 12525 catalog_manager.cc:5697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 12 to 13, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 13 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:05.196360 18436 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:05.198237 18496 log.cc:927] Time spent T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:20:05.198596 18439 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:05.124547 18227 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:05.124181 18435 raft_consensus.cc:697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:05.077191 18373 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:05.040813 17734 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:05.084621 18336 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:20:05.116173 18474 log.cc:927] Time spent T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:20:04.964262 18450 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:20:05.125538 18401 consensus_meta.cc:287] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 1.098s	user 0.014s	sys 0.002s
W20260709 13:20:05.154506 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 250126us
W20260709 13:20:05.181370 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 234499us
I20260709 13:20:05.186712 18108 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:05.192050 18267 raft_consensus.cc:697] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:04.950714 18482 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.130184 18328 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:05.203898 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 238299us
W20260709 13:20:05.211036 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 54771.4us
I20260709 13:20:05.144873 18487 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:05.181313 18371 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:05.181946 18201 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:20:05.213346 18403 log.cc:927] Time spent T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:20:05.232038 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:05.204577 18363 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.220636 18100 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:05.183878 18454 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:05.222457 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 295468us
I20260709 13:20:05.238024 18352 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:05.222481 17734 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:05.238924 18507 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:05.274535 18427 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:05.275332 18427 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:20:05.255682 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:05.248760 18491 log.cc:927] Time spent T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.318s	user 0.000s	sys 0.001s
I20260709 13:20:05.264851 12673 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:05.254256 12597 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:05.260066 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:05.232520 18502 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:05.251650 18328 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:05.258143 18310 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.261126 18363 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:05.283080 18490 log.cc:927] Time spent T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:20:05.253646 18401 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.291443 18427 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:05.278590 18454 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:05.242110 18482 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:05.281066 18371 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:05.321828 18492 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:20:05.327234 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 67259.3us
I20260709 13:20:05.330652 18373 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:05.333547 13383 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.006s	sys 0.011s
W20260709 13:20:05.294085 18330 consensus_meta.cc:287] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 1.010s	user 0.014s	sys 0.000s
I20260709 13:20:05.291741 12673 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:05.287642 18501 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:05.201262 12525 catalog_manager.cc:5697] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:05.326542 18411 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:05.344503 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:20:05.327787 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 73629.8us
W20260709 13:20:05.343080 18362 consensus_meta.cc:287] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.985s	user 0.016s	sys 0.001s
W20260709 13:20:05.346323 13383 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.006s	sys 0.011s
W20260709 13:20:05.345104 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 113713us
I20260709 13:20:05.319851 18511 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:20:05.333189 18480 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:05.335243 18508 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.326879 18201 consensus_queue.cc:1059] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:20:05.354763 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:20:05.398205 18498 log.cc:927] Time spent T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 13:20:05.355350 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 63708.7us
I20260709 13:20:05.354919 12597 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:20:05.424041 18495 log.cc:927] Time spent T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
I20260709 13:20:05.403510 18100 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:05.385411 13383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 1005) took 1195 ms. Trace:
I20260709 13:20:05.332942 18267 consensus_queue.cc:237] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:05.398962 18516 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.332316 18310 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:05.270027 18450 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:20:05.395628 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 51209.1us
I20260709 13:20:05.385964 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:05.453739 12746 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:20:05.454357 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 68518us
I20260709 13:20:05.458027 18507 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:05.458461 18507 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:05.458822 18507 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:05.335371 18401 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:05.450187 18180 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:20:05.463500 18401 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:05.464609 12747 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:05.341786 18484 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:05.292907 18435 consensus_queue.cc:237] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:05.357365 18475 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.004s	user 0.005s	sys 0.020s
I20260709 13:20:05.385243 18392 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:20:05.424751 18499 log.cc:927] Time spent T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
W20260709 13:20:05.425530 18297 consensus_meta.cc:287] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 1.313s	user 0.015s	sys 0.000s
W20260709 13:20:05.397387 18503 log.cc:927] Time spent T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
W20260709 13:20:05.439247 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 84444.1us
W20260709 13:20:05.486515 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 210970us
W20260709 13:20:05.480324 18475 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.132s	user 0.006s	sys 0.022s
I20260709 13:20:05.353178 18330 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:05.440059 18480 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:05.442670 12673 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:05.384415 18362 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:05.495190 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 52598.5us
I20260709 13:20:05.495535 18362 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:05.496134 18362 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:05.433075 18508 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:05.498128 12782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 3050) took 2158 ms. Trace:
I20260709 13:20:05.498342 18508 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:05.498786 18504 raft_consensus.cc:3055] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:05.478348 18513 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:05.449270 18226 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:05.499363 18504 raft_consensus.cc:740] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:05.500433 18504 consensus_queue.cc:260] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:05.467489 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:05.438321 18436 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:05.466809 12746 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:05.477984 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:05.482954 13385 pending_rounds.cc:85] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Aborting all ops after (but not including) 10
I20260709 13:20:05.449347 18328 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:05.464572 18108 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:05.437642 13383 rpcz_store.cc:276] 0709 13:20:04.190488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.191197 (+   709us) service_pool.cc:224] Handling call
0709 13:20:04.203617 (+ 12420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:04.203849 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:20:04.797711 (+593862us) log.cc:844] Serialized 14 byte log entry
0709 13:20:05.385059 (+587348us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:05.385116 (+    57us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:05.385160 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:05.385229 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:05.385355 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29223,"prepare.queue_time_us":560264,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":5472,"raft.queue_time_us":43639,"raft.run_cpu_time_us":67,"raft.run_wall_time_us":67,"thread_start_us":1129575,"threads_started":2}
I20260709 13:20:05.465359 18512 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:20:05.528481 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 61755.6us
I20260709 13:20:05.528115 18487 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.498540 12782 rpcz_store.cc:276] 0709 13:20:03.333153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.333288 (+   135us) service_pool.cc:224] Handling call
0709 13:20:03.333538 (+   250us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:03.334843 (+  1305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:03.334876 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:04.408488 (+1073612us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b180043e700. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2932a6 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6bcaba11 00007f4c6bcbb372 00007f4c6d3eda01 00007f4c6d3b243a
0709 13:20:04.926517 (+518029us) spinlock_profiling.cc:243] Waited 4.58 ms on lock 0x7b2c00160c88. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6bcaba11 00007f4c6bcbb372 00007f4c6d3eda01 00007f4c6d3b243a 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2
0709 13:20:04.955399 (+ 28882us) log.cc:844] Serialized 26 byte log entry
0709 13:20:05.233251 (+277852us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:05.233304 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:05.233410 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:05.491227 (+257817us) raft_consensus.cc:1685] finished
0709 13:20:05.491251 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:05.491379 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11862,"raft.queue_time_us":5217,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"spinlock_wait_cycles":14956544,"thread_start_us":271045,"threads_started":1,"wal-append.queue_time_us":386243}
I20260709 13:20:05.532464 18487 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:05.487917 12632 raft_consensus.cc:2410] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:20:05.490558 12597 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:05.464223 18100 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:20:05.496307 12673 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:05.502144 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:05.537118 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 46651.9us
I20260709 13:20:05.434527 18227 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:05.507186 13385 pending_rounds.cc:107] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Aborting uncommitted NO_OP operation due to leader change: 16.11
W20260709 13:20:05.540261 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 44028.3us
I20260709 13:20:05.508543 12777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 2730) took 1098 ms. Trace:
I20260709 13:20:05.510351 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:05.541604 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 63662.8us
I20260709 13:20:05.511757 18363 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:05.512969 12700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 2513) took 1408 ms. Trace:
I20260709 13:20:05.487563 12598 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
W20260709 13:20:05.514195 18522 log.cc:927] Time spent T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:20:05.514632 18482 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:05.510462 12783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 2731) took 1100 ms. Trace:
I20260709 13:20:05.542428 12700 rpcz_store.cc:276] 0709 13:20:04.104785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.460745 (+355960us) service_pool.cc:224] Handling call
0709 13:20:04.461055 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:04.461369 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:04.461396 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:04.489749 (+ 28353us) log.cc:844] Serialized 26 byte log entry
0709 13:20:04.774640 (+284891us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:05.040897 (+266257us) log.cc:844] Serialized 14 byte log entry
0709 13:20:05.066526 (+ 25629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:05.066661 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:05.512712 (+446051us) raft_consensus.cc:1685] finished
0709 13:20:05.512740 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:05.512895 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":469154,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":16723,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":200,"thread_start_us":538898,"threads_started":2,"wal-append.queue_time_us":595417}
I20260709 13:20:05.478336 12525 catalog_manager.cc:5697] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 11 to 13, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 13 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:05.544734 18297 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:05.537012 18513 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:05.549726 18513 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:05.544108 18512 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:05.532909 18330 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:05.538295 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:05.538295 12597 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:05.540318 13385 raft_consensus.cc:2889] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:05.549963 12525 catalog_manager.cc:5697] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:05.539253 18487 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:05.507169 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:05.539414 18108 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:05.561935 12630 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:20:05.562095 18310 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:05.542744 12673 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:05.502156 17734 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:05.522468 18352 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:05.562995 18363 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:05.563099 18352 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:05.563511 18363 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:05.563946 18363 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:05.541083 12777 rpcz_store.cc:276] 0709 13:20:04.410063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.412781 (+  2718us) service_pool.cc:224] Handling call
0709 13:20:04.441780 (+ 28999us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:04.442214 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:04.442255 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:04.449257 (+  7002us) log.cc:844] Serialized 52 byte log entry
0709 13:20:04.945213 (+495956us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:04.945251 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:04.945335 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:05.508250 (+562915us) raft_consensus.cc:1685] finished
0709 13:20:05.508282 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:05.508465 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":120826,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":18184,"thread_start_us":385603,"threads_started":1,"wal-append.queue_time_us":767745}
I20260709 13:20:05.566267 12673 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:05.566686 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:05.568192 18512 raft_consensus.cc:697] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:05.570717 18427 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:05.573873 17734 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:20:05.575184 18483 log.cc:927] Time spent T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.485s	user 0.002s	sys 0.000s
I20260709 13:20:05.575251 12705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 2153) took 1485 ms. Trace:
I20260709 13:20:05.575884 12705 rpcz_store.cc:276] 0709 13:20:04.125663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.475203 (+349540us) service_pool.cc:224] Handling call
0709 13:20:04.486411 (+ 11208us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:04.760312 (+273901us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:20:04.760688 (+   376us) log.cc:844] Serialized 14 byte log entry
0709 13:20:05.096313 (+335625us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:05.096825 (+   512us) log.cc:844] Serialized 26 byte log entry
0709 13:20:05.096962 (+   137us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:20:05.097151 (+   189us) log.cc:844] Serialized 14 byte log entry
0709 13:20:05.097321 (+   170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:05.097410 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:05.574987 (+477577us) raft_consensus.cc:1685] finished
0709 13:20:05.575013 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:05.575182 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":273383,"prepare.queue_time_us":102,"raft.queue_time_us":300552,"raft.run_cpu_time_us":426,"raft.run_wall_time_us":42599,"spinlock_wait_cycles":6959872,"thread_start_us":333586,"threads_started":1,"wal-append.queue_time_us":477783}
I20260709 13:20:05.578115 12774 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 3070) took 1423 ms. Trace:
I20260709 13:20:05.578928 12629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:33786 (request call id 40) took 1371 ms. Trace:
I20260709 13:20:05.579303 18488 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:05.579708 18488 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:05.580044 18488 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:05.580076 12673 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:05.578537 12774 rpcz_store.cc:276] 0709 13:20:04.154337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.154454 (+   117us) service_pool.cc:224] Handling call
0709 13:20:04.154742 (+   288us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:04.156987 (+  2245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:04.157015 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:04.174309 (+ 17294us) log.cc:844] Serialized 52 byte log entry
0709 13:20:04.791915 (+617606us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:04.791969 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:04.792037 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:05.574551 (+782514us) spinlock_profiling.cc:243] Waited 5.04 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:20:05.577832 (+  3281us) raft_consensus.cc:1685] finished
0709 13:20:05.577854 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:05.578058 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1882,"raft.queue_time_us":52428,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":6999,"spinlock_wait_cycles":11592576,"thread_start_us":587604,"threads_started":1,"wal-append.queue_time_us":753953}
W20260709 13:20:05.548049 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 60593.2us
I20260709 13:20:05.544860 12783 rpcz_store.cc:276] 0709 13:20:04.410376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.412863 (+  2487us) service_pool.cc:224] Handling call
0709 13:20:04.413185 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:04.413538 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:04.413591 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:04.414144 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 13:20:04.838308 (+424164us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:04.838373 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:04.838476 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:05.510243 (+671767us) raft_consensus.cc:1685] finished
0709 13:20:05.510266 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:05.510401 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":42417,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":15265,"thread_start_us":274269,"threads_started":1,"wal-append.queue_time_us":604571}
I20260709 13:20:05.524475 12776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 2734) took 1113 ms. Trace:
I20260709 13:20:05.584292 12776 rpcz_store.cc:276] 0709 13:20:04.411430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.416945 (+  5515us) service_pool.cc:224] Handling call
0709 13:20:04.417280 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:04.525958 (+108678us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:04.525996 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:04.526044 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:04.526087 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:05.524198 (+998111us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54003700d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:20:05.524254 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:05.524398 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":249371520}
I20260709 13:20:05.537891 18504 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:05.451409 18516 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:05.542799 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:05.550458 18492 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:05.579332 12629 rpcz_store.cc:276] 0709 13:20:04.207526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.207684 (+   158us) service_pool.cc:224] Handling call
0709 13:20:05.578671 (+1370987us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b54003d8954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3ac853 00007f4c7237a60a 00007f4c6b9007d4 00007f4c6b900762 00007f4c6b900691 00007f4c6b90060c 00007f4c6b8ff8c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:20:05.578785 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":498524032}
I20260709 13:20:05.586835 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:05.587806 12628 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:20:05.588593 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:05.589111 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:05.590507 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:05.596069 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 53322.3us
I20260709 13:20:05.599929 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:05.600903 12633 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:20:05.602056 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:05.591797 18436 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:05.550300 18513 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:05.605540 18108 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:20:05.605826 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 67639.6us
I20260709 13:20:05.606621 12597 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:05.607182 18352 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:05.609053 18309 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:05.609480 18309 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:05.591647 17734 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:20:05.609884 18309 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:05.614213 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:05.617966 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:05.619000 18362 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:05.620254 12632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56832 (request call id 3424) took 1231 ms. Trace:
I20260709 13:20:05.619845 18362 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:05.620661 12632 rpcz_store.cc:276] 0709 13:20:04.389354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.396605 (+  7251us) service_pool.cc:224] Handling call
0709 13:20:05.620054 (+1223449us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54000dc854. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:20:05.620196 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2440663808}
I20260709 13:20:05.627434 12747 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:05.629303 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:05.638413 18518 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:05.626518 18487 consensus_queue.cc:1059] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.013s
I20260709 13:20:05.639036 18385 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.639424 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:05.639515 12629 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:20:05.639448 17734 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.641255 12673 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:05.627875 18370 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:05.645138 18401 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.645714 18517 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:05.646708 18297 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.650200 12705 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:20:05.560976 12525 catalog_manager.cc:5697] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 9 to 10, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 10 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:05.655016 12778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36488 (request call id 2747) took 1095 ms. Trace:
I20260709 13:20:05.697800 12778 rpcz_store.cc:276] 0709 13:20:04.752909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.760309 (+  7400us) service_pool.cc:224] Handling call
0709 13:20:05.654798 (+894489us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b540040ea54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:20:05.654945 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":339400960}
I20260709 13:20:05.569304 18512 consensus_queue.cc:237] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:05.664330 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 59682.4us
W20260709 13:20:05.672179 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 73436us
W20260709 13:20:05.666081 18477 log.cc:927] Time spent T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 1.230s	user 0.005s	sys 0.001s
I20260709 13:20:05.638541 18427 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:05.660789 18513 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:05.667903 18100 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:05.700331 12598 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:05.707527 12598 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:05.704463 18517 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:05.592865 18507 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:05.713804 12597 leader_election.cc:400] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16.
I20260709 13:20:05.691569 18316 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:05.723511 12598 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:05.724934 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:05.725584 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:05.725867 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:05.726243 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:05.726604 12702 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
I20260709 13:20:05.700274 18370 raft_consensus.cc:697] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:05.682075 18309 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:05.714498 18110 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:05.720777 17966 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:05.685074 17734 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:05.686820 18336 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:20:05.681183 18385 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:05.746603 12701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 2504) took 2312 ms. Trace:
I20260709 13:20:05.672058 18518 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:05.713488 18484 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:05.747063 12701 rpcz_store.cc:276] 0709 13:20:03.426101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:03.464551 (+ 38450us) service_pool.cc:224] Handling call
0709 13:20:03.464839 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:03.465181 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:03.465331 (+   150us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:03.467903 (+  2572us) log.cc:844] Serialized 26 byte log entry
0709 13:20:04.041601 (+573698us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b18004871e0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf520d1 00007f4c6d36da0f
0709 13:20:04.239585 (+197984us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:04.239627 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:04.239690 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:05.612880 (+1373190us) spinlock_profiling.cc:243] Waited 3.42 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:20:05.738368 (+125488us) raft_consensus.cc:1685] finished
0709 13:20:05.738393 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:05.738552 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":107262,"raft.queue_time_us":176,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":88047,"spinlock_wait_cycles":474709504,"thread_start_us":197811,"threads_started":1,"wal-append.queue_time_us":904127}
I20260709 13:20:05.755689 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:05.756515 12747 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:05.730737 18394 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:05.741626 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:05.701639 18401 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:05.723791 18316 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:05.727710 18352 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:05.749861 18492 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.703804 18297 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:05.745788 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:05.759152 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:05.762989 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:05.732227 18441 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:05.764680 18362 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:05.769245 18373 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:05.766414 18316 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:05.788450 18370 consensus_queue.cc:237] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:05.776345 18441 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:05.783252 18529 log.cc:927] Time spent T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:20:05.791752 18309 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:05.796139 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:05.800241 18513 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:05.802402 17734 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:05.802770 18517 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:05.804566 18518 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:05.806164 18401 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:05.761469 18484 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:05.808790 18484 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:05.774528 18492 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:05.809484 18297 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:05.813934 18516 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:05.816866 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:05.814404 18535 log.cc:927] Time spent T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:20:05.825456 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:05.828151 12633 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
I20260709 13:20:05.830330 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:05.831014 12627 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:20:05.831095 18363 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:05.831876 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:05.763998 12525 catalog_manager.cc:5697] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 6 to 12, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 12 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:05.836603 18492 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:05.793396 18362 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:05.818439 18305 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:05.829314 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:05.852880 13384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 1001) took 1701 ms. Trace:
I20260709 13:20:05.858038 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:05.861585 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:05.862602 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:05.863368 12707 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 20.
I20260709 13:20:05.863726 12706 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:20:05.856328 13384 rpcz_store.cc:276] 0709 13:20:04.151805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.151976 (+   171us) service_pool.cc:224] Handling call
0709 13:20:04.152280 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:04.362635 (+210355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:04.362687 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:04.363268 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 13:20:04.602591 (+239323us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:04.602645 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:05.848596 (+1245951us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b5400543154. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:20:05.848647 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:05.848693 (+    46us) raft_consensus.cc:1685] finished
0709 13:20:05.848708 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:05.852796 (+  4088us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1866,"raft.queue_time_us":2186,"raft.run_cpu_time_us":2329,"raft.run_wall_time_us":29739,"spinlock_wait_cycles":478309120,"thread_start_us":239009,"threads_started":1,"wal-append.queue_time_us":478281}
I20260709 13:20:05.871445 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:05.883737 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:20:05.853575 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 97116.7us
I20260709 13:20:05.884534 12701 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:05.884841 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:05.887864 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:05.889492 12633 raft_consensus.cc:2410] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 16 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:20:05.892732 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:05.893772 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:05.893774 12709 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:20:05.895943 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:05.896258 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:05.896250 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:05.899502 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:05.905259 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:05.906234 12629 raft_consensus.cc:2410] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:20:05.896561 13376 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:20:05.908536 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:05.910331 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:05.910421 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:05.911072 18385 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:05.911792 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:05.912621 13379 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 18: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:20:05.914119 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:05.915071 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:05.910493 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:05.910452 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:05.916555 12708 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:20:05.917079 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:05.917662 12709 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:20:05.918411 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:05.918912 12709 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:05.920653 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:05.922559 12672 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:05.925297 18363 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:05.896466 18436 consensus_queue.cc:1048] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.042s
W20260709 13:20:05.939710 18537 log.cc:927] Time spent T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.218s	user 0.002s	sys 0.001s
I20260709 13:20:05.836711 12525 catalog_manager.cc:5697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 13 to 14, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 14 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:05.941324 12525 catalog_manager.cc:5697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 12 to 14, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 14 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:05.942598 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:05.943099 12525 catalog_manager.cc:5697] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:05.922817 18482 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:05.948731 12628 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:20:05.944680 12525 catalog_manager.cc:5697] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:05.954995 12630 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:20:05.958983 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:05.846663 18362 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:05.972808 18362 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:05.960446 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:20:05.964104 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 55637.8us
I20260709 13:20:05.974637 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:05.921511 18441 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:05.967491 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:05.972708 17966 raft_consensus.cc:697] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:05.965056 12704 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:20:05.921465 18450 raft_consensus.cc:3055] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:05.984069 12704 raft_consensus.cc:1240] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:05.981101 12709 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:20:05.987422 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:05.988413 12783 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:20:05.996234 12701 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:20:06.001592 12783 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:20:05.964347 17734 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:05.960068 18507 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:06.004379 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:05.961731 18110 raft_consensus.cc:697] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:06.011034 12778 raft_consensus.cc:2368] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 18. Current term is 19.
I20260709 13:20:05.981863 18427 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:06.012821 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:06.025671 12747 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:05.981954 18357 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:06.026901 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 66165.1us
I20260709 13:20:05.982132 18516 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:05.984233 18450 raft_consensus.cc:740] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:06.017338 18454 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:06.034353 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:06.034929 12634 raft_consensus.cc:2410] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:20:06.035596 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:06.047566 12704 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 17.
I20260709 13:20:06.048503 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:06.049500 12704 raft_consensus.cc:2393] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 13: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:06.050297 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:06.051069 12704 raft_consensus.cc:2410] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:20:06.035817 12747 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:06.003566 18330 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:06.036604 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:06.037849 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:06.052214 12708 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:06.032487 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:05.982372 18488 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:06.056074 12704 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:06.056847 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:06.057641 12672 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:06.058791 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:06.059468 12701 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:20:06.040091 18352 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:06.054517 18484 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:06.058010 18511 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:06.074671 12672 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:06.102236 12708 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 19.
I20260709 13:20:06.033265 18427 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:06.069651 18180 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:06.046792 18516 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:06.061977 18454 raft_consensus.cc:697] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:06.109993 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:06.134083 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:06.136134 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:06.136175 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:06.137194 12775 raft_consensus.cc:2368] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 17. Current term is 18.
I20260709 13:20:06.137915 12633 raft_consensus.cc:1275] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:20:06.138047 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:06.138895 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:06.140002 18336 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:06.140427 18336 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:06.140751 18336 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:06.142159 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:06.142599 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:06.142721 12779 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:20:06.144224 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:06.144829 13376 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:20:06.144884 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:06.146582 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:06.147652 13379 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:06.147802 18441 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:06.148326 18441 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:06.148731 18441 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:06.145529 18357 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:20:06.074761 18316 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:06.154552 12705 raft_consensus.cc:606] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:06.154996 12705 raft_consensus.cc:614] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:20:06.155076 18486 log.cc:927] Time spent T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 1.091s	user 0.000s	sys 0.011s
I20260709 13:20:06.106810 18435 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:06.161726 18357 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:20:06.128505 18545 log.cc:927] Time spent T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.223s	user 0.000s	sys 0.002s
I20260709 13:20:06.156658 12703 raft_consensus.cc:2393] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 19: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:06.157207 12597 leader_election.cc:400] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 17. Current term is 18.
I20260709 13:20:06.157526 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:06.157212 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:06.034160 12525 catalog_manager.cc:5697] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:06.123557 18484 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:06.176999 12780 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:20:06.175217 13380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 998) took 2043 ms. Trace:
I20260709 13:20:06.177767 13380 rpcz_store.cc:276] 0709 13:20:04.132001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:04.132120 (+   119us) service_pool.cc:224] Handling call
0709 13:20:04.132398 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:04.132705 (+   307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:04.132733 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:04.168487 (+ 35754us) log.cc:844] Serialized 26 byte log entry
0709 13:20:04.824374 (+655887us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:04.824440 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:04.824562 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:06.077470 (+1252908us) spinlock_profiling.cc:243] Waited 23.4 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:20:06.164095 (+ 86625us) raft_consensus.cc:1685] finished
0709 13:20:06.164118 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:06.175145 (+ 11027us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":577,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":12410,"spinlock_wait_cycles":53913344,"thread_start_us":616264,"threads_started":1,"wal-append.queue_time_us":772755}
I20260709 13:20:06.180300 12780 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:06.181692 13349 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:06.182701 13349 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:06.174322 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:06.190945 18428 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:06.193243 18226 consensus_queue.cc:1059] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:20:06.194482 18226 raft_consensus.cc:3055] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:06.194864 18226 raft_consensus.cc:740] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:06.195822 18226 consensus_queue.cc:260] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:06.197223 18226 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:06.166489 18504 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:06.198012 18504 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:06.205734 18504 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:06.209671 18438 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:06.210130 18438 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:06.210500 18438 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:06.210647 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:06.212594 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:06.213296 12633 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 20.
I20260709 13:20:06.218853 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:06.222144 18541 log.cc:927] Time spent T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:20:06.227435 12629 raft_consensus.cc:1275] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:20:06.228906 12631 raft_consensus.cc:1240] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:06.105301 18110 consensus_queue.cc:237] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:06.179735 18507 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:06.239200 18316 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:06.240236 18516 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:06.241967 18427 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:06.244295 18488 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:06.124722 17966 consensus_queue.cc:237] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:06.230893 12673 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:06.183701 18484 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:06.257838 18436 consensus_queue.cc:1059] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:06.259966 18428 raft_consensus.cc:697] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:06.254510 18385 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:06.271827 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:06.272528 13376 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:06.272500 18385 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:06.276917 18385 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:06.265025 18454 consensus_queue.cc:237] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:06.290236 18428 consensus_queue.cc:237] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:06.303542 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:20:06.315075 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:06.317349 12631 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:20:06.319809 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:06.272858 18482 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:06.327440 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:06.338696 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:06.340495 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:06.365581 12747 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:06.379491 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:06.390678 13376 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:20:06.403690 13348 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:06.386811 18316 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:06.422425 13348 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:06.427009 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:06.444883 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:06.394325 18080 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:06.323369 18438 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:06.452339 12597 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:06.399438 18511 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:06.457547 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:06.459986 13377 raft_consensus.cc:2393] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 17: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:20:06.460666 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:06.417663 18516 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:06.465305 13379 raft_consensus.cc:2393] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 11: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:20:06.449107 18442 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:06.468137 13384 raft_consensus.cc:1240] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:06.429997 18492 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:06.481408 13385 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:06.486745 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:06.499435 18543 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:06.500200 18543 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:06.460105 18441 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:06.510915 18441 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:06.493757 18492 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:06.515762 18492 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:06.518407 18336 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:06.458447 18480 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.037s
W20260709 13:20:06.534663 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 231224us
I20260709 13:20:06.547936 13380 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 18. Current term is 19. Ops: []
W20260709 13:20:06.548059 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 291636us
I20260709 13:20:06.491391 18321 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:06.552454 12672 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:06.524461 18512 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:06.563551 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:06.585598 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:06.586731 12625 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:06.646695 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:06.663405 12625 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 17.
I20260709 13:20:06.582901 18321 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:06.655692 18297 consensus_queue.cc:1059] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:20:06.746280 18080 raft_consensus.cc:697] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:06.714572 18180 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:06.690235 18484 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:06.768848 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 333290us
I20260709 13:20:06.717701 18385 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:06.814235 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 44465.8us
W20260709 13:20:06.816488 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 371672us
I20260709 13:20:06.816870 12746 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:20:06.817544 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:06.819720 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:06.820482 12703 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:06.820690 12706 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:20:06.820685 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:06.819696 12747 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:06.826679 12709 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 16. Current term is 18. Ops: []
I20260709 13:20:06.827009 12747 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:06.828023 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:20:06.818317 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 265970us
W20260709 13:20:07.032413 12598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.004s	sys 0.012s
W20260709 13:20:07.063005 12598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.007s	sys 0.014s
W20260709 13:20:07.031997 12597 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.000s	sys 0.006s
W20260709 13:20:07.038117 12633 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.005s	sys 0.017s
I20260709 13:20:06.851636 12705 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:20:07.084306 12597 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.000s	sys 0.012s
W20260709 13:20:06.959167 18548 log.cc:927] Time spent T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.901s	user 0.005s	sys 0.000s
I20260709 13:20:06.923184 18572 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:06.850649 13376 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:07.193141 12627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56838 (request call id 2043) took 1441 ms. Trace:
I20260709 13:20:07.230479 12627 rpcz_store.cc:276] 0709 13:20:05.751246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:05.831623 (+ 80377us) service_pool.cc:224] Handling call
0709 13:20:07.192642 (+1361019us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54003d8954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:20:07.192863 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2757043456}
I20260709 13:20:06.848183 12775 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 17. Current term is 18. Ops: []
W20260709 13:20:06.928908 18564 log.cc:927] Time spent T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:20:06.809736 18511 raft_consensus.cc:697] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:20:06.928077 18565 log.cc:927] Time spent T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.121s	user 0.000s	sys 0.004s
W20260709 13:20:06.920219 18568 log.cc:927] Time spent T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
I20260709 13:20:06.841140 12703 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
W20260709 13:20:07.187989 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 122022us
I20260709 13:20:07.173550 18370 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:07.189088 18316 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:20:06.873965 18441 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:07.270855 18267 consensus_queue.cc:1059] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:07.196066 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:07.274415 18267 raft_consensus.cc:3055] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:07.258373 18100 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:07.260023 18201 raft_consensus.cc:3055] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:06.872737 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:07.274919 18267 raft_consensus.cc:740] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:07.275105 12631 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:07.275431 18100 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:07.276571 18226 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:20:07.277099 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 404355us
I20260709 13:20:06.857211 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:07.278744 18352 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:20:07.279853 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 422594us
I20260709 13:20:06.815809 18492 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:07.281116 18267 consensus_queue.cc:260] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:07.281867 18100 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:07.282390 18492 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:07.283043 18267 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:20:07.158092 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 684207us
I20260709 13:20:07.286005 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:07.288206 18576 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:07.288528 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:07.288849 18576 raft_consensus.cc:697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:06.878715 18336 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:07.179433 18488 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:07.290303 18576 consensus_queue.cc:237] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:07.291319 18488 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:07.292335 12672 leader_election.cc:400] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 18. Current term is 19.
I20260709 13:20:07.292963 12672 leader_election.cc:403] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:20:07.255201 18511 consensus_queue.cc:237] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:07.161666 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 505590us
I20260709 13:20:07.269217 18504 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:06.915904 18362 raft_consensus.cc:3055] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:06.920158 18578 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:07.273923 18441 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:06.879123 18385 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:06.921058 18589 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:07.166137 18583 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:07.265703 18572 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:06.867125 18475 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:07.294991 18557 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:07.297230 18362 raft_consensus.cc:740] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:06.913028 18321 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:07.304509 18482 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:20:07.305722 18557 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:07.307336 12778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36478 (request call id 3091) took 1277 ms. Trace:
I20260709 13:20:07.307807 12778 rpcz_store.cc:276] 0709 13:20:06.029302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.029821 (+   519us) service_pool.cc:224] Handling call
0709 13:20:07.305506 (+1275685us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b5400088c54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:20:07.307052 (+  1546us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1009605504}
I20260709 13:20:07.309377 18362 consensus_queue.cc:260] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:07.311224 13380 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 13:20:07.311564 18362 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:07.281405 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:07.314970 18504 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:07.315589 18504 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:07.297302 18336 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 18. Current term is 19.
I20260709 13:20:07.275148 18201 raft_consensus.cc:740] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:07.295095 12672 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:07.285483 18492 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:06.916290 18484 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:06.858594 18438 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:06.966774 18543 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:07.317381 18572 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:07.334306 12633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 3455) took 1198 ms. Trace:
I20260709 13:20:07.334755 18370 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:07.336443 18547 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:07.334995 12633 rpcz_store.cc:276] 0709 13:20:06.136060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.221948 (+ 85888us) service_pool.cc:224] Handling call
0709 13:20:06.222312 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:06.254836 (+ 32524us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:20:06.264367 (+  9531us) log.cc:844] Serialized 14 byte log entry
0709 13:20:06.299506 (+ 35139us) spinlock_profiling.cc:243] Waited 1.59 ms on lock 0x7b1800436840. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c5557 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:20:07.174038 (+874532us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:07.267522 (+ 93484us) spinlock_profiling.cc:243] Waited 45.3 ms on lock 0x7b2c00000c88. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6bcaba11 00007f4c6bcbb372 00007f4c6d3eda01 00007f4c6d3b243a 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2
0709 13:20:07.268318 (+   796us) log.cc:844] Serialized 26 byte log entry
0709 13:20:07.268550 (+   232us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:07.268619 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:07.271626 (+  3007us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:07.333980 (+ 62354us) raft_consensus.cc:1685] finished
0709 13:20:07.334004 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:07.334213 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50161,"prepare.queue_time_us":9373,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":38,"raft.queue_time_us":363325,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":18780,"spinlock_wait_cycles":108026112,"thread_start_us":801412,"threads_started":2,"wal-append.queue_time_us":413779}
I20260709 13:20:07.337116 18547 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:07.337604 18547 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:20:07.334414 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 53331.2us
I20260709 13:20:07.338143 18482 consensus_queue.cc:1059] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:07.339282 18482 consensus_queue.cc:1243] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 522fd2ba0e9f402691b5ef29af8579cd log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:20:07.339953 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:07.340143 18100 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:07.319996 18557 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:07.344050 18227 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:07.340986 18100 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:07.348774 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:07.345041 18226 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:07.339969 18482 raft_consensus.cc:3055] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:07.319689 18441 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:07.350529 12629 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:07.327581 18336 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:07.354064 18492 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:07.352758 18226 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:07.357833 18492 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:07.358531 18100 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:07.359139 18385 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:07.359298 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:07.359915 18385 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:07.360001 18492 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:06.910238 18080 consensus_queue.cc:237] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:07.352963 18482 raft_consensus.cc:740] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:07.374107 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:07.371721 12673 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:07.361524 18352 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:06.963182 18450 consensus_queue.cc:260] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:07.390235 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:07.385749 18578 raft_consensus.cc:697] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:07.396045 13380 raft_consensus.cc:1813] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Handling vote request from an unknown peer 19cd864c053744bebd96093a7982537e
I20260709 13:20:07.399351 18488 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:07.406154 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:07.407606 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:07.408375 18450 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:07.413749 12775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 2767) took 1187 ms. Trace:
I20260709 13:20:07.420498 13380 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 13: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:07.355221 18583 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:07.422757 12629 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 21.
W20260709 13:20:07.423734 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 83983.9us
I20260709 13:20:07.414419 12775 rpcz_store.cc:276] 0709 13:20:06.139120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:06.139217 (+    97us) service_pool.cc:224] Handling call
0709 13:20:06.139454 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:07.274889 (+1135435us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:07.312965 (+ 38076us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b540011c954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:20:07.326252 (+ 13287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1629302144}
I20260709 13:20:07.429267 18598 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:07.429848 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:07.430668 18598 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:07.432909 18583 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:07.433652 18336 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:07.439400 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:07.441118 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:07.433866 18482 consensus_queue.cc:260] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:07.430810 18385 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:07.457858 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 162965us
I20260709 13:20:07.433387 18598 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:07.457847 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:20:07.475466 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 124732us
I20260709 13:20:07.476059 18482 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:07.481630 18589 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:07.491226 12672 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:07.493818 18385 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:07.450340 12630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:07.502293 12630 raft_consensus.cc:606] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:07.502825 12630 raft_consensus.cc:614] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:07.504114 18582 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:07.505045 18582 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:07.505755 12625 pending_rounds.cc:85] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 9
I20260709 13:20:07.506176 12625 pending_rounds.cc:107] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:20:07.506480 12625 raft_consensus.cc:2889] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:07.419414 18201 consensus_queue.cc:260] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:07.519665 12631 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
I20260709 13:20:07.522341 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:07.462167 18352 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:07.541465 18492 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:07.543692 18492 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:07.544471 12598 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:07.546978 18580 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:07.547318 12598 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:07.547597 18580 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:07.548123 18580 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:07.547785 18316 consensus_queue.cc:1059] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:07.550098 18561 raft_consensus.cc:3055] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:07.550567 18561 raft_consensus.cc:740] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:07.551868 18561 consensus_queue.cc:260] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:07.553920 18597 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:20:07.554324 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 89062.5us
I20260709 13:20:07.550091 18543 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 13:20:07.557753 18227 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:07.558349 18226 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:07.558518 18484 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:07.559736 18561 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:07.564523 18504 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:07.569130 18580 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:07.571178 18201 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:07.572033 18580 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:07.576462 18596 consensus_queue.cc:1059] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:07.576959 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:07.577725 18543 raft_consensus.cc:3055] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:07.578259 18543 raft_consensus.cc:740] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:07.578431 18582 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:07.579401 18352 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:07.579387 18578 consensus_queue.cc:237] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:07.579522 18543 consensus_queue.cc:260] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:07.582211 18543 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:07.528272 18336 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:07.588613 18336 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:07.591293 18336 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:07.591802 18580 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:07.592507 18580 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:07.595389 18580 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:07.599793 18504 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:07.533860 18385 consensus_queue.cc:1048] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:20:07.535115 18370 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:07.597285 18580 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:07.613303 12598 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:07.626394 18580 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:07.630563 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:20:07.639366 18603 log.cc:927] Time spent T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.098s	user 0.003s	sys 0.000s
I20260709 13:20:07.639797 18580 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:07.640533 18589 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:07.640439 12598 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:07.631697 18598 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:20:07.641575 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 64782.1us
W20260709 13:20:07.645989 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:07.628412 18316 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:07.648497 13377 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:20:07.631229 18504 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:20:07.661382 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 51805.1us
I20260709 13:20:07.653097 18598 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:07.680446 12673 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:07.654126 18482 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:07.659957 18267 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:07.684208 12597 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:07.686167 18598 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:07.685168 18583 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:07.699064 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:07.697857 18504 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:07.705399 13383 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:20:07.687911 18488 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:07.709527 18488 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:07.712393 18488 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:07.713166 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:07.714054 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 73803.3us
I20260709 13:20:07.713193 18482 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:07.715022 12598 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:07.716967 12598 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:07.717819 18583 raft_consensus.cc:2764] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:20:07.722795 18080 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:07.723546 18080 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:07.726550 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:07.727947 12709 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:07.729336 12703 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:07.729714 18492 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:07.700959 18316 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:07.733924 18316 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:07.681267 18547 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:07.736251 18547 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:07.717221 18482 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:07.739390 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 55333.3us
I20260709 13:20:07.743001 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:20:07.763104 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:07.781658 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:07.781838 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:07.783371 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:07.784466 18267 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:07.786163 12778 raft_consensus.cc:2393] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 19: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
W20260709 13:20:07.726579 18601 log.cc:927] Time spent T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.099s	user 0.002s	sys 0.000s
W20260709 13:20:07.789124 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 72366.5us
I20260709 13:20:07.742449 18572 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:07.790390 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:07.790635 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:07.790854 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:20:07.793879 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 91863.1us
I20260709 13:20:07.797423 18492 raft_consensus.cc:697] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 21 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:07.798533 18492 consensus_queue.cc:237] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:07.801404 18590 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:07.802024 18590 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:07.802475 18590 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:07.790334 18611 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:07.806380 18578 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:07.810386 12598 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:07.816826 12705 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:20:07.817066 18267 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:07.818070 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:07.818753 12702 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:20:07.807106 18611 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:07.823136 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:07.824874 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:07.826102 18611 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:07.827200 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:07.828181 12782 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:20:07.829480 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:07.794305 18370 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:07.836627 12709 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:20:07.757540 18504 consensus_queue.cc:1059] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:20:07.854871 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:07.856227 12776 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:07.859570 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:07.837412 18080 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:07.859957 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:07.870329 12597 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:07.875735 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:20:07.879568 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 69328us
W20260709 13:20:07.883167 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 69614.5us
I20260709 13:20:07.888337 12706 raft_consensus.cc:1275] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:20:07.862901 18580 raft_consensus.cc:3055] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:07.880563 12673 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:07.891577 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:07.891065 18580 raft_consensus.cc:740] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:07.892911 12775 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:20:07.893028 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:07.893831 13384 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:20:07.894114 18580 consensus_queue.cc:260] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:07.894840 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:07.895862 18504 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:07.900849 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:07.871511 18583 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:07.902089 17966 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:07.901868 18583 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:07.902680 17966 raft_consensus.cc:697] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:07.903697 17966 consensus_queue.cc:237] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:07.876549 18578 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:07.907013 18578 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:07.908879 18572 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:07.909174 18590 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:07.890599 18370 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:07.910998 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:07.911792 12781 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:20:07.911911 18590 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:07.912772 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:07.913156 18370 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:07.889006 18080 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:07.915186 18080 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:07.920416 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 66493.6us
I20260709 13:20:07.922091 18363 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:07.922701 18363 raft_consensus.cc:697] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:07.923806 18363 consensus_queue.cc:237] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:07.929214 18080 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:07.936170 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:07.900523 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:20:07.900395 18594 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:07.911015 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:07.904068 18583 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:07.937970 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:07.937323 12783 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 17.
I20260709 13:20:07.942462 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:07.942857 12782 raft_consensus.cc:2393] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 18: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:07.898617 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:07.946231 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:07.947451 18594 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:07.948360 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:07.895876 18580 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:07.950372 18357 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:07.950971 12632 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:07.950984 18357 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:07.954602 12634 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:20:07.956945 18594 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:07.958276 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:07.958746 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:07.958981 12777 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:20:07.960713 18547 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:07.963048 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:07.963948 12783 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:20:07.960233 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:07.965167 18611 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:07.981927 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:07.971244 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:07.960275 12780 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:20:07.969215 12597 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:08.009702 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:08.014261 12783 raft_consensus.cc:1240] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:08.019542 12783 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:08.021878 12631 raft_consensus.cc:1240] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:08.006250 18362 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:08.023198 18362 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:08.031703 18591 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:08.032531 18591 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:08.033193 12628 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:08.036212 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:08.037739 18611 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:08.040488 18611 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:08.042937 18598 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:08.045253 18598 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:07.974924 18582 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:08.045691 12782 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:20:08.047408 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:08.068852 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:08.061641 12776 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:20:08.071286 12632 raft_consensus.cc:2410] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 18 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 13:20:08.079779 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:07.954654 18357 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:08.090741 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:08.095295 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 126251us
I20260709 13:20:08.103792 12597 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:08.094111 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:08.075224 18594 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:08.113300 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.120693 12709 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
W20260709 13:20:08.121423 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.118183 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.088989 18362 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:08.091409 18201 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.100556 18227 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:08.138633 12632 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:08.138011 18582 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:08.145880 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:08.146358 18582 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:08.140115 17734 consensus_queue.cc:1059] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 13:20:08.149042 18504 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:08.150750 18594 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:08.146942 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 186890us
I20260709 13:20:08.107280 18580 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:08.154290 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:08.143066 18591 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:08.162906 12707 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 21 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:08.093077 18441 consensus_queue.cc:1059] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 13:20:08.088192 18226 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 13:20:08.101308 18611 consensus_queue.cc:1059] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:20:08.172739 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:08.131347 18598 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:08.178179 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.176513 18598 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:08.182427 18598 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:08.160615 18504 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:08.183593 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.162313 18580 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:08.189637 12780 raft_consensus.cc:2393] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 17: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:08.156922 18201 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:08.185467 17734 raft_consensus.cc:3055] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:08.207451 12702 raft_consensus.cc:1813] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Handling vote request from an unknown peer 19cd864c053744bebd96093a7982537e
I20260709 13:20:08.193086 18580 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:08.175624 18597 consensus_queue.cc:1059] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:20:08.218518 12702 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 13: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
W20260709 13:20:08.220134 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 103490us
I20260709 13:20:08.210384 17734 raft_consensus.cc:740] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:08.234077 12709 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:08.235991 12631 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:08.240079 18454 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.240631 18454 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:08.242756 18454 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:08.243211 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:08.245134 18591 raft_consensus.cc:3055] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:20:08.245491 13349 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.516s (SENT) [suppressed 4 similar messages]
I20260709 13:20:08.246811 12625 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:20:08.247713 18597 consensus_queue.cc:1243] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer 3d539f8d95d343d8b1696803f18b30d2 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:20:08.248520 18597 raft_consensus.cc:3055] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:08.248893 18597 raft_consensus.cc:740] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:08.249883 18597 consensus_queue.cc:260] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:08.250515 18594 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:08.251300 18597 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:08.245540 18591 raft_consensus.cc:740] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:08.261847 12626 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:08.260353 12706 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
W20260709 13:20:08.269147 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:08.271122 18617 log.cc:927] Time spent T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:20:08.272796 17734 consensus_queue.cc:260] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:08.274358 17734 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:08.278213 18627 log.cc:927] Time spent T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:20:08.269397 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:08.284001 18591 consensus_queue.cc:260] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:08.286602 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.289582 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:08.289934 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.290807 12701 raft_consensus.cc:2393] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 15: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
W20260709 13:20:08.298700 18624 log.cc:927] Time spent T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:20:08.301119 18441 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:08.303534 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:08.322145 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:08.323036 12709 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:08.332034 18201 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:08.330715 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.335896 13377 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:20:08.337977 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 157645us
I20260709 13:20:08.301955 18441 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:08.340646 12703 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:20:08.341465 18441 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:08.342024 18475 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:08.342830 18591 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:20:08.347738 18630 log.cc:927] Time spent T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 13:20:08.349740 18488 raft_consensus.cc:3055] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:20:08.350258 18488 raft_consensus.cc:740] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:08.351459 18488 consensus_queue.cc:260] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:08.353080 18488 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:08.354598 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:08.355100 18633 log.cc:927] Time spent T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:20:08.355129 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:08.357149 18635 log.cc:927] Time spent T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:20:08.363201 18580 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:08.364506 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.366829 12708 raft_consensus.cc:1240] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:08.367489 12747 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:08.365356 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:08.368441 12701 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:08.368643 18352 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:08.370476 18352 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:08.370918 18352 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:08.371361 18352 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:08.377875 12703 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:20:08.387557 12634 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:20:08.395766 12709 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 21.
W20260709 13:20:08.396822 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 227533us
I20260709 13:20:08.396888 12701 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 20.
I20260709 13:20:08.383615 12747 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:08.398061 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:20:08.401635 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 135883us
I20260709 13:20:08.403275 12705 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:20:08.417339 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 58698.4us
I20260709 13:20:08.364499 18080 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
W20260709 13:20:08.443324 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.424196 18589 consensus_queue.cc:1059] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:20:08.447412 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:08.434942 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:08.428165 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.427371 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.449271 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.450115 12704 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:20:08.450233 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.449333 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.451187 12700 raft_consensus.cc:2393] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 16: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:20:08.451606 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:08.445127 18352 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:08.452111 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:08.452208 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.450438 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:08.452929 12708 raft_consensus.cc:2393] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 18: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:20:08.453198 12777 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:20:08.424935 18454 consensus_queue.cc:1048] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:20:08.404613 18583 consensus_queue.cc:1059] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:20:08.465917 18594 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:20:08.466095 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.468135 18580 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:08.468257 18636 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.469156 18636 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:08.471916 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:08.471978 18636 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:08.473308 18385 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.473807 18385 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:08.474351 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:08.474958 18632 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.475435 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.475548 18632 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:08.476588 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:08.479619 12700 raft_consensus.cc:1240] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:08.451117 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.376998 18598 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:08.485800 18589 consensus_queue.cc:1243] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 639ab2db265f4747bb7afb93e833f530 log is divergent from this leader: its last log entry 21.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:20:08.486713 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.486639 12597 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:08.422372 18357 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.491173 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.492211 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:08.493583 12634 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 13:20:08.494109 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.497143 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:08.486563 18583 consensus_queue.cc:1243] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer 3d539f8d95d343d8b1696803f18b30d2 log is divergent from this leader: its last log entry 18.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:20:08.504828 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.505278 12625 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:08.509487 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:08.509969 12701 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:20:08.488897 18594 raft_consensus.cc:697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:08.514003 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.490891 18352 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:08.518723 13348 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:08.518965 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:08.504916 18597 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:08.519810 13348 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:08.519953 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.520828 13348 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:08.521828 18226 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:08.522783 18576 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:08.523663 18576 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:08.524117 18576 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:08.524533 18576 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:08.520069 18597 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:08.515990 12633 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:20:08.525815 12701 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:20:08.521189 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.547400 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:08.548236 12629 raft_consensus.cc:2410] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:20:08.546846 12705 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:20:08.552878 12700 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:08.546772 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:08.562877 12705 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:20:08.530615 18597 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:08.565588 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:08.570349 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.571535 18441 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:08.578629 18226 raft_consensus.cc:697] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 17 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:08.513195 18357 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:08.576483 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:08.575062 12627 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
W20260709 13:20:08.572985 18640 log.cc:927] Time spent T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:20:08.574260 18385 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:08.579912 18226 consensus_queue.cc:237] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:08.584342 12629 raft_consensus.cc:2393] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 19: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:08.586942 12628 raft_consensus.cc:2393] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 16: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:20:08.591562 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:08.596228 12704 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:20:08.598614 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:08.559834 18611 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:20:08.585722 18636 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:08.608454 12705 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:20:08.611865 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 105764us
I20260709 13:20:08.613691 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:08.608203 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.624531 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.630906 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:20:08.634259 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 61743.2us
I20260709 13:20:08.639317 12630 raft_consensus.cc:2393] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 18: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:20:08.617024 18632 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:08.580226 18594 consensus_queue.cc:237] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:08.643023 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:08.649498 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.638783 18547 raft_consensus.cc:3055] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:20:08.656715 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:08.660415 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:08.663398 12707 raft_consensus.cc:2368] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 17. Current term is 18.
I20260709 13:20:08.630416 18636 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:08.654016 18632 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:08.667340 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:08.659826 18547 raft_consensus.cc:740] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:08.690601 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:08.691371 12707 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:20:08.672446 18632 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:08.707639 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 69704.8us
I20260709 13:20:08.695327 12672 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:20:08.723845 18649 log.cc:927] Time spent T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:20:08.724294 18547 consensus_queue.cc:260] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:08.712878 18632 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:08.726087 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 58383.3us
I20260709 13:20:08.726886 12672 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:08.728099 12672 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:08.729118 17734 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:08.730280 18634 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:08.730553 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:08.732547 18201 raft_consensus.cc:3055] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:08.733006 18201 raft_consensus.cc:740] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:08.734005 18201 consensus_queue.cc:260] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:08.735302 18201 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:08.740489 12673 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:08.730281 18572 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:20:08.746905 18572 raft_consensus.cc:3055] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:08.747352 18572 raft_consensus.cc:740] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:08.748332 18572 consensus_queue.cc:260] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:08.750228 18647 log.cc:927] Time spent T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:20:08.778554 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:08.755722 18547 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:08.789601 18636 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:08.790443 17734 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:08.793606 18597 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:08.773746 18572 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:08.783871 18227 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:20:08.804164 18227 consensus_queue.cc:1243] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:20:08.805048 18597 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:08.782629 18598 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:08.828514 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.802295 18080 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:08.841840 18582 consensus_queue.cc:1059] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:08.842978 18636 raft_consensus.cc:3055] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:08.843405 18636 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:08.844332 18636 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:08.844765 17734 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:08.845657 18636 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:08.847913 18632 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:08.853667 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.857057 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:08.863729 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:08.868788 12630 raft_consensus.cc:1275] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:20:08.878031 12630 pending_rounds.cc:85] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 8
I20260709 13:20:08.878304 12630 pending_rounds.cc:107] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:20:08.878525 12630 raft_consensus.cc:2889] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:08.795240 18357 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:08.877398 12777 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:08.877476 12633 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:20:08.883437 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:08.916196 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.919368 12673 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:08.794797 18370 consensus_queue.cc:1059] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 13:20:08.922171 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:20:08.926594 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.928946 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:08.876291 17734 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:08.881349 18591 consensus_meta.cc:287] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.538s	user 0.012s	sys 0.000s
I20260709 13:20:08.940423 12631 raft_consensus.cc:2393] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 14: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:20:08.943500 12780 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:08.917817 18362 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:08.960279 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:08.960873 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:08.961581 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:08.962102 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:08.963740 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:08.964737 13385 raft_consensus.cc:2393] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 18: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:20:08.966150 12777 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:20:08.967738 12774 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 18. Current term is 19. Ops: []
W20260709 13:20:08.978811 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:09.015167 18352 raft_consensus.cc:3055] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:09.015902 18352 raft_consensus.cc:740] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:09.019412 12708 raft_consensus.cc:2410] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 17 because replica has last-logged OpId of term: 16 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
W20260709 13:20:09.019413 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:09.035274 12707 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 15. (term mismatch)
I20260709 13:20:09.035753 12707 pending_rounds.cc:85] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 14
I20260709 13:20:09.036005 12707 pending_rounds.cc:107] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 16.15
I20260709 13:20:09.036289 12707 raft_consensus.cc:2889] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:08.947705 18316 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:20:09.045776 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:09.049835 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 64710.6us
W20260709 13:20:09.058578 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 280106us
I20260709 13:20:09.059080 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:09.056540 12597 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:09.067890 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:09.066541 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:09.061527 12747 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:09.054505 18316 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:09.052091 18370 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.091089 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:20:09.094110 13383 consensus_meta.cc:287] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.592s	user 0.011s	sys 0.002s
I20260709 13:20:09.094609 13383 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:20:09.099051 12775 raft_consensus.cc:1275] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:20:09.078222 18543 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:09.098646 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:09.113818 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:09.117460 12747 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:09.017204 18352 consensus_queue.cc:260] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:09.092666 18316 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:09.129989 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 74779.7us
I20260709 13:20:09.122552 12747 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:09.097146 18227 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:09.095715 18657 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.129547 12597 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:09.142186 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:20:09.147043 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 76017.1us
I20260709 13:20:09.112990 18488 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:20:09.153379 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 51429.8us
I20260709 13:20:09.129666 18589 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:09.160384 12598 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:09.150733 18482 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:09.135424 18543 raft_consensus.cc:697] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:09.161757 12747 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:09.099332 18370 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:09.164348 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 47820.4us
I20260709 13:20:09.164847 12598 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:09.152643 18596 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:09.208324 12672 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:09.165482 18580 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:09.240021 12779 raft_consensus.cc:2410] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:20:09.167337 18579 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:09.214085 18594 consensus_queue.cc:1048] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:20:09.199919 18632 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:09.241900 18666 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:09.253760 18666 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:09.254381 18666 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:09.161655 18657 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:09.259899 18310 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:09.260602 18310 raft_consensus.cc:697] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:09.261484 18310 consensus_queue.cc:237] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:09.263522 18576 consensus_meta.cc:287] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.738s	user 0.005s	sys 0.005s
W20260709 13:20:09.250850 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 50959.6us
I20260709 13:20:09.272310 12598 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:09.275883 12672 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:09.278450 12780 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:20:09.285269 12598 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:09.283814 18590 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:09.287503 12598 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:09.286504 18590 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:09.289203 18590 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:09.289487 13377 raft_consensus.cc:1275] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:20:09.290680 18654 log.cc:927] Time spent T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
W20260709 13:20:09.293006 18661 log.cc:927] Time spent T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 13:20:09.295029 18656 log.cc:927] Time spent T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
W20260709 13:20:09.297008 18663 log.cc:927] Time spent T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
W20260709 13:20:09.299189 18658 log.cc:927] Time spent T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:20:09.301832 18363 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:20:09.305856 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 106679us
I20260709 13:20:09.306823 12598 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:20:09.273785 18655 log.cc:927] Time spent T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.323s	user 0.002s	sys 0.000s
W20260709 13:20:09.281337 18201 consensus_meta.cc:287] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.546s	user 0.009s	sys 0.003s
I20260709 13:20:09.314190 18597 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:09.314743 18597 raft_consensus.cc:697] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:09.315034 18370 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:09.315485 18657 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:09.315659 18363 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:09.315716 18597 consensus_queue.cc:237] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:09.317884 18363 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:09.302099 18669 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:09.320668 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:09.321766 12781 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:09.323470 12701 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 19. Current term is 20. Ops: [18.9-19.10]
I20260709 13:20:09.324153 12705 raft_consensus.cc:1275] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:20:09.322000 12780 raft_consensus.cc:2368] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 17. Current term is 18.
I20260709 13:20:09.344818 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:09.264119 18576 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:09.358937 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 83090.1us
I20260709 13:20:09.360378 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:09.365520 12672 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:09.371932 12672 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:09.351086 18363 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:09.382874 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:09.383589 12708 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:20:09.384789 18639 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:09.385488 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:09.386031 18576 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:09.386309 12627 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:09.387332 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:09.380504 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:09.308007 18543 consensus_queue.cc:237] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:09.402493 18668 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:09.402801 18639 raft_consensus.cc:697] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:09.404197 18639 consensus_queue.cc:237] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:09.387108 18357 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:09.407213 18357 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:09.407604 18357 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:09.378213 18634 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:09.421123 13349 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:09.422034 18576 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:09.415701 18666 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:09.426349 18362 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:09.426821 18362 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:09.427485 18666 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:09.428160 18441 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:09.431671 13349 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:09.389761 13379 raft_consensus.cc:2393] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 14: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:20:09.389269 12747 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:20:09.444078 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 61733us
I20260709 13:20:09.445348 12672 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:09.451059 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
W20260709 13:20:09.453946 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 64824us
I20260709 13:20:09.454214 13349 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:09.454952 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:09.459432 12747 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:09.462299 18576 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:09.457069 18591 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:09.455189 18652 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:09.468894 13349 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:09.460762 18482 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:09.457335 18674 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:09.472256 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:09.467229 18669 raft_consensus.cc:697] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 21 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:09.470114 18370 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:20:09.482064 18679 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:09.482585 18679 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:09.482925 18679 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:20:09.491866 18671 log.cc:927] Time spent T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:20:09.498306 18316 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:09.494488 18659 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:20:09.475574 18441 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:20:09.502396 17734 consensus_meta.cc:287] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.550s	user 0.007s	sys 0.004s
W20260709 13:20:09.502415 18667 log.cc:927] Time spent T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.255s	user 0.002s	sys 0.000s
I20260709 13:20:09.503041 17734 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:09.505064 17734 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:09.506747 18547 consensus_meta.cc:287] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.706s	user 0.008s	sys 0.002s
I20260709 13:20:09.507505 12777 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:20:09.500886 18316 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:09.510994 13349 leader_election.cc:400] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 17. Current term is 18.
I20260709 13:20:09.511350 13349 leader_election.cc:403] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:09.513651 18659 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:09.513917 18441 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:20:09.514200 18659 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:09.520543 18454 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:09.516390 18659 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:09.521073 18454 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:09.524993 18454 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:09.526602 18454 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:09.527045 18454 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:09.528009 12597 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:09.528955 18454 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:09.534578 17734 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.535281 17734 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:09.537667 17734 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:09.545962 18572 consensus_meta.cc:287] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.736s	user 0.011s	sys 0.000s
I20260709 13:20:09.478518 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:20:09.556990 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 78527.1us
I20260709 13:20:09.557924 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:20:09.566437 18672 log.cc:927] Time spent T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:20:09.568813 18540 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:09.574193 18362 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:09.579066 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:09.580569 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:09.569345 18540 raft_consensus.cc:697] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:09.583709 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:20:09.583940 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:09.589600 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:09.592104 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:09.593323 12747 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:09.564445 18579 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:09.501360 18590 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:09.600172 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:09.584681 18652 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.606289 18652 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:09.608604 18652 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:09.603508 18484 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:09.609760 18227 raft_consensus.cc:2764] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:20:09.591393 18572 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:09.588001 18362 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:09.582072 18454 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:20:09.620198 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:09.625842 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 148064us
I20260709 13:20:09.627781 12631 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:20:09.628106 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:09.627374 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:09.628789 12631 pending_rounds.cc:85] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 9
I20260709 13:20:09.629220 12631 pending_rounds.cc:107] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:20:09.630537 18504 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:09.631194 18504 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:09.599686 18482 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:09.633397 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:09.634330 18441 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 17. Current term is 18.
I20260709 13:20:09.634644 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:09.630167 18669 consensus_queue.cc:237] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:09.640403 12631 raft_consensus.cc:2889] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:09.643090 12633 pending_rounds.cc:85] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 8
I20260709 13:20:09.643400 12633 pending_rounds.cc:107] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:20:09.643618 12633 raft_consensus.cc:2889] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:09.644050 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:09.651146 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:09.651861 12783 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:20:09.652837 12597 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:09.624485 18572 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:20:09.615144 18357 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:09.666857 12747 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:09.646260 18504 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:09.666522 12672 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:09.646667 18362 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:09.642779 18482 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 13:20:09.692950 18678 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.693552 18678 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:09.695798 18678 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:09.696236 18362 raft_consensus.cc:697] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 18 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:09.697332 18362 consensus_queue.cc:237] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:09.704734 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:09.714803 13376 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:09.716646 18310 consensus_queue.cc:1048] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:09.672684 18540 consensus_queue.cc:237] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:09.732594 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:09.733346 13376 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:20:09.680943 18659 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:09.661301 18579 consensus_queue.cc:1048] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:20:09.732633 17966 consensus_queue.cc:1059] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:20:09.736220 12597 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:09.736713 17966 raft_consensus.cc:3055] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:09.737053 18583 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:09.671583 18673 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:09.737550 18583 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:09.749882 18583 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:09.750077 18352 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:09.696048 18316 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:09.749790 18673 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:09.737231 17966 raft_consensus.cc:740] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:09.679600 12747 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:20:09.764295 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 84795.4us
I20260709 13:20:09.759790 12627 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:20:09.762513 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:09.767428 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:09.779690 13348 leader_election.cc:400] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 17. Current term is 18.
I20260709 13:20:09.781759 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:09.781451 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:09.789682 18674 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:09.790794 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:09.793742 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:09.793864 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:09.794972 18675 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:09.795616 12777 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:20:09.737514 18357 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:09.810998 12747 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:09.762975 18673 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:09.815436 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:09.818178 12701 raft_consensus.cc:1240] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:09.786109 18484 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:20:09.828173 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 109419us
I20260709 13:20:09.831048 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:09.832209 17966 consensus_queue.cc:260] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:09.812938 18201 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.810300 18675 raft_consensus.cc:697] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:09.821985 18484 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:09.850835 18674 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:09.821847 18657 consensus_queue.cc:1059] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 13:20:09.810144 18578 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
W20260709 13:20:09.925529 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 114617us
I20260709 13:20:09.927063 13348 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:09.923347 12628 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:09.929086 12632 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:20:09.924811 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:09.876770 17966 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:09.938280 12774 raft_consensus.cc:2393] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 17: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:09.903147 18590 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:09.904685 18454 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.857105 18201 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:09.958707 18454 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:09.923950 18484 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:20:09.880590 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 214135us
I20260709 13:20:09.974609 12672 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:09.981416 12672 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:09.983037 12672 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:09.999296 12704 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:20:10.005769 12598 leader_election.cc:336] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.550s (SENT)
I20260709 13:20:10.006425 12598 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:10.011307 18582 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:10.011722 12672 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:09.982420 18657 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:09.952656 18572 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:20:10.026546 12672 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:09.989849 18316 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:09.991159 18611 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:10.058471 12672 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:20:10.058532 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 131598us
W20260709 13:20:10.041728 18596 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.794s	user 0.006s	sys 0.030s
I20260709 13:20:09.989274 18679 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:10.036605 18363 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:10.067298 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:10.092211 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:10.096526 12704 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:10.069499 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 134579us
I20260709 13:20:10.073987 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:10.044572 18683 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:20:10.074945 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 69234.8us
I20260709 13:20:10.076467 12703 raft_consensus.cc:1240] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:10.088578 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:10.112133 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:10.112680 12709 raft_consensus.cc:2376] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18. Re-sending same reply.
I20260709 13:20:10.056586 18582 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.061247 18693 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:10.094430 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:10.130833 12706 raft_consensus.cc:2393] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 15: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:10.107223 12747 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:10.129220 12703 raft_consensus.cc:1275] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:20:10.091516 18596 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.845s	user 0.007s	sys 0.031s
I20260709 13:20:10.129644 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:10.079090 17734 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:10.072008 18634 consensus_queue.cc:1048] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:20:10.141839 12700 raft_consensus.cc:2376] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16. Re-sending same reply.
I20260709 13:20:10.012488 18675 consensus_queue.cc:237] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:10.076056 18685 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.072s
I20260709 13:20:10.090377 18652 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:10.133073 18636 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:20:10.157478 18688 log.cc:927] Time spent T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:20:10.119678 18589 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.187767 18316 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.213975 18669 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.165381 18583 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:10.167152 18679 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:10.212915 18589 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:10.215031 18669 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:10.217541 18589 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:10.173623 18693 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:10.214046 18316 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:10.218806 18685 consensus_queue.cc:1243] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer 3d539f8d95d343d8b1696803f18b30d2 log is divergent from this leader: its last log entry 20.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:20:10.217833 18669 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:10.219324 18679 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:20:10.203317 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 96151.5us
I20260709 13:20:10.221930 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:10.182278 18597 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.223181 18597 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:10.224016 18578 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:10.141366 18582 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:10.225021 18578 consensus_queue.cc:1059] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:10.226219 18597 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:10.162917 18697 log.cc:927] Time spent T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:20:10.227932 18363 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.228601 18363 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:10.214896 18652 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:10.232242 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:10.174499 18201 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:10.202435 18352 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:10.169013 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:10.236989 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:20:10.242080 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:10.249219 12702 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:20:10.256170 12777 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:20:10.256623 12704 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:20:10.256666 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:10.257861 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:10.258208 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:10.258451 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:10.258939 12705 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:20:10.259845 12782 raft_consensus.cc:1275] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 10. (term mismatch)
I20260709 13:20:10.261835 12782 pending_rounds.cc:85] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 9
I20260709 13:20:10.262068 12782 pending_rounds.cc:107] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 13:20:10.262288 12782 raft_consensus.cc:2889] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:10.271894 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:10.274988 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:10.273329 12705 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 21.
W20260709 13:20:10.198158 18694 log.cc:927] Time spent T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:20:10.271623 12626 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:20:10.285384 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:10.286510 12705 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:10.286652 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:10.146574 18596 raft_consensus.cc:697] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:10.199275 18454 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:10.293247 18578 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.293843 18578 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:10.299722 18316 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:10.300109 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:10.300721 18547 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.301452 18547 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:10.304352 18547 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:10.256260 18583 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:10.307673 18363 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:10.272472 18674 raft_consensus.cc:3055] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:10.310055 18674 raft_consensus.cc:740] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:10.310654 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:10.311444 12706 raft_consensus.cc:2393] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 14: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:10.311017 18674 consensus_queue.cc:260] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:10.312266 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:10.312899 12706 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:20:10.310535 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:10.313728 18674 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:10.314350 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:10.266044 18589 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:20:10.300755 12709 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:10.330102 18484 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:10.336452 18578 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:10.321831 12747 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:10.341886 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:10.332783 18673 consensus_meta.cc:287] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.517s	user 0.010s	sys 0.001s
I20260709 13:20:10.353715 12706 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 17.
I20260709 13:20:10.356958 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:10.365789 18711 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.365260 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:10.371652 12704 raft_consensus.cc:2410] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:20:10.372427 12774 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:20:10.372509 12709 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:20:10.376679 12703 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:20:10.343354 18578 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:20:10.380311 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 287856us
I20260709 13:20:10.387410 12705 raft_consensus.cc:2410] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 18 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 13:20:10.380375 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:10.342893 18652 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:10.399500 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:10.388742 12709 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:10.390051 12703 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:20:10.392195 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:10.402544 13379 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:10.393501 13376 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:20:10.400545 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:10.405802 12774 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:20:10.379079 18579 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.373729 18636 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.411046 12703 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:10.388892 18578 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:10.418974 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:10.421847 12782 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:10.423521 12705 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:20:10.426854 18579 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:10.430104 18683 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:20:10.431707 18693 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:10.424378 18582 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:10.435037 18441 raft_consensus.cc:3055] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:10.497754 18441 raft_consensus.cc:740] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:10.436156 12700 raft_consensus.cc:1240] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:10.438100 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:10.506032 12776 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 19.
I20260709 13:20:10.425019 18636 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:10.440713 12779 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:10.366416 18711 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:10.525781 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 49777us
I20260709 13:20:10.443079 18576 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:10.370290 18673 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:10.411131 18594 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:10.455336 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:10.455592 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:20:10.456243 18707 log.cc:927] Time spent T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:20:10.456514 12781 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:20:10.460666 17966 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:10.461966 18201 consensus_queue.cc:1048] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:10.463580 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:10.535320 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:10.535754 12706 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
I20260709 13:20:10.463798 18484 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:10.419703 18596 consensus_queue.cc:237] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:10.425889 18578 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:10.467631 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:10.469483 12774 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:20:10.470005 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:10.470676 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:20:10.472671 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 45850.1us
I20260709 13:20:10.475095 12597 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:10.475133 18669 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:10.477717 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:10.479318 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:10.482954 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:10.483184 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:10.483527 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:10.487025 13379 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
W20260709 13:20:10.550930 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 75787.4us
I20260709 13:20:10.565261 12672 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:10.568320 12775 pending_rounds.cc:85] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 11
I20260709 13:20:10.568668 12775 pending_rounds.cc:107] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 13:20:10.568910 12775 raft_consensus.cc:2889] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:10.570508 12597 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:10.527058 18710 consensus_queue.cc:1059] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:20:10.583818 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:10.584511 12783 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:20:10.546336 18582 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.562165 18683 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:10.536501 18589 consensus_queue.cc:1059] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:20:10.591770 18683 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:10.592200 18683 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:10.556298 18441 consensus_queue.cc:260] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:10.595912 18441 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:10.597237 18352 raft_consensus.cc:3055] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:10.597782 18352 raft_consensus.cc:740] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:10.495539 18579 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:10.526371 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
W20260709 13:20:10.599408 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 73122.5us
I20260709 13:20:10.599228 18352 consensus_queue.cc:260] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:10.600312 18673 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:10.601239 18636 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:10.589908 18362 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:10.601897 18711 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:10.602831 18352 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:10.531930 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:10.601125 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:10.551769 12629 raft_consensus.cc:2393] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 18: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:20:10.534390 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:10.606040 12776 raft_consensus.cc:2393] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 17: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:10.552990 12673 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:10.494962 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:10.607288 13380 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:20:10.586993 12672 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:10.590847 18582 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:10.603498 18657 raft_consensus.cc:3055] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:10.542665 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:10.490213 12703 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:20:10.610670 12781 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:20:10.611269 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:10.615097 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:20:10.621587 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 68661.7us
W20260709 13:20:10.621997 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 90655.4us
I20260709 13:20:10.625873 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:10.630959 12703 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
W20260709 13:20:10.633993 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 100431us
I20260709 13:20:10.589238 18696 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:10.650506 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:10.656987 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:10.679044 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 66284.1us
W20260709 13:20:10.681545 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 196758us
W20260709 13:20:10.647537 18385 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.005s	sys 0.009s
I20260709 13:20:10.614411 18679 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:10.614398 18657 raft_consensus.cc:740] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:10.693792 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:10.697798 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 47396.8us
I20260709 13:20:10.632817 18362 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:10.678503 18652 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:20:10.679284 18702 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:20:10.708547 12598 leader_election.cc:336] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.713s (SENT)
I20260709 13:20:10.695277 18696 raft_consensus.cc:697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:10.698792 18711 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:10.703048 18475 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:10.716807 12746 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:10.704655 18362 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:10.711268 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 60041.7us
I20260709 13:20:10.741609 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:10.710563 18659 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:20:10.760361 12701 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.008s	sys 0.018s
W20260709 13:20:10.761637 12701 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.008s	sys 0.018s
I20260709 13:20:10.761405 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:10.762226 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:10.769651 13385 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:20:10.721345 18475 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:10.782305 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:10.726238 18362 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:10.783134 12781 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:10.767910 18706 log.cc:927] Time spent T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 13:20:10.790695 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 86010.6us
I20260709 13:20:10.738794 18675 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:10.762619 18589 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:20:10.793584 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 101717us
I20260709 13:20:10.797793 12598 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:10.797245 12672 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:10.775388 18710 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:10.805405 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:10.784220 18475 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:10.806231 12625 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
W20260709 13:20:10.841241 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 43518.2us
I20260709 13:20:10.800228 18589 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:10.777743 17966 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:20:10.783852 18669 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:20:10.851763 12629 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:20:10.852461 12598 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:10.810461 18673 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:20:10.854166 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 65528.9us
I20260709 13:20:10.762449 18657 consensus_queue.cc:260] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:10.774917 18659 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:10.801774 18583 consensus_queue.cc:1059] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
W20260709 13:20:10.885134 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 88002.3us
I20260709 13:20:10.887526 12672 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:10.852586 18589 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:10.802126 18675 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:10.899199 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:10.901948 12672 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:10.818598 18702 raft_consensus.cc:697] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:10.854839 18597 consensus_queue.cc:1048] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:20:10.940536 12701 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:20:10.898303 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:20:10.931671 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 79285.3us
I20260709 13:20:10.889128 18685 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:10.808686 18696 consensus_queue.cc:237] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:10.950629 12706 raft_consensus.cc:1275] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:20:10.892436 18636 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:20:10.954696 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:10.955664 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:10.956952 12701 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:20:10.957428 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:10.898967 18310 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:10.924665 18582 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:10.961376 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:10.963655 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:10.963598 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:10.964308 12704 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:20:10.970247 18723 log.cc:927] Time spent T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:20:10.974095 12705 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:20:10.978807 18733 log.cc:927] Time spent T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:20:10.984783 12672 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:10.949298 18591 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:10.885735 18735 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.083s
W20260709 13:20:11.004627 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 51502.6us
I20260709 13:20:11.006616 12598 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:11.006719 12779 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
W20260709 13:20:10.977455 18720 log.cc:927] Time spent T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
W20260709 13:20:10.988744 18724 log.cc:927] Time spent T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 13:20:10.997466 18591 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:11.012981 18718 log.cc:927] Time spent T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.293s	user 0.000s	sys 0.001s
I20260709 13:20:10.961444 18734 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 13:20:11.065441 18659 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:11.033102 18666 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:20:11.081055 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 392339us
I20260709 13:20:11.036197 18669 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:11.002825 18310 raft_consensus.cc:697] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:11.086568 12704 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
W20260709 13:20:11.034795 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 50166.2us
I20260709 13:20:11.057049 18591 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:11.091287 12672 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:20:11.081770 18720 log.cc:927] Time spent T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:20:11.101024 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:11.102613 12627 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: []
W20260709 13:20:11.104919 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 70604.6us
W20260709 13:20:11.087504 18724 log.cc:927] Time spent T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:20:11.107710 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:11.116093 12672 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:11.071332 18597 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:20:11.056288 18596 consensus_queue.cc:1059] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:20:11.118243 18685 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:11.126286 18685 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:20:11.126936 18731 log.cc:927] Time spent T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:20:11.127681 18685 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:20:11.128420 18732 log.cc:927] Time spent T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 13:20:11.101373 18669 raft_consensus.cc:697] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:11.139168 12781 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:20:11.141479 12778 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: []
W20260709 13:20:11.123294 18441 consensus_meta.cc:287] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.525s	user 0.011s	sys 0.000s
I20260709 13:20:11.066969 18702 consensus_queue.cc:237] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:11.210317 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 277989us
W20260709 13:20:11.217284 18721 log.cc:927] Time spent T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
W20260709 13:20:11.168996 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 68573.1us
I20260709 13:20:11.169664 13377 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
W20260709 13:20:11.192816 18720 log.cc:927] Time spent T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:20:11.227397 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 111433us
I20260709 13:20:11.230604 12672 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:11.230855 12709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 826) took 1053 ms. Trace:
I20260709 13:20:11.237705 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:11.240379 18357 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:11.240933 12672 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:11.242429 18722 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:11.244136 12672 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:11.244218 18591 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:11.245067 18722 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:11.245661 18722 raft_consensus.cc:697] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:11.245699 18669 consensus_queue.cc:237] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:11.246755 18582 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:11.246806 18722 consensus_queue.cc:237] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:11.225720 18578 consensus_meta.cc:287] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.660s	user 0.008s	sys 0.003s
I20260709 13:20:11.254073 13349 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:20:11.246487 18729 log.cc:927] Time spent T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:20:11.231200 12709 rpcz_store.cc:276] 0709 13:20:10.177205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.473621 (+296416us) service_pool.cc:224] Handling call
0709 13:20:10.479268 (+  5647us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:10.480047 (+   779us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:20:10.480752 (+   705us) log.cc:844] Serialized 14 byte log entry
0709 13:20:10.686908 (+206156us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:10.694395 (+  7487us) log.cc:844] Serialized 52 byte log entry
0709 13:20:10.694852 (+   457us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:10.694885 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:10.695009 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:11.230638 (+535629us) raft_consensus.cc:1685] finished
0709 13:20:11.230661 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:11.230800 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9232,"prepare.queue_time_us":208,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":13949,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":5322,"thread_start_us":196312,"threads_started":1,"wal-append.queue_time_us":231827}
W20260709 13:20:11.228858 18737 log.cc:927] Time spent T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.267s	user 0.001s	sys 0.000s
I20260709 13:20:11.286916 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:11.276242 18675 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:11.275463 18578 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:11.344426 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 235218us
I20260709 13:20:11.312867 17966 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:20:11.343005 18745 log.cc:927] Time spent T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.263s	user 0.001s	sys 0.000s
I20260709 13:20:11.234491 18310 consensus_queue.cc:237] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:11.357016 12598 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:11.334170 18597 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:11.366436 12598 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:11.367141 12782 raft_consensus.cc:2368] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18.
I20260709 13:20:11.375645 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:11.364768 18678 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:11.387503 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:11.363592 18730 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:11.353091 18669 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:20:11.354995 17966 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:11.374425 18591 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:11.365175 18696 consensus_queue.cc:1048] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:20:11.361141 18747 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.027s
W20260709 13:20:11.390002 18475 consensus_meta.cc:287] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.540s	user 0.008s	sys 0.003s
I20260709 13:20:11.392133 18582 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:20:11.403976 18352 consensus_meta.cc:287] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.790s	user 0.009s	sys 0.004s
I20260709 13:20:11.401759 18730 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:11.416393 17966 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:11.416998 18746 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:20:11.423623 18757 log.cc:927] Time spent T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
W20260709 13:20:11.427753 18758 log.cc:927] Time spent T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:20:11.442011 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:20:11.493261 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 51373.7us
W20260709 13:20:11.444218 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 72111.7us
W20260709 13:20:11.494313 18362 consensus_meta.cc:287] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.696s	user 0.011s	sys 0.000s
I20260709 13:20:11.461521 18475 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:11.462404 18751 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.091s
I20260709 13:20:11.496124 12780 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 20. Current term is 21. Ops: [20.10-20.10]
I20260709 13:20:11.497074 18475 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:20:11.466531 18741 log.cc:927] Time spent T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.430s	user 0.000s	sys 0.003s
I20260709 13:20:11.477888 18582 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:11.474547 18710 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:11.501598 18710 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:11.503571 18736 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:11.504071 18736 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:11.504470 18736 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:11.377300 18597 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:11.406025 18739 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:20:11.514783 18716 log.cc:927] Time spent T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.721s	user 0.000s	sys 0.002s
W20260709 13:20:11.488761 18683 consensus_meta.cc:287] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.896s	user 0.006s	sys 0.004s
W20260709 13:20:11.518616 18760 log.cc:927] Time spent T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:20:11.522385 18639 raft_consensus.cc:3055] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:11.523036 18639 raft_consensus.cc:740] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:11.530285 18673 raft_consensus.cc:3055] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:11.530848 18673 raft_consensus.cc:740] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:11.531486 18639 consensus_queue.cc:260] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:11.531893 18673 consensus_queue.cc:260] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:11.538301 18634 consensus_queue.cc:1059] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:11.539747 18634 raft_consensus.cc:3055] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:11.540134 18634 raft_consensus.cc:740] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:11.571197 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:11.572857 12673 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:20:11.573380 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 79015.4us
I20260709 13:20:11.494761 18362 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:11.573735 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:11.587069 12709 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:20:11.588662 12782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36478 (request call id 3130) took 1122 ms. Trace:
I20260709 13:20:11.589982 12776 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:20:11.540966 18634 consensus_queue.cc:260] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:11.605437 12597 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:11.589084 12782 rpcz_store.cc:276] 0709 13:20:10.465894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.469658 (+  3764us) service_pool.cc:224] Handling call
0709 13:20:11.588475 (+1118817us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b54002cae54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:20:11.588607 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1860741376}
I20260709 13:20:11.607661 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:11.614589 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:11.596019 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:11.619297 12706 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:11.595111 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:20:11.595655 18589 consensus_meta.cc:287] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.668s	user 0.012s	sys 0.000s
I20260709 13:20:11.579233 18597 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:11.579445 18730 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:11.584969 18578 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:11.629528 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:20:11.637763 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 63680.1us
I20260709 13:20:11.635794 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:11.640650 12780 raft_consensus.cc:2393] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 17: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:11.607510 18673 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:11.601681 18639 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:11.654700 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:11.660248 12709 raft_consensus.cc:2376] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19. Re-sending same reply.
I20260709 13:20:11.656066 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:11.635666 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:11.578743 18683 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:11.658067 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:11.629993 18750 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:11.622145 18693 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:11.674113 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:11.614271 18668 consensus_queue.cc:1059] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:20:11.683497 12702 raft_consensus.cc:2368] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16.
I20260709 13:20:11.684934 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:11.653899 18739 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:11.679234 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:11.636050 18352 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:11.690214 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 51859.4us
I20260709 13:20:11.692898 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:11.685741 12706 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 21.
I20260709 13:20:11.631968 18441 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:20:11.658715 18316 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:11.697006 12630 raft_consensus.cc:1240] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:11.673326 18751 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:11.700146 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:11.694674 18578 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:11.700985 18352 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:11.637440 18589 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:11.681887 18636 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:11.727155 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:11.739811 12779 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:11.741421 12747 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:11.751605 13383 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:11.757148 12673 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:11.719815 18582 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:11.682374 18767 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:20:11.690313 18750 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:11.721272 18751 raft_consensus.cc:697] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:11.708052 18316 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:11.756901 18743 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:20:11.788215 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 79917.1us
I20260709 13:20:11.699119 18739 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:11.730454 18636 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:11.790166 18743 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:11.803128 18743 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:11.804374 18636 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:11.807721 12626 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:20:11.791306 18594 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:11.830904 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:11.830962 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:11.806162 18227 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:11.844659 12630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 3558) took 1190 ms. Trace:
I20260709 13:20:11.848636 12630 rpcz_store.cc:276] 0709 13:20:10.654635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.654781 (+   146us) service_pool.cc:224] Handling call
0709 13:20:10.655115 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:11.702757 (+1047642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:11.844441 (+141684us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54002db4d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:20:11.844600 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2393758976}
I20260709 13:20:11.812711 18362 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:11.854492 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:11.826586 18710 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:11.860145 18710 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:11.835652 18594 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:11.862322 18750 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:11.863140 18751 consensus_queue.cc:237] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:11.863346 18227 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:11.828318 18678 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:20:11.865661 18770 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:20:11.866204 18657 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:11.865711 18678 raft_consensus.cc:3055] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:20:11.903926 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 76264.5us
I20260709 13:20:11.907182 12708 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:20:11.888474 18683 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:11.921630 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:11.906566 18678 raft_consensus.cc:740] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:11.899859 18734 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:11.922935 12778 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:20:11.893190 18722 consensus_queue.cc:1059] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:20:11.968365 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:11.918469 18726 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:11.984244 18711 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:11.986238 18711 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:11.988406 18711 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:11.990198 18784 log.cc:927] Time spent T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 13:20:11.991423 18768 log.cc:927] Time spent T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:20:11.994462 12704 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:11.970278 18441 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:11.997107 18441 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:12.001190 18788 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:12.001978 18788 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:12.004741 18788 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:12.009951 18678 consensus_queue.cc:260] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:12.011745 18678 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:20:12.021286 18778 log.cc:927] Time spent T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:20:12.026432 18739 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:11.973732 18734 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:11.981171 18736 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:12.032642 18736 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:11.985107 18771 log.cc:927] Time spent T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 13:20:12.057065 18734 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:12.065018 18744 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:12.065791 18744 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:12.068130 18744 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:20:12.087311 18772 log.cc:927] Time spent T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.434s	user 0.004s	sys 0.000s
I20260709 13:20:12.103058 18734 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:12.103749 18634 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:12.103696 18734 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:12.105645 18743 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:12.106959 13382 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:20:12.108132 18227 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:12.110693 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:12.111397 18739 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:12.112166 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:12.113166 18734 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:20:12.113476 18594 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:12.113482 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:12.111974 18739 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:12.114568 13378 raft_consensus.cc:1240] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 21. Current term is 22. Ops: []
I20260709 13:20:12.116415 18739 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:12.120885 12779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 3149) took 1144 ms. Trace:
I20260709 13:20:12.121393 12779 rpcz_store.cc:276] 0709 13:20:10.976577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:11.015128 (+ 38551us) service_pool.cc:224] Handling call
0709 13:20:11.057462 (+ 42334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:11.740448 (+682986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:12.120598 (+380150us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b54002cb854. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:20:12.120813 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1567616384}
I20260709 13:20:12.111431 13382 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:20:12.140122 18441 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:12.140031 13380 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
I20260709 13:20:12.144644 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:12.147027 13348 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:12.159180 18736 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:12.159828 18736 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:12.163831 18589 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:12.109019 18743 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:12.155710 18739 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:12.149966 18611 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:12.167428 18611 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:12.156044 18712 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:12.170205 18712 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:12.156992 18693 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:12.171624 18693 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:12.172502 18712 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:12.173699 18693 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:12.174537 12702 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 20. Current term is 21. Ops: [18.9-20.10]
I20260709 13:20:12.181156 18352 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:12.182274 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:12.184969 18751 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:12.186699 18673 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:12.187079 12598 leader_election.cc:400] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16.
I20260709 13:20:12.191985 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:12.196411 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:12.199038 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:12.199787 12598 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:12.201543 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:12.203980 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:12.205204 12778 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:12.209112 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:12.210245 12776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 802) took 1585 ms. Trace:
I20260709 13:20:12.213064 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:12.210652 12776 rpcz_store.cc:276] 0709 13:20:10.625319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.625456 (+   137us) service_pool.cc:224] Handling call
0709 13:20:11.589684 (+964228us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:11.590538 (+   854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:12.210045 (+619507us) spinlock_profiling.cc:243] Waited 936 ms on lock 0x7b54002cae50. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:20:12.210180 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2153847552}
I20260709 13:20:12.214161 12626 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:20:12.216361 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:12.220621 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:12.225378 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:12.225623 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:12.231570 12747 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:12.212064 18712 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:12.244122 12779 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:20:12.262391 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:12.238852 12597 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:12.265255 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:12.236618 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:12.233455 18767 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:12.287642 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:12.294633 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:12.294812 12709 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
I20260709 13:20:12.292815 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:12.295333 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:20:12.301175 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 64677.4us
I20260709 13:20:12.306761 12707 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:20:12.308761 18770 raft_consensus.cc:3055] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:12.309345 18770 raft_consensus.cc:740] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:12.310531 18770 consensus_queue.cc:260] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:12.315558 18316 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:12.319475 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:12.323141 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:12.331734 12629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:12.332445 12629 raft_consensus.cc:606] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:12.332839 12629 raft_consensus.cc:614] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:12.330556 18770 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:12.335963 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:12.335978 12630 raft_consensus.cc:1240] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:20:12.344399 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:12.353994 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:12.331166 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:12.346596 18540 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:12.355655 18540 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:12.358083 18540 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:12.285609 18639 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:12.350972 18743 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:12.358778 18639 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:12.359280 18639 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:12.359229 18743 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:12.362115 18743 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:12.240741 18657 consensus_queue.cc:1059] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.123s
W20260709 13:20:12.363983 18786 log.cc:927] Time spent T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.463s	user 0.004s	sys 0.000s
I20260709 13:20:12.246496 18475 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:20:12.366322 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:12.367926 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:12.353538 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:12.368969 12774 raft_consensus.cc:2393] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 17: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:12.234023 18611 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:12.381428 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 79262.8us
I20260709 13:20:12.352949 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:12.385789 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:12.392403 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:12.294600 18736 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:12.391842 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:12.409035 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:20:12.400259 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 197016us
I20260709 13:20:12.402563 12707 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:12.241976 18791 consensus_queue.cc:1059] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.137s
I20260709 13:20:12.235996 18751 consensus_queue.cc:1059] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.154s
I20260709 13:20:12.243383 18712 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:12.421587 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:12.422326 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:12.431562 12780 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:20:12.389780 18594 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:12.437196 12706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "866523bf2d2f442f971d504267ed200f"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:20:12.438198 12706 raft_consensus.cc:606] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:12.438613 12706 raft_consensus.cc:614] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:12.427249 13378 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:20:12.424314 12672 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:12.436043 12747 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:12.444128 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:12.444764 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:12.448604 12672 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:12.423771 18703 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:12.423415 18668 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:12.458604 17734 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:20:12.464360 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 49780.2us
I20260709 13:20:12.452260 18736 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:12.450155 12747 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:12.458441 18743 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:12.483291 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:12.442605 18385 consensus_queue.cc:1059] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:20:12.446883 18201 consensus_queue.cc:1059] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:20:12.464216 18703 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:12.463769 18668 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:12.512791 18657 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:12.491685 18227 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:12.468580 18787 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:20:12.490527 18743 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:12.538237 12778 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:20:12.519047 18710 raft_consensus.cc:3055] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:12.578811 12781 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:20:12.556591 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:12.584980 12706 raft_consensus.cc:2393] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 14: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:12.559513 12707 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:20:12.555917 12747 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
W20260709 13:20:12.544756 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 192488us
I20260709 13:20:12.540335 18201 consensus_queue.cc:1243] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [LEADER]: Peer 522fd2ba0e9f402691b5ef29af8579cd log is divergent from this leader: its last log entry 16.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:20:12.576308 18743 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:12.590022 12747 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:12.596272 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:20:12.539765 18794 log.cc:927] Time spent T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
W20260709 13:20:12.540661 13380 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51768 (request call id 976) took 1792 ms (client timeout 1809 ms). Trace:
I20260709 13:20:12.599721 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:12.542085 18639 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:12.600028 13380 rpcz_store.cc:269] 0709 13:20:10.741092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:10.741263 (+   171us) service_pool.cc:224] Handling call
0709 13:20:12.532936 (+1791673us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b540026cad4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:20:12.533080 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3172990208}
I20260709 13:20:12.578255 18710 raft_consensus.cc:740] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:12.601462 18482 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:12.606562 18744 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:12.593825 18352 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:20:12.609910 18799 log.cc:927] Time spent T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:20:12.611054 18703 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:12.616047 18726 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:12.627916 18726 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:12.607630 18227 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:12.547892 18657 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:12.622522 18744 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:12.634897 13348 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:20:12.635066 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 176910us
I20260709 13:20:12.642557 12672 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:12.647728 12672 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:12.632938 18767 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:12.653272 12629 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:20:12.642735 18744 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:12.655218 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 83140.4us
I20260709 13:20:12.655409 12672 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:12.649832 18316 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:12.664562 18657 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:12.651145 18711 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:12.667924 12672 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:12.652057 18594 raft_consensus.cc:697] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:12.673003 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:12.675091 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:12.675944 12706 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 14: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:12.676040 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:12.676530 12707 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:20:12.678340 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:12.684139 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:12.686066 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:12.687161 12707 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:12.695864 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:12.699133 18809 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:12.700580 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:12.701301 12777 raft_consensus.cc:2368] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18.
I20260709 13:20:12.701589 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:12.702086 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:12.679171 12632 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:20:12.660100 18385 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:12.665479 18657 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:12.662356 18352 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:12.721894 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:12.668979 18736 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:12.647094 18710 consensus_queue.cc:260] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:12.727541 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 125112us
I20260709 13:20:12.689621 18482 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:20:12.725382 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 135071us
I20260709 13:20:12.704056 18668 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:12.755213 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 54724.1us
W20260709 13:20:12.764122 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 129333us
I20260709 13:20:12.797051 13348 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:12.726171 18657 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:12.766285 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:12.806752 13348 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:12.808557 12775 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:12.808257 12598 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:12.726030 18385 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:12.807879 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 47635.6us
I20260709 13:20:12.756309 18734 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:20:12.815811 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 49678.2us
I20260709 13:20:12.802217 18710 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:12.820152 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:12.821213 12672 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:12.824014 18683 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:12.824410 18696 consensus_queue.cc:1059] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:12.825417 18696 raft_consensus.cc:3055] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:12.826128 18696 raft_consensus.cc:740] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:12.818728 18441 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:12.826925 18696 consensus_queue.cc:260] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:12.805455 18657 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:12.829895 12672 leader_election.cc:400] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18.
I20260709 13:20:12.830539 12672 leader_election.cc:403] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:12.804961 18639 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:12.761860 18482 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:12.735186 18352 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:12.833235 13349 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:12.802116 18791 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.018s
I20260709 13:20:12.821889 18820 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:12.816618 18363 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:12.813421 18726 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:12.922408 18363 raft_consensus.cc:697] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:12.834061 18357 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:12.923374 18363 consensus_queue.cc:237] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:12.800757 18750 consensus_queue.cc:1059] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
W20260709 13:20:12.925060 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 95305.2us
I20260709 13:20:12.921926 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:20:12.894238 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 61057.1us
I20260709 13:20:12.927733 12777 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
W20260709 13:20:12.929927 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 170894us
I20260709 13:20:12.926093 12672 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:12.818555 18809 raft_consensus.cc:697] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:12.929319 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:12.802474 18594 consensus_queue.cc:237] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:12.920643 18827 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:12.828723 18734 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:12.920809 18806 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:12.830348 18441 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.110s
W20260709 13:20:12.920382 18816 log.cc:927] Time spent T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:20:12.930625 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:12.954301 12708 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:49378
I20260709 13:20:12.973258 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:12.974148 12707 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:20:12.928483 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:12.968894 18744 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:12.975601 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 47345.2us
I20260709 13:20:12.975904 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:12.982802 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:12.983019 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:12.973932 12708 raft_consensus.cc:606] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Received request to transfer leadership
W20260709 13:20:12.933821 18819 log.cc:927] Time spent T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
W20260709 13:20:12.940449 18770 consensus_meta.cc:287] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.605s	user 0.012s	sys 0.003s
I20260709 13:20:12.984885 12707 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 19: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:13.024744 12779 raft_consensus.cc:1240] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:12.953833 18812 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:13.026974 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:12.978431 12632 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:20:12.950716 18385 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:13.027758 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:13.031152 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:13.029448 12704 raft_consensus.cc:2410] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 20 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:20:12.939368 18482 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:12.949381 18316 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:13.029918 18812 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:12.977931 18668 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:13.017635 18703 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:12.959563 18754 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 13:20:13.030081 18829 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:20:12.932235 18696 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:13.051117 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:12.957746 18743 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:13.055493 18812 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:12.954512 18809 consensus_queue.cc:237] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:13.066934 18744 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:13.070011 18747 raft_consensus.cc:3055] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:20:13.082180 18352 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:13.022709 18811 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:20:13.085723 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 262104us
I20260709 13:20:12.955921 18657 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:13.053642 18835 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:13.087815 13381 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:20:13.027026 18736 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:13.094197 18441 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:13.087719 18787 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.088248 18703 raft_consensus.cc:697] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:13.100548 12775 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:13.096372 18747 raft_consensus.cc:740] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:13.119535 18712 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:13.120307 18747 consensus_queue.cc:260] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:13.121330 18747 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:13.125373 18787 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:13.101050 18835 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:13.128116 18835 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:13.129796 18787 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:13.130329 18693 raft_consensus.cc:3055] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:13.119454 18736 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:13.050731 18316 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:13.131349 18811 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:13.126947 18441 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:13.109885 12626 raft_consensus.cc:3055] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:13.138473 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:13.142437 12775 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:20:13.146492 12672 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:13.148488 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:13.151171 12626 raft_consensus.cc:740] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:13.110265 18657 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:13.157713 12632 raft_consensus.cc:2393] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 17: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:20:13.176319 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:13.186167 13380 raft_consensus.cc:2376] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20. Re-sending same reply.
I20260709 13:20:13.185231 12626 consensus_queue.cc:260] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:13.143956 18693 raft_consensus.cc:740] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:13.191939 12626 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:13.194270 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:13.201725 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:13.202631 13377 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:13.151183 18579 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:13.208307 18441 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:13.209597 18743 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:13.153832 18712 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:20:13.217768 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.160357 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:13.219491 13382 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:20:13.219213 18693 consensus_queue.cc:260] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:13.220695 18693 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:13.221809 12627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:13.222397 12627 raft_consensus.cc:606] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:13.222708 12627 raft_consensus.cc:614] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:13.217690 18827 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18.
I20260709 13:20:13.224164 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:13.238641 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:13.239122 13380 raft_consensus.cc:2393] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 17: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:20:13.224431 13382 raft_consensus.cc:1240] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 21. Current term is 22. Ops: []
I20260709 13:20:13.225245 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:13.233253 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:13.243131 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:13.194955 12746 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:13.254967 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:13.262952 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 68104us
I20260709 13:20:13.213811 18712 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:13.266279 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:13.152158 18703 consensus_queue.cc:237] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:13.270903 17734 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.272445 13381 raft_consensus.cc:2368] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 16. Current term is 17.
I20260709 13:20:13.271504 17734 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:13.215113 18579 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:13.292204 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 242274us
I20260709 13:20:13.268500 18712 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:13.293946 12746 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:20:13.310258 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.271602 18730 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:13.264947 18827 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:13.320013 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:13.326560 12783 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:20:13.320013 12672 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:13.322271 12781 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:20:13.322923 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:13.323412 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:13.330703 12746 leader_election.cc:400] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 16. Current term is 17.
I20260709 13:20:13.331174 12746 leader_election.cc:403] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:20:13.267166 18815 consensus_queue.cc:1059] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:20:13.333644 17734 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:13.333925 12672 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:13.342748 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:20:13.315737 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 52426.2us
I20260709 13:20:13.344839 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:13.293507 12747 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:13.345469 12777 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
W20260709 13:20:13.346086 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 52664us
I20260709 13:20:13.347199 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:13.334141 18825 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:13.359426 12703 raft_consensus.cc:1240] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:13.333099 18730 raft_consensus.cc:697] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:13.362310 17734 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:13.363334 17734 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:13.349390 18787 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:13.355191 18594 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:13.361757 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:13.369472 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:13.332918 18827 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:13.359972 18825 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:13.377882 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:13.360406 18678 raft_consensus.cc:3055] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:13.382833 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:13.368084 18787 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:13.384197 13348 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:13.374843 18594 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:13.386154 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:13.395982 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:13.376751 18815 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:13.381376 18825 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:13.379411 18678 raft_consensus.cc:740] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:13.381332 18227 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 16. Current term is 17.
I20260709 13:20:13.386293 18787 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:13.400039 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:13.406962 13377 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:20:13.385489 18316 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:13.396184 18594 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:13.410727 18725 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:13.398164 18839 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:13.397804 18834 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:20:13.446170 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 95131.7us
I20260709 13:20:13.447641 12626 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:20:13.448823 12626 pending_rounds.cc:85] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 11
I20260709 13:20:13.449100 12626 pending_rounds.cc:107] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:20:13.449767 12626 raft_consensus.cc:2889] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:13.450519 12626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 1107) took 1099 ms. Trace:
W20260709 13:20:13.452118 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 125307us
I20260709 13:20:13.450845 12626 rpcz_store.cc:276] 0709 13:20:12.350991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:12.351521 (+   530us) service_pool.cc:224] Handling call
0709 13:20:12.351872 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:13.109593 (+757721us) spinlock_profiling.cc:243] Waited 91.6 ms on lock 0x7b54002781a0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d329dc0 00007f4c6d3b41c0 00007f4c6d3b5231 00007f4c6d3b1d8f 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c
0709 13:20:13.450279 (+340686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:13.450465 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":210764544}
I20260709 13:20:13.431109 18839 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:13.457384 12626 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:13.458658 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:13.458786 12626 raft_consensus.cc:606] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:13.459395 12626 raft_consensus.cc:614] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:13.445977 18811 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:13.444891 18579 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:13.455749 18831 log.cc:927] Time spent T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
W20260709 13:20:13.455353 18785 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:20:13.469097 18848 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:13.472434 17734 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:13.406351 18730 consensus_queue.cc:237] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:13.480388 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:20:13.490268 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 94219.1us
I20260709 13:20:13.462045 18839 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:13.491503 12746 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:13.448597 18834 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:13.494639 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:13.495523 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:13.447324 18678 consensus_queue.cc:260] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:13.495750 13348 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:13.486414 18848 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:13.502604 13348 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:13.503561 18678 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:13.504562 13348 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:13.498231 18227 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:20:13.515239 18841 log.cc:927] Time spent T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:20:13.516489 18850 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:13.517222 18850 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:13.521051 18227 raft_consensus.cc:697] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:13.522585 18227 consensus_queue.cc:237] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:13.507701 18845 log.cc:927] Time spent T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:20:13.507404 18811 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:13.501264 18848 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:13.493832 18825 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:13.612478 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:13.612584 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:13.625802 12598 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:13.629722 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:13.614359 18747 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:13.641727 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 137234us
I20260709 13:20:13.640576 12632 raft_consensus.cc:2368] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 16. Current term is 17.
I20260709 13:20:13.610042 18815 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:13.643061 18596 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:20:13.663324 12747 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:13.664911 12706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc7827272b424bad88349b6b7437844b"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:20:13.665529 18703 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:13.667647 12747 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:13.669489 18703 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:13.671092 12747 leader_election.cc:400] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 16. Current term is 17.
I20260709 13:20:13.671945 18834 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:13.676681 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:13.678380 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:13.679845 12746 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:13.680891 17734 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:13.681380 17734 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:13.665589 12706 raft_consensus.cc:606] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:13.683641 12706 raft_consensus.cc:614] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:13.688987 13349 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:13.647656 18815 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:13.645577 18747 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:13.693336 12706 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:13.671057 18736 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:13.690248 18540 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.686480 18744 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:13.698326 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:20:13.652385 18594 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:13.706638 12701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 2613) took 1022 ms. Trace:
W20260709 13:20:13.707336 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.724072 12701 rpcz_store.cc:276] 0709 13:20:12.684404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:12.684522 (+   118us) service_pool.cc:224] Handling call
0709 13:20:12.684775 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:13.705766 (+1020991us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:13.705798 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:13.705838 (+    40us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:13.705865 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:13.706421 (+   556us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54001090d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:20:13.706464 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:13.706583 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2347563904}
I20260709 13:20:13.726838 18739 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:13.719272 18540 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:13.728556 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:13.729053 18815 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:13.729915 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:13.730415 18540 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:13.711679 17734 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:13.712841 12597 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:13.701593 18825 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:13.732470 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 49521.4us
I20260709 13:20:13.734601 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:13.729254 12627 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
I20260709 13:20:13.737072 12625 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:20:13.737582 12625 pending_rounds.cc:85] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 9
I20260709 13:20:13.737841 12625 pending_rounds.cc:107] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:20:13.738148 12625 raft_consensus.cc:2889] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:13.753710 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:20:13.810314 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 63398.1us
I20260709 13:20:13.690135 18837 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:20:13.812570 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 77122.3us
I20260709 13:20:13.813772 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:13.732609 18696 consensus_meta.cc:287] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.643s	user 0.009s	sys 0.002s
W20260709 13:20:13.818717 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:13.820947 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:13.823331 17734 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:20:13.825783 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 138354us
I20260709 13:20:13.811817 18596 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:13.827387 17734 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:13.813724 18734 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:13.844487 12673 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:13.846244 12673 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:13.846830 18711 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:13.848438 18711 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:13.846033 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:13.849874 12673 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:13.850054 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:13.850633 18363 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:13.851469 12673 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:13.852238 18363 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:13.852685 18363 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:13.853041 18363 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:13.853986 18711 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:13.854300 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:13.855300 18747 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:13.857236 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:13.858775 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:13.859092 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:13.859588 12705 raft_consensus.cc:1813] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Handling vote request from an unknown peer 19cd864c053744bebd96093a7982537e
I20260709 13:20:13.860080 12705 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 14: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:13.860647 18596 raft_consensus.cc:697] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:13.861740 18596 consensus_queue.cc:237] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:13.871717 18363 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:13.873906 18363 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:13.874380 18712 consensus_meta.cc:287] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.546s	user 0.010s	sys 0.001s
I20260709 13:20:13.874786 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:13.874871 18712 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:13.875530 12701 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 21.
W20260709 13:20:13.876112 18852 log.cc:927] Time spent T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.260s	user 0.003s	sys 0.000s
I20260709 13:20:13.876319 12709 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:13.877238 18712 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:13.884248 18475 consensus_queue.cc:1059] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:20:13.895601 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:13.896217 12626 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:13.819862 18837 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:13.884608 12702 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:20:13.883780 18787 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:13.879150 18594 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:13.886284 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:13.873442 12708 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:20:13.923446 12709 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:13.904562 18441 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.905726 17734 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:13.907461 18227 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:13.908632 18744 raft_consensus.cc:3055] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:13.906554 12701 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:49378
I20260709 13:20:13.933344 12701 raft_consensus.cc:606] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Received request to transfer leadership
I20260709 13:20:13.914674 12626 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 20.
I20260709 13:20:13.916484 18827 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:13.934711 12598 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:13.925828 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:13.937839 18837 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:13.940690 18848 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:13.934757 12673 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:20:13.943348 12517 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 981fb2f435e24d06a81fc02cf4591cc5 already in progress [suppressed 11 similar messages]
I20260709 13:20:13.947248 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:13.948383 12704 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:13.929462 12708 raft_consensus.cc:3055] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:13.932269 18441 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:13.902288 18835 consensus_meta.cc:287] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.774s	user 0.011s	sys 0.001s
I20260709 13:20:13.954195 13382 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:13.954967 13378 raft_consensus.cc:2393] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 20: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:13.950268 12708 raft_consensus.cc:740] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:13.952198 12598 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:13.967446 12776 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:20:13.968838 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:13.963838 12708 consensus_queue.cc:260] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:13.933126 18744 raft_consensus.cc:740] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:13.976971 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:20:13.962576 18828 log.cc:927] Time spent T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:20:13.974900 13383 raft_consensus.cc:1813] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Handling vote request from an unknown peer 19cd864c053744bebd96093a7982537e
I20260709 13:20:13.980543 13383 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 14: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:20:13.970829 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:13.974087 12708 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:13.959857 18201 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:13.979254 18859 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:13.979841 18669 consensus_queue.cc:1059] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:20:13.988749 18859 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:13.990985 18859 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:13.992313 12672 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:13.992633 18848 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:13.962507 18862 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:13.994117 18862 raft_consensus.cc:697] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 20 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:13.995061 18363 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:13.996780 12703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 2619) took 1310 ms. Trace:
I20260709 13:20:13.997140 12703 rpcz_store.cc:276] 0709 13:20:12.686803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:12.687914 (+  1111us) service_pool.cc:224] Handling call
0709 13:20:12.722330 (+ 34416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:13.360027 (+637697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:13.996538 (+636511us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b54001d8b54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:20:13.996699 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1460518656}
I20260709 13:20:13.997967 18744 consensus_queue.cc:260] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:13.998550 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:13.999394 18744 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:13.969790 18785 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:14.001551 18785 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:14.001935 18785 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:14.002390 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:14.004500 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:14.005162 12632 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 20.
I20260709 13:20:14.006873 18787 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:14.007405 12673 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:13.958256 18839 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:14.008859 18787 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:14.009238 18787 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:14.009588 18839 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:13.966018 18835 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:14.009610 18787 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:14.010726 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:14.013298 12783 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:20:14.014703 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:13.952093 18540 consensus_queue.cc:1059] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:20:14.019466 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:20:14.019393 12517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 74336.4us
I20260709 13:20:14.020438 12783 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:14.020478 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:14.021358 12774 raft_consensus.cc:1813] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Handling vote request from an unknown peer 19cd864c053744bebd96093a7982537e
I20260709 13:20:13.995087 18862 consensus_queue.cc:237] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:14.022505 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:14.028849 18597 raft_consensus.cc:3055] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:14.030346 18597 raft_consensus.cc:740] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:14.031291 18597 consensus_queue.cc:260] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:14.031761 12706 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:20:14.032517 18597 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:14.034845 18865 log.cc:927] Time spent T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:20:14.039866 12774 raft_consensus.cc:2368] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15.
I20260709 13:20:14.040596 12709 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:20:14.040694 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:14.032781 18751 raft_consensus.cc:3055] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:14.050462 12777 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:14.057111 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:14.028939 18540 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:20:14.049782 18751 raft_consensus.cc:740] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:14.085461 12783 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 20.
I20260709 13:20:14.082293 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:14.088760 12777 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
I20260709 13:20:14.094530 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:14.103444 12672 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:14.100690 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:14.105098 12774 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:14.108117 12672 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:14.118305 18441 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:14.122476 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:14.122560 12700 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:49378
I20260709 13:20:14.123241 12700 raft_consensus.cc:606] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Received request to transfer leadership
I20260709 13:20:14.095897 18835 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:14.128285 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:14.131660 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:14.136050 12708 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 13:20:14.136600 12708 pending_rounds.cc:85] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 13
I20260709 13:20:14.136866 12708 pending_rounds.cc:107] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:20:14.137148 12708 raft_consensus.cc:2889] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:14.164153 18873 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.144838 18736 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:14.150058 18827 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:14.149369 18363 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.144450 18785 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:14.175957 18785 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:14.180883 17734 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.181957 18201 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:14.182570 18201 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:14.182987 18363 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:14.184769 18201 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:14.185110 18363 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:14.181502 17734 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:14.167595 18862 raft_consensus.cc:993] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530: : Instructing follower 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:20:14.196338 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:14.204043 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:14.192994 12774 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
I20260709 13:20:14.204638 12629 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:20:14.189939 17734 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:14.206485 12673 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:14.191017 12672 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:14.209321 18363 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:14.209759 18363 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:14.210101 18363 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:14.216511 12780 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:20:14.159812 18734 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:20:14.221693 17734 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:20:14.149077 18751 consensus_queue.cc:260] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:14.219375 18862 raft_consensus.cc:1081] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Signalling peer 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:20:14.219630 18589 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:14.170763 18873 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:14.225698 18589 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:14.227402 18873 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:14.227898 18589 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:14.232797 12627 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:20:14.234151 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:14.235704 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:14.236416 12631 raft_consensus.cc:2368] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 19. Current term is 20.
I20260709 13:20:14.222340 12672 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:14.238974 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:14.239295 18862 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:14.241489 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:14.242019 12779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 2890) took 1426 ms. Trace:
I20260709 13:20:14.242735 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:14.243582 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:14.244550 18668 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:14.247025 12672 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:14.248435 18693 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:14.245098 18668 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:14.250622 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:14.251277 18668 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:14.254719 12747 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:20:14.256191 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 104380us
I20260709 13:20:14.242375 12779 rpcz_store.cc:276] 0709 13:20:12.815957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:13.031466 (+215509us) service_pool.cc:224] Handling call
0709 13:20:13.090047 (+ 58581us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:13.090269 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:13.090307 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:13.090361 (+    54us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:13.090404 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:13.090473 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:14.241863 (+1151390us) spinlock_profiling.cc:243] Waited 40.5 ms on lock 0x7b540044c0d0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:20:14.241968 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93253888}
I20260709 13:20:14.257115 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:14.260654 12626 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:20:14.264405 12597 leader_election.cc:400] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15.
I20260709 13:20:14.264750 12597 leader_election.cc:403] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:14.181995 18597 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:14.267777 12672 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:14.269461 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:14.276381 17734 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:14.277161 17734 raft_consensus.cc:697] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:14.279170 18785 consensus_queue.cc:1059] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:14.280087 18837 raft_consensus.cc:3055] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:14.280555 18837 raft_consensus.cc:740] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:14.282107 12703 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:20:14.281841 18837 consensus_queue.cc:260] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:14.287397 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:14.288048 12774 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:20:14.288417 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:14.289144 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:14.289950 12701 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 18.
I20260709 13:20:14.293018 13349 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:14.293160 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:14.294066 12704 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:14.198014 18827 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 13:20:14.296967 17734 consensus_queue.cc:237] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:14.301993 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 47336.8us
I20260709 13:20:14.190380 18787 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:14.304244 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:14.305838 12774 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:20:14.261701 18862 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:14.307777 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:14.266996 18441 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:14.269938 18668 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:14.310124 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:14.313766 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:14.310500 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:14.291772 18752 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:20:14.311108 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:14.319697 18829 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:14.339468 18829 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:14.339921 18829 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:14.312507 18835 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:20:14.328186 18821 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.314023 18810 consensus_queue.cc:1059] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:20:14.346454 18821 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:14.348608 18821 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:14.272622 18597 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:14.350904 18597 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:14.354604 18829 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:14.283396 18837 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:14.308037 12672 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:20:14.368945 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 61013.3us
I20260709 13:20:14.315018 18839 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:14.376482 18839 raft_consensus.cc:697] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:14.377578 18839 consensus_queue.cc:237] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:14.374063 18722 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:14.375805 18441 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.310592 12782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:36478
I20260709 13:20:14.390883 12782 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:14.391395 12782 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:14.402932 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:14.403749 12700 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:20:14.413651 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:14.419678 12704 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 22.
I20260709 13:20:14.420688 12705 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:20:14.415426 12782 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:14.401060 18441 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:14.440572 18441 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:14.410012 18834 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:20:14.470423 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:14.471771 12633 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:20:14.472831 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 45789.2us
I20260709 13:20:14.434028 18882 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:14.474048 12782 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:14.466414 18743 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.481896 18834 consensus_queue.cc:1243] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 20.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:20:14.489091 12629 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:20:14.464440 18363 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:14.490461 18787 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:14.460945 18829 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:14.500069 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 166424us
I20260709 13:20:14.501427 12747 leader_election.cc:400] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 19. Current term is 20.
I20260709 13:20:14.508335 12747 leader_election.cc:403] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:20:14.509999 13378 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:20:14.506392 18743 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:14.512032 18834 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 19. Current term is 20.
I20260709 13:20:14.514912 18743 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:14.517239 18363 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:14.517869 18751 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:14.521492 18883 raft_consensus.cc:3055] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:20:14.522328 12709 raft_consensus.cc:2368] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15.
I20260709 13:20:14.522049 18883 raft_consensus.cc:740] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:14.505786 18752 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15.
I20260709 13:20:14.533453 12597 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:14.534646 18856 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:14.534080 18883 consensus_queue.cc:260] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:14.542503 18883 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:14.512933 13378 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:20:14.549418 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:14.550068 12629 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:14.551203 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:20:14.551491 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 140117us
I20260709 13:20:14.554003 12633 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 21.
I20260709 13:20:14.554950 12672 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:14.509532 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:14.556284 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:20:14.556799 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 47473us
I20260709 13:20:14.560653 12708 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:20:14.563042 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:14.563755 12628 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:14.564106 12631 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:20:14.564584 12631 pending_rounds.cc:85] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 9
I20260709 13:20:14.555755 12673 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:14.565006 12631 pending_rounds.cc:107] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:20:14.565294 12631 raft_consensus.cc:2889] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:14.564589 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:14.566797 12779 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:20:14.566833 12708 raft_consensus.cc:2410] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:20:14.572571 12598 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:14.578014 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:14.575016 12597 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:14.585675 12775 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:14.586401 12629 raft_consensus.cc:2468] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:20:14.581614 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:14.575975 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:14.575539 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:14.583792 12781 raft_consensus.cc:1240] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:14.574215 12598 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:14.578238 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:14.563122 18827 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:14.618429 12709 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 20.
I20260709 13:20:14.618777 12778 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:20:14.581333 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:14.623096 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:14.626255 18785 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:14.627039 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:14.628096 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:14.630162 13380 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:20:14.627599 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 52669.4us
I20260709 13:20:14.631283 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:14.588214 12747 leader_election.cc:400] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15.
I20260709 13:20:14.569092 18839 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:14.631922 12781 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:20:14.638110 13378 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:14.546458 18722 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
W20260709 13:20:14.641230 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 51931us
I20260709 13:20:14.640717 13383 raft_consensus.cc:2393] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 19: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
W20260709 13:20:14.641580 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 53550.8us
I20260709 13:20:14.645370 18743 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:14.633268 18877 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:14.666766 18877 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:14.669034 18877 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:14.636759 12704 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:20:14.672739 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 98703.7us
W20260709 13:20:14.673884 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:14.650425 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:14.574039 18787 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:14.571103 18791 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.636490 12673 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:14.637451 18825 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:14.651957 13348 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:14.640188 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:14.691267 12709 raft_consensus.cc:3055] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:14.691749 12709 raft_consensus.cc:740] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:14.583863 18696 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:14.693194 12704 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 19.
I20260709 13:20:14.694459 12709 consensus_queue.cc:260] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:14.695968 18696 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:14.696748 18696 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:14.705734 18693 consensus_queue.cc:1059] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:14.706001 18862 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:14.706912 18696 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:14.710351 18696 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:14.710820 18825 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:14.682359 18834 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:20:14.713207 18825 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:14.713876 12775 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 19.
I20260709 13:20:14.717795 12632 raft_consensus.cc:2410] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:20:14.726233 18597 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:14.679466 18750 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:20:14.729677 18750 raft_consensus.cc:697] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 22 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:14.730894 18750 consensus_queue.cc:237] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:14.732940 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:14.677161 18827 raft_consensus.cc:697] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:14.735648 18827 consensus_queue.cc:237] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:14.737773 18877 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:20:14.738761 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 47093.1us
I20260709 13:20:14.741099 18791 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:14.743682 18791 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:14.746966 13349 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:14.747596 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:14.750223 18754 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:14.765975 18754 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:14.769166 12708 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 19.
I20260709 13:20:14.750224 18839 raft_consensus.cc:697] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:14.771361 18839 consensus_queue.cc:237] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:14.768729 18754 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:14.755761 13349 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:14.777153 18877 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:14.777902 18877 raft_consensus.cc:697] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 19 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:14.779104 18877 consensus_queue.cc:237] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:14.783898 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:14.775648 12628 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:20:14.788514 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:14.789369 13380 raft_consensus.cc:2393] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 22: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
W20260709 13:20:14.795961 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 159529us
I20260709 13:20:14.800904 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:14.801901 12629 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 22.
I20260709 13:20:14.807431 12629 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:20:14.808084 12629 pending_rounds.cc:85] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 10
I20260709 13:20:14.808459 12629 pending_rounds.cc:107] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:20:14.808856 12629 raft_consensus.cc:2889] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:14.809963 18874 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:20:14.811038 18754 raft_consensus.cc:2804] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:14.812374 18754 raft_consensus.cc:697] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:14.815553 12709 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:14.815788 18754 consensus_queue.cc:237] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:14.826025 13378 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:20:14.826990 13377 raft_consensus.cc:1240] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:14.829530 12778 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 21.
I20260709 13:20:14.834951 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:20:14.836968 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 43865.4us
I20260709 13:20:14.837781 12709 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:20:14.835592 13385 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 22.
I20260709 13:20:14.842677 12673 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:14.844657 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:14.849397 17734 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:14.853700 18815 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:14.845799 18743 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:20:14.857040 18482 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:14.861924 12597 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:14.863987 18752 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:14.864398 18752 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:14.864741 18752 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:14.849385 12673 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:20:14.868927 12579 auto_rebalancer.cc:893] Removing replace marker failed: Replica 639ab2db265f4747bb7afb93e833f530 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:20:14.869402 12579 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 981fb2f435e24d06a81fc02cf4591cc5 already in progress
W20260709 13:20:14.869720 12579 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 981fb2f435e24d06a81fc02cf4591cc5 already in progress
I20260709 13:20:14.872161 12775 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:20:14.876083 18668 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:14.876556 18668 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:14.876963 18668 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:14.861294 12747 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:14.878841 18589 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:14.882267 13349 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:14.882913 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:14.883471 12747 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:14.884007 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:14.884281 18813 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:14.884570 13385 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:20:14.885540 12747 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:14.885718 18851 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:14.886092 18851 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:14.886230 18859 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:14.886869 18734 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:14.887702 18482 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:14.886482 18851 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:14.888258 18482 raft_consensus.cc:697] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:14.888576 18734 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:14.889075 18734 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:14.891183 12632 raft_consensus.cc:1240] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 12. Current term is 13. Ops: []
W20260709 13:20:14.892063 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 211728us
I20260709 13:20:14.883574 12778 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
I20260709 13:20:14.845263 18787 raft_consensus.cc:697] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 21 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:14.898312 18591 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:14.899290 18787 consensus_queue.cc:237] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:14.896549 12526 catalog_manager.cc:5697] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 6 to 11, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 11 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:14.910427 18752 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:14.924427 18752 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:14.889340 18482 consensus_queue.cc:237] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:14.917268 12709 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:14.931200 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:14.932077 12774 raft_consensus.cc:2393] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 19: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:14.935236 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:20:14.950911 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 44724.6us
I20260709 13:20:14.956789 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:14.957576 13380 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:14.950492 18722 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:14.986318 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:14.994486 18722 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:14.994999 18722 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:15.004036 13380 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 23.
I20260709 13:20:14.953220 18883 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:20:15.027976 18851 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:15.037873 18668 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:15.040531 18668 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:15.049932 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:15.053225 12705 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 22.
I20260709 13:20:15.081676 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:15.082576 12709 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:20:15.089829 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 113438us
I20260709 13:20:15.126770 12598 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:15.134070 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:15.134887 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:15.135686 12634 raft_consensus.cc:2368] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 22. Current term is 23.
I20260709 13:20:15.136760 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:20:15.138626 12525 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54792 (request call id 229) took 23497 ms (client timeout 15000 ms). Trace:
I20260709 13:20:15.142419 13382 raft_consensus.cc:2368] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 22. Current term is 23.
I20260709 13:20:15.150571 12709 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 23.
I20260709 13:20:15.082178 18851 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:15.161705 12672 leader_election.cc:400] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 22. Current term is 23.
I20260709 13:20:15.162961 12672 leader_election.cc:403] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:20:15.143095 18750 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:15.168154 12673 leader_election.cc:400] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 22. Current term is 23.
I20260709 13:20:15.141855 18734 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:15.191846 12598 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:15.176903 18668 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 22. Current term is 23.
W20260709 13:20:15.220206 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 185829us
I20260709 13:20:15.212383 18853 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:20:15.139168 12525 rpcz_store.cc:269] 0709 13:19:51.641428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:51.641815 (+   387us) service_pool.cc:224] Handling call
0709 13:19:52.459539 (+817724us) spinlock_profiling.cc:243] Waited 3.4 ms on lock 0x7b5400002210. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a28de9c 00007f4c6a28e0f6 00007f4c73207a89 00007f4c7319fda6 00007f4c73171088 00007f4c7326f37e 00007f4c717302a4 00007f4c71730232 00007f4c71730161 00007f4c717300dc
0709 13:20:15.138375 (+22678836us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:07.632848 (+     0us) spinlock_profiling.cc:243] Waited 55.3 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6a2a80ac 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c2455 00007f4c70de716c
0709 13:20:07.758726 (+125878us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:07.759679 (+   953us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:07.759740 (+    61us) write_op.cc:435] Acquired schema lock
0709 13:20:07.759785 (+    45us) tablet.cc:662] Decoding operations
0709 13:20:07.763256 (+  3471us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:07.763355 (+    99us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:07.763390 (+    35us) tablet.cc:685] Acquiring locks for 32 operations
0709 13:20:07.801937 (+ 38547us) tablet.cc:701] Row locks acquired
0709 13:20:07.801986 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:20:07.802175 (+   189us) write_op.cc:270] Start()
0709 13:20:07.802399 (+   224us) write_op.cc:276] Timestamp: P: 1783603207802114 usec, L: 0
0709 13:20:07.802438 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 13:20:07.803278 (+   840us) log.cc:844] Serialized 12467 byte log entry
0709 13:20:08.028492 (+225214us) spinlock_profiling.cc:243] Waited 5.18 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:20:08.232578 (+204086us) op_driver.cc:464] REPLICATION: finished
0709 13:20:08.344531 (+111953us) spinlock_profiling.cc:243] Waited 27.2 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6a2a80ac 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c70de8ef0
0709 13:20:08.610919 (+266388us) write_op.cc:301] APPLY: starting
0709 13:20:08.624199 (+ 13280us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:10.467706 (+1843507us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:10.467754 (+    48us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:12.042728 (+1574974us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:12.043473 (+   745us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=64,key_file_lookups=64,delta_file_lookups=64,mrs_lookups=0
0709 13:20:12.043567 (+    94us) write_op.cc:312] APPLY: finished
0709 13:20:12.044534 (+   967us) log.cc:844] Serialized 276 byte log entry
0709 13:20:12.247014 (+202480us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:12.361648 (+114634us) write_op.cc:454] Released schema lock
0709 13:20:12.361797 (+   149us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:12.362521 (+   724us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:12.362559 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":138209,"reported_tablets":44,"spinlock_wait_cycles":7808896,"child_traces":[["op",{"apply.queue_time_us":335986,"apply.run_cpu_time_us":88077,"apply.run_wall_time_us":3766089,"cfile_cache_hit":130,"cfile_cache_hit_bytes":138015,"num_ops":32,"prepare.queue_time_us":215126,"prepare.run_cpu_time_us":13835,"prepare.run_wall_time_us":76006,"raft.queue_time_us":181571,"raft.run_cpu_time_us":49506,"raft.run_wall_time_us":265386,"replication_time_us":421563,"spinlock_wait_cycles":201680128,"thread_start_us":718277,"threads_started":5,"wal-append.queue_time_us":604172,"wal-append.run_cpu_time_us":70937,"wal-append.run_wall_time_us":2237234}]]}
I20260709 13:20:15.245841 12634 raft_consensus.cc:1240] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:15.253989 12633 raft_consensus.cc:1240] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:15.238166 18853 raft_consensus.cc:697] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 23 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:15.296501 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:15.297298 12774 raft_consensus.cc:2393] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 19: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:15.300562 12705 raft_consensus.cc:1240] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:15.308960 12780 raft_consensus.cc:1275] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
W20260709 13:20:15.311445 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 149891us
W20260709 13:20:15.309036 18892 log.cc:927] Time spent T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.140s	user 0.001s	sys 0.001s
W20260709 13:20:15.359140 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 186917us
I20260709 13:20:15.334445 18874 consensus_queue.cc:1048] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:20:15.337141 18589 consensus_queue.cc:1059] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:20:15.335276 18813 consensus_queue.cc:1059] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:20:15.377111 12633 raft_consensus.cc:1275] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 13:20:15.374846 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 52384.7us
I20260709 13:20:15.318791 18853 consensus_queue.cc:237] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:15.364001 18734 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:15.378791 18874 consensus_queue.cc:1243] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [LEADER]: Peer 3d539f8d95d343d8b1696803f18b30d2 log is divergent from this leader: its last log entry 21.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:20:15.383150 18907 log.cc:927] Time spent T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:20:15.345696 18722 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:15.381951 18750 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:15.363377 12526 catalog_manager.cc:5697] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 9 to 10, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 10 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:15.408818 12526 catalog_manager.cc:5697] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 10 to 14, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 14 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:15.414688 18827 consensus_queue.cc:1048] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:15.417692 12708 raft_consensus.cc:2368] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 19. Current term is 20.
I20260709 13:20:15.401989 18750 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:15.393240 18914 raft_consensus.cc:3055] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:20:15.401310 18905 log.cc:927] Time spent T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:20:15.438426 13383 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:15.439436 13380 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:15.431138 18750 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:15.395120 18906 raft_consensus.cc:3055] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:20:15.424172 18897 log.cc:927] Time spent T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
W20260709 13:20:15.416033 18899 log.cc:927] Time spent T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:20:15.465101 13380 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:15.458298 18906 raft_consensus.cc:740] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:15.428416 18914 raft_consensus.cc:740] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:15.484354 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:15.486279 18906 consensus_queue.cc:260] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:15.487661 18906 raft_consensus.cc:3060] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:15.489285 12775 raft_consensus.cc:1217] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Deduplicated request from leader. Original: 17.8->[18.9-22.11]   Dedup: 18.9->[20.10-22.11]
I20260709 13:20:15.489811 12775 pending_rounds.cc:85] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 9
I20260709 13:20:15.490049 12775 pending_rounds.cc:107] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 21.10
I20260709 13:20:15.490281 12775 raft_consensus.cc:2889] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:20:15.503516 18898 log.cc:927] Time spent T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.216s	user 0.000s	sys 0.003s
I20260709 13:20:15.509902 18722 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:15.511685 18821 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:20:15.513913 18900 log.cc:927] Time spent T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.228s	user 0.002s	sys 0.000s
I20260709 13:20:15.515362 18914 consensus_queue.cc:260] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:15.511580 18730 consensus_queue.cc:1059] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:15.526919 13349 leader_election.cc:400] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 19. Current term is 20.
I20260709 13:20:15.527470 13349 leader_election.cc:403] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:20:15.529999 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:15.533887 12774 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:15.539834 13382 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:15.541905 12706 raft_consensus.cc:1275] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:20:15.546882 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:15.547474 12707 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:20:15.549520 12782 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:20:15.550601 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:15.557082 12598 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:15.562412 12774 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 20.
I20260709 13:20:15.560807 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:15.550697 18833 raft_consensus.cc:3055] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:20:15.549602 18441 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:15.563884 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:15.575793 18914 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:15.596612 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:15.602496 13348 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:15.562340 18851 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:20:15.579247 18833 raft_consensus.cc:740] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:15.586283 18597 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:15.584131 18888 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:15.573812 18856 consensus_queue.cc:1048] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:20:15.609486 12709 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:20:15.607883 18901 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:20:15.614109 18597 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:15.673285 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 69646.3us
I20260709 13:20:15.674142 18597 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:15.629444 18888 raft_consensus.cc:697] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 20 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:15.689002 18833 consensus_queue.cc:260] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:15.693612 13382 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:20:15.698828 12629 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:20:15.702790 18597 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:15.705971 12707 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:20:15.680631 18877 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
W20260709 13:20:15.757516 18903 log.cc:927] Time spent T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.463s	user 0.000s	sys 0.003s
I20260709 13:20:15.758162 18924 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:15.758708 18924 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:15.761152 18924 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:15.773046 12780 raft_consensus.cc:1275] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:20:15.773615 12780 pending_rounds.cc:85] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 7
I20260709 13:20:15.773921 12780 pending_rounds.cc:107] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:20:15.774196 12780 raft_consensus.cc:2889] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:15.776646 18833 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:15.725320 18934 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:20:15.780493 18883 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:20:15.805084 18597 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:15.767930 18888 consensus_queue.cc:237] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:15.813284 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:15.813970 12700 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:15.815023 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:15.815666 12703 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:20:15.831465 12629 raft_consensus.cc:1275] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
W20260709 13:20:15.836145 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 204255us
I20260709 13:20:15.853216 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:15.837216 18482 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:15.855964 12700 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 20.
W20260709 13:20:15.918092 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 58112.5us
W20260709 13:20:15.917887 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 60512.9us
W20260709 13:20:15.926616 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 93919.3us
I20260709 13:20:15.988312 17734 consensus_queue.cc:1048] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:20:15.992606 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 73463.2us
W20260709 13:20:15.993923 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 75852.9us
I20260709 13:20:15.994036 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:15.860128 18824 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:15.924420 18482 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:16.025408 18482 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:16.026111 12780 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:20:16.011663 18825 consensus_queue.cc:1048] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:20:16.035538 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:16.049847 12598 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:16.055354 12598 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:20:16.062465 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 67214.2us
I20260709 13:20:16.059484 18201 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:16.066852 18201 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:16.067335 18201 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:16.067664 18815 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:16.000105 18824 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:16.093636 18937 log.cc:927] Time spent T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
W20260709 13:20:16.106201 18946 log.cc:927] Time spent T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:20:16.062914 18888 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.049s
I20260709 13:20:16.129638 18815 raft_consensus.cc:697] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:20:16.139417 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 81098.9us
W20260709 13:20:16.141170 18914 consensus_meta.cc:287] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.537s	user 0.006s	sys 0.004s
I20260709 13:20:16.144332 18824 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:16.149816 18930 log.cc:927] Time spent T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.403s	user 0.002s	sys 0.003s
W20260709 13:20:16.159864 18441 consensus_meta.cc:287] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.574s	user 0.010s	sys 0.000s
I20260709 13:20:16.160329 18441 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 19. Current term is 20.
I20260709 13:20:16.161778 13376 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:20:16.167347 18815 consensus_queue.cc:237] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:16.145120 18201 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:16.193516 13377 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 20.
I20260709 13:20:16.223662 18201 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:16.245404 12776 raft_consensus.cc:1275] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:20:16.245858 18813 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:16.246433 18813 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:16.249105 18813 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:16.222211 18853 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 13:20:16.254364 12708 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:20:16.283558 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:16.284206 12781 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:20:16.286031 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:20:16.294478 18947 log.cc:927] Time spent T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:20:16.295825 13383 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:20:16.337237 18931 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.343523 12781 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
W20260709 13:20:16.353773 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 355023us
I20260709 13:20:16.359239 18925 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:20:16.367395 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:16.368280 12700 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:16.337898 18931 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:16.386785 18931 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:16.388655 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:16.393004 12700 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:20:16.393721 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:16.394327 12707 raft_consensus.cc:2368] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 19. Current term is 20.
W20260709 13:20:16.394821 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 119665us
I20260709 13:20:16.396087 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:16.399466 12781 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:20:16.416072 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:16.416513 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 110869us
W20260709 13:20:16.416587 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 86585.2us
I20260709 13:20:16.424345 12625 raft_consensus.cc:2368] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 18. Current term is 19.
I20260709 13:20:16.434019 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:16.434719 13383 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:20:16.452838 18915 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:16.459465 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:16.495842 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:16.455643 18915 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:16.502730 12747 leader_election.cc:400] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 19. Current term is 20.
I20260709 13:20:16.503161 12747 leader_election.cc:403] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:20:16.512109 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 45986us
W20260709 13:20:16.514554 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 64674.3us
I20260709 13:20:16.442979 18911 consensus_queue.cc:1048] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
W20260709 13:20:16.511324 18958 log.cc:927] Time spent T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:20:16.446719 18919 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 13:20:16.527127 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:16.513302 18950 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:20:16.548007 13377 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:20:16.553011 18960 log.cc:927] Time spent T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:20:16.541752 12597 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:16.539227 18918 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:20:16.578662 12598 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:16.571691 18931 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:16.586035 12747 leader_election.cc:400] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 18. Current term is 19.
I20260709 13:20:16.594748 12747 leader_election.cc:403] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:16.583491 18915 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:16.598706 18931 raft_consensus.cc:697] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:16.609908 18931 consensus_queue.cc:237] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:16.599453 18839 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 18. Current term is 19.
I20260709 13:20:16.642168 13348 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:16.644691 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:16.645748 18951 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:16.647462 18877 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:16.671661 18918 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 19. Current term is 20.
W20260709 13:20:16.677587 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 138304us
W20260709 13:20:16.742301 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 47806.2us
I20260709 13:20:16.736166 18839 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.795871 18839 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:16.801749 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:16.804235 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:16.808370 12775 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:20:16.735077 18910 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.067s
I20260709 13:20:16.798751 18839 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:16.820590 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:16.806301 18482 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:16.822235 18730 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:16.822418 18482 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:16.824971 18482 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:16.825852 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:20:16.829933 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 71135.8us
I20260709 13:20:16.894011 18971 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:20:17.007102 18970 log.cc:927] Time spent T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:20:17.001434 18971 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:17.018455 12708 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:20:17.019970 18971 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:17.029429 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:17.032315 12625 raft_consensus.cc:2393] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 17: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
W20260709 13:20:17.041261 12579 auto_rebalancer.cc:629] Failed to schedule move for tablet 612a4639a7a749be8856bbc60766e312: Illegal state: Replica 3d539f8d95d343d8b1696803f18b30d2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:17.045776 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:17.046507 12775 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:20:17.052273 12672 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:17.053149 18972 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:20:17.057613 12579 auto_rebalancer.cc:629] Failed to schedule move for tablet ea604dddf4c244a181921c4dbdd0e720: Illegal state: Replica 3d539f8d95d343d8b1696803f18b30d2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:17.059689 18972 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:17.063481 18972 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:17.071395 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:17.073714 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:17.074442 13385 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 21.
W20260709 13:20:17.075815 18966 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.282s	user 0.000s	sys 0.011s
I20260709 13:20:17.050904 18752 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.027s
W20260709 13:20:17.122936 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 43996.4us
I20260709 13:20:17.123769 12746 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:20:17.124285 12579 auto_rebalancer.cc:629] Failed to schedule move for tablet 37e99a6b09d744a9b056b83be1033820: Illegal state: Replica 3d539f8d95d343d8b1696803f18b30d2 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:17.126387 18743 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:17.125242 12746 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:17.128979 18906 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:17.129495 18906 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:17.129961 18906 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:17.145855 18972 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:17.149164 18972 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:17.152356 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:17.153172 12633 raft_consensus.cc:2393] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 17: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:20:17.154629 18906 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:17.158114 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:17.159011 18906 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:17.166957 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:17.175402 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:17.176198 13383 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:17.210664 12775 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:20:17.200162 12634 consensus_queue.cc:237] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:17.225615 12672 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:17.243214 13383 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 22.
I20260709 13:20:17.245128 12746 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:17.223937 18363 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:17.259102 18837 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:17.262365 18730 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:20:17.263188 18730 raft_consensus.cc:697] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:17.264457 18730 consensus_queue.cc:237] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:17.268077 18363 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:17.270349 18363 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:17.273223 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:17.250242 18972 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:20:17.290210 18979 log.cc:927] Time spent T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.147s	user 0.000s	sys 0.004s
W20260709 13:20:17.300429 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 59334.3us
W20260709 13:20:17.311403 18962 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.013s	sys 0.057s
W20260709 13:20:17.316839 18962 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.013s	sys 0.057s
I20260709 13:20:17.319603 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:17.322836 13383 raft_consensus.cc:2368] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 21. Current term is 22.
W20260709 13:20:17.363155 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 105875us
I20260709 13:20:17.382946 18972 raft_consensus.cc:697] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:17.385852 18972 consensus_queue.cc:237] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:17.390190 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 109483us
I20260709 13:20:17.413616 12705 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:20:17.427047 12672 leader_election.cc:400] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 21. Current term is 22.
I20260709 13:20:17.432612 12672 leader_election.cc:403] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:17.446285 12775 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:20:17.463027 18936 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:20:17.466257 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:17.480036 18951 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.481438 18951 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:17.485002 18951 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:17.485836 18837 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:17.516088 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:17.539388 18926 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:17.542209 12524 catalog_manager.cc:5697] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:17.548795 18936 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 21. Current term is 22.
W20260709 13:20:17.549831 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 47314.8us
W20260709 13:20:17.570305 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 71802.5us
I20260709 13:20:17.579473 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:20:17.573501 12526 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40528 (request call id 138) took 55230 ms (client timeout 15000 ms). Trace:
I20260709 13:20:17.619488 12780 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:20:17.577415 12524 catalog_manager.cc:5697] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 12 to 13, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 13 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:17.632128 12524 catalog_manager.cc:5697] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 17 to 20, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 20 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:17.669525 18991 log.cc:927] Time spent T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.072s	user 0.002s	sys 0.000s
I20260709 13:20:17.690434 12597 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:17.710726 18837 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:17.711230 18837 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:17.711687 18837 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:17.725013 18837 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:17.616349 12526 rpcz_store.cc:269] 0709 13:19:22.307119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:22.345694 (+ 38575us) service_pool.cc:224] Handling call
0709 13:20:17.537335 (+55191641us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:15.602800 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:15.709766 (+106966us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:15.709800 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:20:15.709817 (+    17us) tablet.cc:662] Decoding operations
0709 13:20:15.710379 (+   562us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:15.710450 (+    71us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:15.710473 (+    23us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:20:15.710819 (+   346us) tablet.cc:701] Row locks acquired
0709 13:20:15.710840 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:20:15.710983 (+   143us) write_op.cc:270] Start()
0709 13:20:15.711098 (+   115us) write_op.cc:276] Timestamp: P: 1783603215710933 usec, L: 0
0709 13:20:15.711119 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:20:15.715934 (+  4815us) log.cc:844] Serialized 1293 byte log entry
0709 13:20:16.023249 (+307315us) op_driver.cc:464] REPLICATION: finished
0709 13:20:16.155284 (+132035us) write_op.cc:301] APPLY: starting
0709 13:20:16.155380 (+    96us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:16.163283 (+  7903us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:16.163400 (+   117us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:16.296379 (+132979us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:16.296591 (+   212us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=6,mrs_lookups=0
0709 13:20:16.296644 (+    53us) write_op.cc:312] APPLY: finished
0709 13:20:16.296898 (+   254us) log.cc:844] Serialized 41 byte log entry
0709 13:20:16.297068 (+   170us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:16.297170 (+   102us) write_op.cc:454] Released schema lock
0709 13:20:16.297216 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:16.332602 (+ 35386us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:16.338371 (+  5769us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1081958,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":131731,"apply.run_cpu_time_us":7018,"apply.run_wall_time_us":142325,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16679,"num_ops":3,"prepare.queue_time_us":159382,"prepare.run_cpu_time_us":22928,"prepare.run_wall_time_us":263578,"raft.queue_time_us":184628,"raft.run_cpu_time_us":25491,"raft.run_wall_time_us":93803,"replication_time_us":309126,"thread_start_us":400297,"threads_started":4,"wal-append.queue_time_us":117485,"wal-append.run_cpu_time_us":42470,"wal-append.run_wall_time_us":1467400}]]}
I20260709 13:20:17.739805 18837 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:17.747567 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:17.748298 12781 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:17.756415 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:20:17.757972 12597 consensus_peers.cc:597] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d1362166bcbb48e88ac7b5862a23517e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:17.782560 12781 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
W20260709 13:20:17.853344 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 70280.9us
I20260709 13:20:17.892997 13376 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:20:17.765352 12524 catalog_manager.cc:5697] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 15 to 20, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 20 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:17.894239 12706 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:20:17.895651 12524 catalog_manager.cc:5697] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:17.899221 18918 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:17.913838 12524 catalog_manager.cc:5697] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 9 to 14, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 14 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:17.944248 12524 catalog_manager.cc:5697] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
W20260709 13:20:17.946520 18993 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.249s	user 0.000s	sys 0.005s
I20260709 13:20:17.946161 12524 catalog_manager.cc:5697] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:17.953166 12597 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:17.961956 18931 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:17.947877 12524 catalog_manager.cc:5697] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:17.941206 18813 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 13:20:17.970311 18931 raft_consensus.cc:697] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:17.990000 18931 consensus_queue.cc:237] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:17.985034 18990 log.cc:927] Time spent T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.324s	user 0.004s	sys 0.005s
I20260709 13:20:17.985711 12709 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:20:18.004151 12775 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:20:17.963440 12524 catalog_manager.cc:5697] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 14 to 16, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 16 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:18.007238 12524 catalog_manager.cc:5697] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 10 to 14, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 14 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:18.018386 18751 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:20:18.024951 12524 catalog_manager.cc:5697] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 7 to 12, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 12 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:18.027132 12524 catalog_manager.cc:5697] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 14 to 16, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 16 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:18.040751 18856 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:20:18.028844 12524 catalog_manager.cc:5697] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 12 to 16, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 16 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:18.067560 12524 catalog_manager.cc:5697] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 12 to 16, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 16 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:18.087952 12524 catalog_manager.cc:5697] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 16 to 17, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 17 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:18.088750 18931 raft_consensus.cc:2955] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Committing config change with OpId 19.15: config changed from index -1 to 15, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:18.092783 12775 raft_consensus.cc:2955] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Committing config change with OpId 19.15: config changed from index -1 to 15, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:18.090965 12524 catalog_manager.cc:5697] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:18.103479 12524 catalog_manager.cc:5697] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 15 to 16, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 16 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:18.105082 12524 catalog_manager.cc:5697] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 7 to 14, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 14 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:18.105847 12709 raft_consensus.cc:2955] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Committing config change with OpId 19.15: config changed from index -1 to 15, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:18.106832 12524 catalog_manager.cc:5697] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 13 to 16, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 16 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:18.119956 12775 raft_consensus.cc:1240] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:18.121729 18936 consensus_queue.cc:1059] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:18.113710 12524 catalog_manager.cc:5697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 14 to 17, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 17 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:18.122588 18936 raft_consensus.cc:3055] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:18.122967 18936 raft_consensus.cc:740] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:18.124224 18936 consensus_queue.cc:260] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:18.125886 18936 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:18.127946 12524 catalog_manager.cc:5697] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 10 to 13, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 13 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:18.130096 12524 catalog_manager.cc:5697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 13 to 16, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 16 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:18.134325 12524 catalog_manager.cc:5697] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:18.136469 12524 catalog_manager.cc:5697] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 11 to 17, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 17 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:18.138129 12524 catalog_manager.cc:5697] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:18.139375 12524 catalog_manager.cc:5697] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 9 to 19. New cstate: current_term: 19 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:18.140988 12524 catalog_manager.cc:5697] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:18.142450 12524 catalog_manager.cc:5697] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 8 to 11, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 11 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:18.144135 12524 catalog_manager.cc:5697] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 17 to 21, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 21 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:18.145610 12524 catalog_manager.cc:5697] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:18.179414 12634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:18.180333 12634 raft_consensus.cc:606] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:18.180653 12634 raft_consensus.cc:614] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:18.199807 12634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:18.200520 12634 raft_consensus.cc:606] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:18.212626 12634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:18.213312 12634 raft_consensus.cc:606] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:18.213691 12634 raft_consensus.cc:614] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:18.218497 12632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:18.219323 12632 raft_consensus.cc:606] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:18.219650 12632 raft_consensus.cc:614] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:18.222311 12709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:20:18.224457 12709 raft_consensus.cc:606] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:20:18.224838 12709 raft_consensus.cc:614] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:18.232959 12634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:18.233592 12634 raft_consensus.cc:606] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:18.235625 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:20:18.237713 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:20:18.394237 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 83720.2us
W20260709 13:20:18.394367 19005 log.cc:927] Time spent T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb: Append to log took a long time: real 0.106s	user 0.003s	sys 0.000s
I20260709 13:20:18.444063 12708 raft_consensus.cc:1275] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:20:18.444475 12708 pending_rounds.cc:85] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 9
I20260709 13:20:18.444718 12708 pending_rounds.cc:107] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:20:18.454051 12708 raft_consensus.cc:2889] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:18.488876 18997 consensus_queue.cc:1048] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
W20260709 13:20:18.595547 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 123450us
W20260709 13:20:18.621233 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 144056us
I20260709 13:20:18.660025 19006 ts_tablet_manager.cc:933] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:20:18.757304 18997 raft_consensus.cc:993] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e: : Instructing follower 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:20:18.806282 18751 raft_consensus.cc:1081] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Signalling peer 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:20:18.825685 12777 raft_consensus.cc:1275] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:20:18.828434 19026 consensus_queue.cc:1048] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:18.828850 19006 tablet_copy_client.cc:323] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.65:43603
I20260709 13:20:18.875257 12782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:36488
I20260709 13:20:18.876084 12782 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:18.876431 12782 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:18.893358 19011 raft_consensus.cc:993] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: : Instructing follower 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:20:18.893929 19011 raft_consensus.cc:1081] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Signalling peer 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:20:18.910411 12780 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:36488
I20260709 13:20:18.911000 12780 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:18.911392 12780 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:18.932511 12780 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:18.930508 12782 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:18.934864 12775 raft_consensus.cc:1240] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:18.946373 18751 consensus_queue.cc:1059] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:18.947470 18751 raft_consensus.cc:3055] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:18.966022 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:18.975498 12634 raft_consensus.cc:3055] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:18.978195 12634 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:18.964089 18751 raft_consensus.cc:740] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:18.988915 12780 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:18.994315 12634 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:18.999915 12634 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:19.032373 18751 consensus_queue.cc:260] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:19.052537 18751 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:19.058403 12782 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:19.066211 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:19.068186 12705 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:19.114464 12634 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
I20260709 13:20:19.119776 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:19.120437 12709 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:19.123175 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:19.173933 12709 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
W20260709 13:20:19.175199 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 60824.4us
W20260709 13:20:19.193709 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 381401us
I20260709 13:20:19.196640 12644 tablet_copy_service.cc:140] P 19cd864c053744bebd96093a7982537e: Received BeginTabletCopySession request for tablet d1362166bcbb48e88ac7b5862a23517e from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:39590)
I20260709 13:20:19.197363 12644 tablet_copy_service.cc:161] P 19cd864c053744bebd96093a7982537e: Beginning new tablet copy session on tablet d1362166bcbb48e88ac7b5862a23517e from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:39590: session id = 522fd2ba0e9f402691b5ef29af8579cd-d1362166bcbb48e88ac7b5862a23517e
I20260709 13:20:19.201817 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:20:19.219976 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:19.225289 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:19.228497 18962 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:19.233682 19013 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:19.235661 12644 tablet_copy_source_session.cc:215] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:19.239238 18962 raft_consensus.cc:697] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:19.240320 18962 consensus_queue.cc:237] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:19.246910 19013 raft_consensus.cc:697] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:19.253129 19006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1362166bcbb48e88ac7b5862a23517e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:19.253145 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:19.248075 19013 consensus_queue.cc:237] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:19.285346 12705 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
I20260709 13:20:19.302302 12632 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:20:19.395269 19006 tablet_copy_client.cc:806] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:19.395980 19006 tablet_copy_client.cc:670] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:19.444676 19006 tablet_copy_client.cc:538] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:19.498036 19006 tablet_bootstrap.cc:492] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:20:19.527388 12528 catalog_manager.cc:5697] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:19.530424 12524 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54792 (request call id 230) took 9835 ms. Trace:
I20260709 13:20:19.530928 12528 catalog_manager.cc:5697] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 8 to 10, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 10 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:19.530880 12524 rpcz_store.cc:276] 0709 13:20:09.695131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:09.708967 (+ 13836us) service_pool.cc:224] Handling call
0709 13:20:19.530348 (+9821381us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:18.215086 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:18.215801 (+   715us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:18.215861 (+    60us) write_op.cc:435] Acquired schema lock
0709 13:20:18.215883 (+    22us) tablet.cc:662] Decoding operations
0709 13:20:18.234264 (+ 18381us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:18.234682 (+   418us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:18.234710 (+    28us) tablet.cc:685] Acquiring locks for 31 operations
0709 13:20:18.240877 (+  6167us) tablet.cc:701] Row locks acquired
0709 13:20:18.240925 (+    48us) write_op.cc:260] PREPARE: finished
0709 13:20:18.241087 (+   162us) write_op.cc:270] Start()
0709 13:20:18.241293 (+   206us) write_op.cc:276] Timestamp: P: 1783603218241032 usec, L: 0
0709 13:20:18.241332 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 13:20:18.242544 (+  1212us) log.cc:844] Serialized 12083 byte log entry
0709 13:20:18.742972 (+500428us) op_driver.cc:464] REPLICATION: finished
0709 13:20:18.791348 (+ 48376us) write_op.cc:301] APPLY: starting
0709 13:20:18.791486 (+   138us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:19.059342 (+267856us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:19.059457 (+   115us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:19.355139 (+295682us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:19.355479 (+   340us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=62,key_file_lookups=62,delta_file_lookups=62,mrs_lookups=0
0709 13:20:19.355521 (+    42us) write_op.cc:312] APPLY: finished
0709 13:20:19.355927 (+   406us) log.cc:844] Serialized 268 byte log entry
0709 13:20:19.356132 (+   205us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:19.356366 (+   234us) write_op.cc:454] Released schema lock
0709 13:20:19.356451 (+    85us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:19.370869 (+ 14418us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:19.370899 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":25280,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":47968,"apply.run_cpu_time_us":47162,"apply.run_wall_time_us":570458,"cfile_cache_hit":126,"cfile_cache_hit_bytes":133831,"num_ops":31,"prepare.queue_time_us":43008,"prepare.run_cpu_time_us":22642,"prepare.run_wall_time_us":77691,"raft.queue_time_us":171147,"raft.run_cpu_time_us":8830,"raft.run_wall_time_us":93311,"replication_time_us":491302,"thread_start_us":199465,"threads_started":4,"wal-append.queue_time_us":45745}]]}
I20260709 13:20:19.538609 12528 catalog_manager.cc:5697] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 11 to 14, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 14 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:19.548959 12528 catalog_manager.cc:5697] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:19.595293 12528 catalog_manager.cc:5697] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 12 to 16, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 16 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:19.597630 12528 catalog_manager.cc:5697] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 4 to 9, term changed from 8 to 9, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66), VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New cstate: current_term: 9 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:19.599291 12528 catalog_manager.cc:5697] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 14 to 16, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 16 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:19.651628 12629 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:20:19.653357 18839 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 13:20:19.672281 12708 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:20:19.676631 18482 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:20:19.728018 12708 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:20:19.730712 18589 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:19.748803 12634 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:20:19.769821 18482 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:19.799640 12524 catalog_manager.cc:5697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 15 to 18, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 18 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
W20260709 13:20:19.826045 12528 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40528 (request call id 139) took 32764 ms (client timeout 15000 ms). Trace:
I20260709 13:20:19.827194 12624 tablet_service.cc:1558] Processing DeleteTablet for tablet af96e52b004d4c8c942cc0ec33c95c99 with delete_type TABLET_DATA_TOMBSTONED (TS 19cd864c053744bebd96093a7982537e not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:56830
W20260709 13:20:19.835496 12746 consensus_peers.cc:597] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:19.826609 12528 rpcz_store.cc:269] 0709 13:19:47.061665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:47.061888 (+   223us) service_pool.cc:224] Handling call
0709 13:19:47.239195 (+177307us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b540011eed4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31c090 00007f4c6d3baf3a 00007f4c731711cc 00007f4c7326f37e 00007f4c717302a4 00007f4c71730232 00007f4c71730161 00007f4c717300dc 00007f4c7172f392 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:20:19.825942 (+32586747us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:19.663833 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:19.664384 (+   551us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:19.664415 (+    31us) write_op.cc:435] Acquired schema lock
0709 13:20:19.664431 (+    16us) tablet.cc:662] Decoding operations
0709 13:20:19.665301 (+   870us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:19.665401 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:19.665451 (+    50us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:20:19.666285 (+   834us) tablet.cc:701] Row locks acquired
0709 13:20:19.666313 (+    28us) write_op.cc:260] PREPARE: finished
0709 13:20:19.666526 (+   213us) write_op.cc:270] Start()
0709 13:20:19.666690 (+   164us) write_op.cc:276] Timestamp: P: 1783603219666463 usec, L: 0
0709 13:20:19.666724 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 13:20:19.671010 (+  4286us) log.cc:844] Serialized 2814 byte log entry
0709 13:20:19.707410 (+ 36400us) op_driver.cc:464] REPLICATION: finished
0709 13:20:19.708609 (+  1199us) write_op.cc:301] APPLY: starting
0709 13:20:19.708695 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:19.731872 (+ 23177us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:19.732765 (+   893us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:19.769809 (+ 37044us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:19.769994 (+   185us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=14,mrs_lookups=0
0709 13:20:19.770038 (+    44us) write_op.cc:312] APPLY: finished
0709 13:20:19.770276 (+   238us) log.cc:844] Serialized 73 byte log entry
0709 13:20:19.773730 (+  3454us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:19.773859 (+   129us) write_op.cc:454] Released schema lock
0709 13:20:19.773907 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:19.774306 (+   399us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:19.774347 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":167573,"reported_tablets":28,"spinlock_wait_cycles":259382016,"child_traces":[["op",{"apply.queue_time_us":967,"apply.run_cpu_time_us":10554,"apply.run_wall_time_us":65642,"cfile_cache_hit":28,"cfile_cache_hit_bytes":29288,"num_ops":7,"prepare.queue_time_us":30146,"prepare.run_cpu_time_us":4601,"prepare.run_wall_time_us":13800,"replication_time_us":40511,"thread_start_us":17766,"threads_started":1}]]}
I20260709 13:20:19.871703 19048 tablet_replica.cc:333] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:20:19.873082 19048 raft_consensus.cc:2243] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:19.802657 12524 catalog_manager.cc:5697] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:19.879356 19048 pending_rounds.cc:70] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e: Trying to abort 1 pending ops.
I20260709 13:20:19.880219 19048 pending_rounds.cc:77] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e: Aborting op as it isn't in flight: id { term: 9 index: 8 } timestamp: 7305638591511855104 op_type: NO_OP noop_request { }
I20260709 13:20:19.882869 19048 raft_consensus.cc:2889] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:20:19.883529 19048 raft_consensus.cc:2272] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:19.877120 12524 catalog_manager.cc:5697] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 13 to 16, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 16 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:19.888958 12524 catalog_manager.cc:5697] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 12 to 18, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 18 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:19.891571 12524 catalog_manager.cc:5697] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 12 to 13, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 13 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:19.904568 19048 ts_tablet_manager.cc:1916] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:19.925174 12524 catalog_manager.cc:5697] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:19.964875 12524 catalog_manager.cc:5697] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 13 to 15, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 15 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:19.966714 12524 catalog_manager.cc:5697] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 16 to 18, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 18 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:19.968566 12524 catalog_manager.cc:5697] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 10 to 14, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 14 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:19.970148 12524 catalog_manager.cc:5697] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 18 to 20, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 20 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:19.995067 12524 catalog_manager.cc:5697] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:20.002470 12524 catalog_manager.cc:5697] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:20.013034 12524 catalog_manager.cc:5697] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 15 to 21, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 21 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:20.058133 12524 catalog_manager.cc:5697] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 16 to 17, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 17 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:20.061262 12524 catalog_manager.cc:5697] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:20.062731 12524 catalog_manager.cc:5697] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:20.039921 19006 tablet_bootstrap.cc:492] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=2 applied=15 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:20.079113 19006 tablet_bootstrap.cc:492] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:20:20.080518 19006 ts_tablet_manager.cc:1403] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.583s	user 0.075s	sys 0.047s
I20260709 13:20:20.118949 12524 catalog_manager.cc:5697] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 10 to 13, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 13 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:20.137678 19006 raft_consensus.cc:359] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:20.139031 19006 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:20:20.139962 19006 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:20.164328 19006 ts_tablet_manager.cc:1434] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.083s	user 0.004s	sys 0.004s
I20260709 13:20:20.121030 12524 catalog_manager.cc:5697] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index -1 to 15, term changed from 12 to 20, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to <none>, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 20 committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:20.165813 12524 catalog_manager.cc:5697] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 13 to 20, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 20 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:20.168044 12524 catalog_manager.cc:5697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 14 to 19, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 19 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:20.129279 19048 ts_tablet_manager.cc:1929] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 9.8
I20260709 13:20:20.191180 19048 log.cc:1199] T af96e52b004d4c8c942cc0ec33c95c99 P 19cd864c053744bebd96093a7982537e: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-0-root/wals/af96e52b004d4c8c942cc0ec33c95c99
I20260709 13:20:20.194478 12517 catalog_manager.cc:5028] TS 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): tablet af96e52b004d4c8c942cc0ec33c95c99 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:20:20.201344 12644 tablet_copy_service.cc:342] P 19cd864c053744bebd96093a7982537e: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-d1362166bcbb48e88ac7b5862a23517e received from {username='slave'} at 127.0.0.1:39590
I20260709 13:20:20.202077 12644 tablet_copy_service.cc:434] P 19cd864c053744bebd96093a7982537e: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-d1362166bcbb48e88ac7b5862a23517e on tablet d1362166bcbb48e88ac7b5862a23517e with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:20.170409 12524 catalog_manager.cc:5697] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 14 to 23, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 23 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:20.219991 12524 catalog_manager.cc:5697] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 12 to 18, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 18 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
W20260709 13:20:20.351101 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 56626.2us
I20260709 13:20:20.367820 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:20:20.461273 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 96393.4us
I20260709 13:20:20.720711 12524 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54792 (request call id 231) took 1149 ms. Trace:
I20260709 13:20:20.732143 12527 catalog_manager.cc:5697] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 10 to 13, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 13 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:20.721138 12524 rpcz_store.cc:276] 0709 13:20:19.571846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:19.572042 (+   196us) service_pool.cc:224] Handling call
0709 13:20:20.720617 (+1148575us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:20.285227 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:20.285894 (+   667us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:20.285927 (+    33us) write_op.cc:435] Acquired schema lock
0709 13:20:20.285945 (+    18us) tablet.cc:662] Decoding operations
0709 13:20:20.287882 (+  1937us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:20.287942 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:20.287981 (+    39us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:20:20.325749 (+ 37768us) tablet.cc:701] Row locks acquired
0709 13:20:20.325780 (+    31us) write_op.cc:260] PREPARE: finished
0709 13:20:20.326346 (+   566us) write_op.cc:270] Start()
0709 13:20:20.326478 (+   132us) write_op.cc:276] Timestamp: P: 1783603220325880 usec, L: 0
0709 13:20:20.326497 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:20:20.327259 (+   762us) log.cc:844] Serialized 8675 byte log entry
0709 13:20:20.348147 (+ 20888us) op_driver.cc:464] REPLICATION: finished
0709 13:20:20.407083 (+ 58936us) write_op.cc:301] APPLY: starting
0709 13:20:20.513703 (+106620us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:20.555605 (+ 41902us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:20.555718 (+   113us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:20.588397 (+ 32679us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:20.588737 (+   340us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=44,mrs_lookups=0
0709 13:20:20.588791 (+    54us) write_op.cc:312] APPLY: finished
0709 13:20:20.589259 (+   468us) log.cc:844] Serialized 196 byte log entry
0709 13:20:20.589444 (+   185us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:20.601715 (+ 12271us) write_op.cc:454] Released schema lock
0709 13:20:20.601786 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:20.602237 (+   451us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:20.602270 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6274,"reported_tablets":26,"child_traces":[["op",{"apply.queue_time_us":58653,"apply.run_cpu_time_us":36848,"apply.run_wall_time_us":195098,"cfile_cache_hit":90,"cfile_cache_hit_bytes":96175,"num_ops":22,"prepare.queue_time_us":22039,"prepare.run_cpu_time_us":6077,"prepare.run_wall_time_us":46762,"replication_time_us":21482,"thread_start_us":46670,"threads_started":2}]]}
I20260709 13:20:20.750506 12527 catalog_manager.cc:5697] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 9 to 12, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 12 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:20:20.819124 12527 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54800 (request call id 232) took 40919 ms (client timeout 15000 ms). Trace:
W20260709 13:20:20.819676 12527 rpcz_store.cc:269] 0709 13:19:39.899883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:39.900205 (+   322us) service_pool.cc:224] Handling call
0709 13:20:20.819050 (+40918845us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:20.759167 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:20.759685 (+   518us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:20.759714 (+    29us) write_op.cc:435] Acquired schema lock
0709 13:20:20.759726 (+    12us) tablet.cc:662] Decoding operations
0709 13:20:20.760130 (+   404us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:20.760180 (+    50us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:20.760211 (+    31us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:20:20.760518 (+   307us) tablet.cc:701] Row locks acquired
0709 13:20:20.760532 (+    14us) write_op.cc:260] PREPARE: finished
0709 13:20:20.760687 (+   155us) write_op.cc:270] Start()
0709 13:20:20.760827 (+   140us) write_op.cc:276] Timestamp: P: 1783603220760641 usec, L: 0
0709 13:20:20.760843 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 13:20:20.761434 (+   591us) log.cc:844] Serialized 913 byte log entry
0709 13:20:20.764281 (+  2847us) op_driver.cc:464] REPLICATION: finished
0709 13:20:20.764653 (+   372us) write_op.cc:301] APPLY: starting
0709 13:20:20.764733 (+    80us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:20.766223 (+  1490us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:20.766319 (+    96us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:20.767911 (+  1592us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:20.768082 (+   171us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:20:20.768127 (+    45us) write_op.cc:312] APPLY: finished
0709 13:20:20.768328 (+   201us) log.cc:844] Serialized 33 byte log entry
0709 13:20:20.768512 (+   184us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:20.768749 (+   237us) write_op.cc:454] Released schema lock
0709 13:20:20.768796 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:20.769178 (+   382us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:20.769206 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":123,"apply.run_cpu_time_us":4466,"apply.run_wall_time_us":4466,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8368,"num_ops":2,"prepare.queue_time_us":108,"prepare.run_cpu_time_us":2533,"prepare.run_wall_time_us":2534,"replication_time_us":3281}]]}
I20260709 13:20:20.836463 12532 catalog_manager.cc:5697] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index 15 to -1, term changed from 20 to 13, leader changed from <none> to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67), NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) evicted. New cstate: current_term: 13 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:20.854521 12524 catalog_manager.cc:5697] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 10 to 15, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 15 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:20:20.856181 12524 catalog_manager.cc:5697] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index -1 to 15, term changed from 13 to 20, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 20 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:20.928689 13375 tablet_service.cc:1558] Processing DeleteTablet for tablet d1362166bcbb48e88ac7b5862a23517e with delete_type TABLET_DATA_TOMBSTONED (TS 522fd2ba0e9f402691b5ef29af8579cd not found in new config with opid_index -1) from {username='slave'} at 127.0.0.1:52424
W20260709 13:20:20.943632 12532 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54814 (request call id 222) took 44669 ms (client timeout 15000 ms). Trace:
I20260709 13:20:20.941722 12529 catalog_manager.cc:5697] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 13 to 14, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 14 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:20.950475 19067 tablet_replica.cc:333] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:20:20.951678 19067 raft_consensus.cc:2243] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEARNER]: Raft consensus shutting down.
I20260709 13:20:20.952368 19067 raft_consensus.cc:2272] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEARNER]: Raft consensus is shut down!
W20260709 13:20:20.944295 12532 rpcz_store.cc:269] 0709 13:19:36.274313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.274621 (+   308us) service_pool.cc:224] Handling call
0709 13:20:20.943564 (+44668943us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:20.842454 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:20.843014 (+   560us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:20.843048 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:20:20.843085 (+    37us) tablet.cc:662] Decoding operations
0709 13:20:20.843417 (+   332us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:20.843474 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:20.843503 (+    29us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:20:20.843863 (+   360us) tablet.cc:701] Row locks acquired
0709 13:20:20.843883 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:20:20.844032 (+   149us) write_op.cc:270] Start()
0709 13:20:20.844130 (+    98us) write_op.cc:276] Timestamp: P: 1783603220843984 usec, L: 0
0709 13:20:20.844151 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:20:20.844759 (+   608us) log.cc:844] Serialized 538 byte log entry
0709 13:20:20.848372 (+  3613us) op_driver.cc:464] REPLICATION: finished
0709 13:20:20.848834 (+   462us) write_op.cc:301] APPLY: starting
0709 13:20:20.848910 (+    76us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:20.849799 (+   889us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:20.849895 (+    96us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:20.850677 (+   782us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:20.850864 (+   187us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:20:20.850914 (+    50us) write_op.cc:312] APPLY: finished
0709 13:20:20.851110 (+   196us) log.cc:844] Serialized 25 byte log entry
0709 13:20:20.851284 (+   174us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:20.851384 (+   100us) write_op.cc:454] Released schema lock
0709 13:20:20.851430 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:20.851829 (+   399us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:20.851870 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":27171,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":234,"apply.run_cpu_time_us":2949,"apply.run_wall_time_us":2948,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4184,"num_ops":1,"prepare.queue_time_us":151,"prepare.run_cpu_time_us":2553,"prepare.run_wall_time_us":2554,"replication_time_us":4078}]]}
I20260709 13:20:20.958520 12632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:20.959196 12632 raft_consensus.cc:606] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:20.959565 12632 raft_consensus.cc:614] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:20.952785 12529 catalog_manager.cc:5697] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:20.966499 12632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:20.967257 12632 raft_consensus.cc:606] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:20.967684 12632 raft_consensus.cc:614] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:20.970474 12632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc7827272b424bad88349b6b7437844b"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:20.971207 12632 raft_consensus.cc:606] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:20.967708 19067 ts_tablet_manager.cc:1916] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:20.974431 12632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:20.979308 12632 raft_consensus.cc:606] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:20.989908 12632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:20.990527 12632 raft_consensus.cc:606] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Received request to transfer leadership to TS 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:20.993649 12709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:20:20.994303 12709 raft_consensus.cc:606] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 21 LEADER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:20:20.997972 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:20:20.999922 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:20:21.000892 12746 consensus_peers.cc:597] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:21.014819 19067 ts_tablet_manager.cc:1929] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 20.16
I20260709 13:20:21.015489 19067 log.cc:1199] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-3-root/wals/d1362166bcbb48e88ac7b5862a23517e
I20260709 13:20:21.024884 12516 catalog_manager.cc:5028] TS 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): tablet d1362166bcbb48e88ac7b5862a23517e (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:20:21.048581 19058 raft_consensus.cc:993] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530: : Instructing follower 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:21.054091 19019 raft_consensus.cc:1081] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 21 LEADER]: Signalling peer 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:21.059333 12634 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa"
dest_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:56832
I20260709 13:20:21.060009 12634 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:21.060384 12634 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:21.123320 18931 raft_consensus.cc:993] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e: : Instructing follower 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:20:21.124445 19055 raft_consensus.cc:1081] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Signalling peer 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:20:21.127080 12780 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:36488
I20260709 13:20:21.127676 12780 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:21.128005 12780 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:21.135365 12780 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:21.138749 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:21.138792 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:21.139391 13383 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:21.139521 12632 raft_consensus.cc:3055] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:21.139987 12632 raft_consensus.cc:740] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:21.141162 12632 consensus_queue.cc:260] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:21.142344 12632 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:21.147119 13383 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:20:21.148658 12746 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:21.150369 19061 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:21.151444 19061 raft_consensus.cc:697] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:21.152870 19061 consensus_queue.cc:237] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:21.165488 12780 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:21.224367 12632 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:20:21.186295 12634 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:21.241782 12634 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:21.243184 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:21.250202 12777 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:20:21.275534 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 48527.1us
I20260709 13:20:21.279434 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:21.280100 12707 raft_consensus.cc:3055] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:20:21.280388 12707 raft_consensus.cc:740] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:21.281107 12707 consensus_queue.cc:260] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:21.282289 12707 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:21.287755 12531 catalog_manager.cc:5697] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:21.292505 12531 catalog_manager.cc:5697] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:21.296398 18931 raft_consensus.cc:993] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e: : Instructing follower 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:20:21.296947 18931 raft_consensus.cc:1081] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Signalling peer 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:20:21.297142 12531 catalog_manager.cc:5697] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 9 to 16, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 16 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:21.297163 12707 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 22.
I20260709 13:20:21.302440 12531 catalog_manager.cc:5697] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 12 to 15, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 15 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:21.305620 12531 catalog_manager.cc:5697] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:21.308607 13383 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878"
dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:51754
I20260709 13:20:21.312952 13383 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:21.313436 13383 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:21.320487 13383 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:21.330570 13383 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:21.339562 12777 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 22.
I20260709 13:20:21.341055 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:21.341670 12707 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:21.366815 12707 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 20.
I20260709 13:20:21.379685 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:21.381245 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:21.381934 12634 raft_consensus.cc:3055] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:21.382287 12634 raft_consensus.cc:740] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:20:21.375885 12529 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54814 (request call id 223) took 26420 ms (client timeout 15000 ms). Trace:
I20260709 13:20:21.404685 12634 consensus_queue.cc:260] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:21.410151 12634 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:21.394356 19057 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:20:21.385376 12529 rpcz_store.cc:269] 0709 13:19:54.955354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:54.955523 (+   169us) service_pool.cc:224] Handling call
0709 13:20:21.145652 (+26190129us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:21.145682 (+    30us) write_op.cc:454] Released schema lock
0709 13:20:21.375800 (+230118us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:21.085605 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:21.086276 (+   671us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:21.086312 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:20:21.086325 (+    13us) tablet.cc:662] Decoding operations
0709 13:20:21.086775 (+   450us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:21.086825 (+    50us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:21.086854 (+    29us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:20:21.087401 (+   547us) tablet.cc:701] Row locks acquired
0709 13:20:21.087416 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:20:21.087573 (+   157us) write_op.cc:270] Start()
0709 13:20:21.087714 (+   141us) write_op.cc:276] Timestamp: P: 1783603221087520 usec, L: 0
0709 13:20:21.087730 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 13:20:21.088363 (+   633us) log.cc:844] Serialized 913 byte log entry
0709 13:20:21.093542 (+  5179us) spinlock_profiling.cc:243] Waited 948 us on lock 0x7b54002a01a0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31c090 00007f4c6d3296b8 00007f4c6d3292fa 00007f4c6d3ad965 00007f4c6d3ade09 00007f4c70de8581 00007f4c70de7d58 00007f4c70dec9f1 00007f4c70dec9a9 00007f4c70dec939 00007f4c70dec901
0709 13:20:21.094026 (+   484us) op_driver.cc:464] REPLICATION: finished
0709 13:20:21.095805 (+  1779us) write_op.cc:301] APPLY: starting
0709 13:20:21.095897 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:21.097515 (+  1618us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:21.097540 (+    25us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:21.103098 (+  5558us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:21.103258 (+   160us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:20:21.103322 (+    64us) write_op.cc:312] APPLY: finished
0709 13:20:21.103536 (+   214us) log.cc:844] Serialized 33 byte log entry
0709 13:20:21.103906 (+   370us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:21.104001 (+    95us) write_op.cc:454] Released schema lock
0709 13:20:21.104045 (+    44us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":12788,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":1413,"apply.run_cpu_time_us":4555,"apply.run_wall_time_us":14983,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8368,"num_ops":2,"prepare.queue_time_us":40070,"prepare.run_cpu_time_us":5931,"prepare.run_wall_time_us":8243,"replication_time_us":6149,"spinlock_wait_cycles":2181504}]]}
I20260709 13:20:21.425280 12598 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:20:21.440272 12531 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40528 (request call id 140) took 16096 ms (client timeout 15000 ms). Trace:
I20260709 13:20:21.440732 18751 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:20:21.440876 12531 rpcz_store.cc:269] 0709 13:20:05.344029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:05.348449 (+  4420us) service_pool.cc:224] Handling call
0709 13:20:21.440179 (+16091730us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:21.310599 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:21.311346 (+   747us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:21.311390 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:20:21.311406 (+    16us) tablet.cc:662] Decoding operations
0709 13:20:21.312048 (+   642us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:21.312103 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:21.312130 (+    27us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:20:21.312626 (+   496us) tablet.cc:701] Row locks acquired
0709 13:20:21.312639 (+    13us) write_op.cc:260] PREPARE: finished
0709 13:20:21.312786 (+   147us) write_op.cc:270] Start()
0709 13:20:21.312907 (+   121us) write_op.cc:276] Timestamp: P: 1783603221312738 usec, L: 0
0709 13:20:21.312923 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 13:20:21.313533 (+   610us) log.cc:844] Serialized 2054 byte log entry
0709 13:20:21.315582 (+  2049us) op_driver.cc:464] REPLICATION: finished
0709 13:20:21.316013 (+   431us) write_op.cc:301] APPLY: starting
0709 13:20:21.316095 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:21.318713 (+  2618us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:21.318794 (+    81us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:21.321844 (+  3050us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:21.322060 (+   216us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0709 13:20:21.322191 (+   131us) write_op.cc:312] APPLY: finished
0709 13:20:21.322403 (+   212us) log.cc:844] Serialized 57 byte log entry
0709 13:20:21.323152 (+   749us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:21.323317 (+   165us) write_op.cc:454] Released schema lock
0709 13:20:21.323368 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:21.323697 (+   329us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:21.323723 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":48811,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":243,"apply.run_cpu_time_us":6740,"apply.run_wall_time_us":7639,"cfile_cache_hit":20,"cfile_cache_hit_bytes":20920,"num_ops":5,"prepare.queue_time_us":129,"prepare.run_cpu_time_us":3262,"prepare.run_wall_time_us":11106,"replication_time_us":2544}]]}
I20260709 13:20:21.445189 12634 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 20.
W20260709 13:20:21.447386 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 55485.8us
I20260709 13:20:21.454224 12530 catalog_manager.cc:5697] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:21.466660 18751 raft_consensus.cc:697] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 22 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:21.467701 18751 consensus_queue.cc:237] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:21.474514 19075 raft_consensus.cc:993] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e: : Instructing follower 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:20:21.475086 19075 raft_consensus.cc:1081] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Signalling peer 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:20:21.476488 12530 catalog_manager.cc:5697] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 16 to 17, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 17 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:21.482475 12782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bc7827272b424bad88349b6b7437844b"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:36488
I20260709 13:20:21.483099 12782 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:21.483482 12782 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:20:21.550829 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 55956.2us
I20260709 13:20:21.564972 12782 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:21.528903 19057 raft_consensus.cc:697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:21.568781 12782 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:21.584436 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:21.585145 12707 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:21.615150 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:21.615832 12625 raft_consensus.cc:3055] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:21.616119 12625 raft_consensus.cc:740] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:21.616839 12625 consensus_queue.cc:260] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:21.632016 12625 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:21.634879 19094 ts_tablet_manager.cc:933] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:21.654249 12707 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
W20260709 13:20:21.659322 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 164204us
W20260709 13:20:21.659360 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 48343.1us
I20260709 13:20:21.660846 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:21.658838 19057 consensus_queue.cc:237] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:21.711081 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 99620.5us
I20260709 13:20:21.698038 19072 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:21.748449 19072 raft_consensus.cc:697] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:21.749445 19072 consensus_queue.cc:237] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:21.750664 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 85131.7us
I20260709 13:20:21.759991 13383 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:21.768512 12625 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:20:21.756966 19094 tablet_copy_client.cc:323] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.67:41181
I20260709 13:20:21.807597 12531 catalog_manager.cc:5697] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 21 to 22, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 22 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:21.809808 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 46386.2us
I20260709 13:20:21.816875 19061 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:21.872887 12793 tablet_copy_service.cc:140] P 3d539f8d95d343d8b1696803f18b30d2: Received BeginTabletCopySession request for tablet d1362166bcbb48e88ac7b5862a23517e from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:54346)
I20260709 13:20:21.873613 12793 tablet_copy_service.cc:161] P 3d539f8d95d343d8b1696803f18b30d2: Beginning new tablet copy session on tablet d1362166bcbb48e88ac7b5862a23517e from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:54346: session id = 522fd2ba0e9f402691b5ef29af8579cd-d1362166bcbb48e88ac7b5862a23517e
I20260709 13:20:21.885084 12793 tablet_copy_source_session.cc:215] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:21.895864 12625 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:21.910920 12525 catalog_manager.cc:5697] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 11 to 15, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 15 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:21.913420 12525 catalog_manager.cc:5697] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 16 to 20, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 20 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:21.915302 12525 catalog_manager.cc:5697] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 17 to 20, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 20 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:21.923267 19094 ts_tablet_manager.cc:1916] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: Deleting tablet data with delete state TABLET_DATA_COPYING
W20260709 13:20:21.925414 12530 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54800 (request call id 233) took 20958 ms (client timeout 15000 ms). Trace:
W20260709 13:20:21.925855 12530 rpcz_store.cc:269] 0709 13:20:00.966825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:00.967078 (+   253us) service_pool.cc:224] Handling call
0709 13:20:21.925341 (+20958263us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:21.629628 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:21.690478 (+ 60850us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:21.690515 (+    37us) write_op.cc:435] Acquired schema lock
0709 13:20:21.690527 (+    12us) tablet.cc:662] Decoding operations
0709 13:20:21.691153 (+   626us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:21.691203 (+    50us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:21.691232 (+    29us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:20:21.691572 (+   340us) tablet.cc:701] Row locks acquired
0709 13:20:21.691584 (+    12us) write_op.cc:260] PREPARE: finished
0709 13:20:21.691733 (+   149us) write_op.cc:270] Start()
0709 13:20:21.691845 (+   112us) write_op.cc:276] Timestamp: P: 1783603221691678 usec, L: 0
0709 13:20:21.691861 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 13:20:21.692439 (+   578us) log.cc:844] Serialized 913 byte log entry
0709 13:20:21.748317 (+ 55878us) op_driver.cc:464] REPLICATION: finished
0709 13:20:21.749076 (+   759us) write_op.cc:301] APPLY: starting
0709 13:20:21.749155 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:21.755621 (+  6466us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:21.757414 (+  1793us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:21.780247 (+ 22833us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:21.780442 (+   195us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:20:21.780493 (+    51us) write_op.cc:312] APPLY: finished
0709 13:20:21.780740 (+   247us) log.cc:844] Serialized 33 byte log entry
0709 13:20:21.781067 (+   327us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:21.781162 (+    95us) write_op.cc:454] Released schema lock
0709 13:20:21.781290 (+   128us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:21.785761 (+  4471us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:21.785799 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":8051,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":472,"apply.run_cpu_time_us":4681,"apply.run_wall_time_us":32531,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8368,"num_ops":2,"prepare.queue_time_us":1279,"prepare.run_cpu_time_us":3169,"prepare.run_wall_time_us":74232,"replication_time_us":40959}]]}
I20260709 13:20:21.929594 12525 catalog_manager.cc:5697] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 16 to 17, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 17 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:21.933485 12525 catalog_manager.cc:5697] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 16 to 19, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 19 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:21.937460 19071 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:21.936157 12525 catalog_manager.cc:5697] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 16 to 18, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 18 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:21.958766 12525 catalog_manager.cc:5697] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 16 to 19, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 19 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:22.000957 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:22.026911 19094 ts_tablet_manager.cc:1929] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 20.16
I20260709 13:20:22.027935 19094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1362166bcbb48e88ac7b5862a23517e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:22.052608 12776 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:20:22.054111 12707 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:20:22.056667 19075 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:22.084122 19074 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:20:22.151440 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 64296.8us
I20260709 13:20:22.191929 19094 tablet_copy_client.cc:806] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:22.192575 19094 tablet_copy_client.cc:670] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:22.200306 12707 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
W20260709 13:20:22.206396 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 43615.5us
I20260709 13:20:22.220283 12625 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:20:22.223887 19087 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:20:22.234555 19094 tablet_copy_client.cc:538] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:22.240199 12525 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54800 (request call id 234) took 5584 ms. Trace:
I20260709 13:20:22.240696 12525 rpcz_store.cc:276] 0709 13:20:16.654576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:16.654764 (+   188us) service_pool.cc:224] Handling call
0709 13:20:22.238684 (+5583920us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:21.964512 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:21.965016 (+   504us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:21.965053 (+    37us) write_op.cc:435] Acquired schema lock
0709 13:20:21.965071 (+    18us) tablet.cc:662] Decoding operations
0709 13:20:21.965882 (+   811us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:21.965935 (+    53us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:21.965961 (+    26us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:20:21.966548 (+   587us) tablet.cc:701] Row locks acquired
0709 13:20:21.966565 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:20:21.966697 (+   132us) write_op.cc:270] Start()
0709 13:20:21.966803 (+   106us) write_op.cc:276] Timestamp: P: 1783603221966654 usec, L: 0
0709 13:20:21.966822 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:20:21.967457 (+   635us) log.cc:844] Serialized 2814 byte log entry
0709 13:20:21.969733 (+  2276us) op_driver.cc:464] REPLICATION: finished
0709 13:20:21.970166 (+   433us) write_op.cc:301] APPLY: starting
0709 13:20:21.970247 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:21.974243 (+  3996us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:21.974348 (+   105us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:21.978798 (+  4450us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:21.978992 (+   194us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=14,mrs_lookups=0
0709 13:20:21.979036 (+    44us) write_op.cc:312] APPLY: finished
0709 13:20:21.979295 (+   259us) log.cc:844] Serialized 73 byte log entry
0709 13:20:21.980107 (+   812us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:21.980214 (+   107us) write_op.cc:454] Released schema lock
0709 13:20:21.980258 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:21.980542 (+   284us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:21.980568 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":480124,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":235,"apply.run_cpu_time_us":9520,"apply.run_wall_time_us":10349,"cfile_cache_hit":28,"cfile_cache_hit_bytes":29288,"num_ops":7,"prepare.queue_time_us":99,"prepare.run_cpu_time_us":3240,"prepare.run_wall_time_us":9222,"replication_time_us":2786}]]}
I20260709 13:20:22.251147 19092 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.035s
I20260709 13:20:22.301730 12530 catalog_manager.cc:5697] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 11 to 12, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 12 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:22.333348 12632 raft_consensus.cc:1275] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:20:22.343984 12707 raft_consensus.cc:1275] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:20:22.360301 19071 consensus_queue.cc:1048] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:22.405417 12528 catalog_manager.cc:5697] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:22.451763 19094 tablet_bootstrap.cc:492] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:20:22.472765 12530 catalog_manager.cc:5697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 19 to 20, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 20 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:22.494304 19071 consensus_queue.cc:1048] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:22.556722 12523 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54814 (request call id 224) took 8327 ms. Trace:
I20260709 13:20:22.560603 12523 rpcz_store.cc:276] 0709 13:20:14.229729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:14.229998 (+   269us) service_pool.cc:224] Handling call
0709 13:20:22.556621 (+8326623us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":393433,"reported_tablets":51}
I20260709 13:20:22.566212 12528 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40528 (request call id 141) took 2010 ms. Trace:
I20260709 13:20:22.591303 12528 rpcz_store.cc:276] 0709 13:20:20.556292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.556452 (+   160us) service_pool.cc:224] Handling call
0709 13:20:22.566141 (+2009689us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:22.447511 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:22.448234 (+   723us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:22.448282 (+    48us) write_op.cc:435] Acquired schema lock
0709 13:20:22.448303 (+    21us) tablet.cc:662] Decoding operations
0709 13:20:22.448734 (+   431us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:22.448807 (+    73us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:22.448848 (+    41us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:20:22.449073 (+   225us) tablet.cc:701] Row locks acquired
0709 13:20:22.449093 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:20:22.449282 (+   189us) write_op.cc:270] Start()
0709 13:20:22.449411 (+   129us) write_op.cc:276] Timestamp: P: 1783603222449224 usec, L: 0
0709 13:20:22.449435 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:20:22.450269 (+   834us) log.cc:844] Serialized 533 byte log entry
0709 13:20:22.452988 (+  2719us) op_driver.cc:464] REPLICATION: finished
0709 13:20:22.454667 (+  1679us) write_op.cc:301] APPLY: starting
0709 13:20:22.454774 (+   107us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:22.455787 (+  1013us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:22.455918 (+   131us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:22.457090 (+  1172us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:22.457346 (+   256us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:20:22.457389 (+    43us) write_op.cc:312] APPLY: finished
0709 13:20:22.460236 (+  2847us) log.cc:844] Serialized 25 byte log entry
0709 13:20:22.460433 (+   197us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:22.460512 (+    79us) write_op.cc:454] Released schema lock
0709 13:20:22.460550 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:22.460849 (+   299us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:22.460877 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":93986,"reported_tablets":29,"spinlock_wait_cycles":130048,"child_traces":[["op",{"apply.queue_time_us":1438,"apply.run_cpu_time_us":3598,"apply.run_wall_time_us":6181,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4184,"num_ops":1,"prepare.queue_time_us":127,"prepare.run_cpu_time_us":3111,"prepare.run_wall_time_us":3109,"replication_time_us":3377}]]}
I20260709 13:20:22.609849 12523 catalog_manager.cc:5697] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 14 to 15, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 15 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
W20260709 13:20:22.632670 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 85287.7us
W20260709 13:20:22.796887 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 110794us
I20260709 13:20:22.952230 19094 tablet_bootstrap.cc:492] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=0 applied=16 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:22.956640 19094 tablet_bootstrap.cc:492] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:20:22.971259 19094 ts_tablet_manager.cc:1403] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.520s	user 0.054s	sys 0.013s
I20260709 13:20:23.003494 19094 raft_consensus.cc:359] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:23.022344 19094 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:20:23.023255 19094 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:23.035437 19094 ts_tablet_manager.cc:1434] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.062s	user 0.005s	sys 0.002s
I20260709 13:20:23.046561 12793 tablet_copy_service.cc:342] P 3d539f8d95d343d8b1696803f18b30d2: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-d1362166bcbb48e88ac7b5862a23517e received from {username='slave'} at 127.0.0.1:54346
I20260709 13:20:23.047343 12793 tablet_copy_service.cc:434] P 3d539f8d95d343d8b1696803f18b30d2: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-d1362166bcbb48e88ac7b5862a23517e on tablet d1362166bcbb48e88ac7b5862a23517e with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:23.243397 19061 raft_consensus.cc:1064] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:20:23.254139 19061 consensus_queue.cc:237] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 15, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:23.367599 12709 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:20:23.370012 12629 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:20:23.375491 19119 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.002s
I20260709 13:20:23.382453 19096 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:20:23.390003 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 65029.2us
I20260709 13:20:23.399413 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:23.412611 13377 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEARNER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:20:23.420964 19096 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.009s
I20260709 13:20:23.466843 19071 raft_consensus.cc:2955] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Committing config change with OpId 20.17: config changed from index 15 to 17, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:23.472278 12705 raft_consensus.cc:2955] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Committing config change with OpId 20.17: config changed from index 15 to 17, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:23.518522 12625 raft_consensus.cc:2955] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Committing config change with OpId 20.17: config changed from index 15 to 17, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:23.534243 12523 catalog_manager.cc:5697] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index 15 to 17, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 20 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:23.553506 13377 raft_consensus.cc:2955] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Committing config change with OpId 20.17: config changed from index 15 to 17, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:23.796996 12709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:20:23.797788 12709 raft_consensus.cc:606] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Received request to transfer leadership to TS 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:23.802302 12777 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:20:23.802924 12777 raft_consensus.cc:606] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:20:23.806120 12776 consensus_queue.cc:237] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:23.829133 12629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:23.833693 12629 raft_consensus.cc:606] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Received request to transfer leadership to TS 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:23.857950 12777 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94e3333498524f14a17661cdb38ef144"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:20:23.871032 12777 raft_consensus.cc:606] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:20:23.872696 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:20:23.874049 13377 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:20:23.874478 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:23.877537 19123 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:20:23.875020 12629 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:20:23.887382 19123 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:20:23.969515 19119 raft_consensus.cc:2955] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Committing config change with OpId 20.18: config changed from index 17 to 18, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:23.988220 12634 raft_consensus.cc:2955] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Committing config change with OpId 20.18: config changed from index 17 to 18, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:24.007166 13385 raft_consensus.cc:2955] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Committing config change with OpId 20.18: config changed from index 17 to 18, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:20:24.008901 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 84500.5us
I20260709 13:20:24.066452 12528 catalog_manager.cc:5697] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 17 to 18, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New cstate: current_term: 20 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:24.071112 12516 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d1362166bcbb48e88ac7b5862a23517e with cas_config_opid_index 17: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:20:24.080847 19098 raft_consensus.cc:993] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e: : Instructing follower 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:20:24.081362 19098 raft_consensus.cc:1081] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Signalling peer 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:20:24.083902 13383 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec"
dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:51754
I20260709 13:20:24.084636 13383 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:24.085016 13383 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:24.099040 13383 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:24.108006 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:24.108722 12629 raft_consensus.cc:3055] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:24.110428 12629 raft_consensus.cc:740] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:24.114050 12629 consensus_queue.cc:260] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:24.115260 12629 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:24.123755 19123 raft_consensus.cc:993] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2: : Instructing follower 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:24.124719 19101 raft_consensus.cc:1081] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Signalling peer 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:24.125077 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:24.125836 12709 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:24.140588 12634 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "94e3333498524f14a17661cdb38ef144"
dest_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:56838
I20260709 13:20:24.140492 12709 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:20:24.150959 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:24.152235 12634 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:24.152684 12634 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:24.156967 19080 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:24.159590 19080 raft_consensus.cc:697] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 12 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:24.161739 19080 consensus_queue.cc:237] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:24.140938 13383 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:24.177980 19069 raft_consensus.cc:993] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530: : Instructing follower 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:20:24.202579 19069 raft_consensus.cc:1081] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Signalling peer 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:20:24.206912 13383 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8"
dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:51762
I20260709 13:20:24.207626 13383 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:24.176666 12695 tablet_service.cc:1558] Processing DeleteTablet for tablet d1362166bcbb48e88ac7b5862a23517e with delete_type TABLET_DATA_TOMBSTONED (TS 639ab2db265f4747bb7afb93e833f530 not found in new config with opid_index 18) from {username='slave'} at 127.0.0.1:49378
I20260709 13:20:24.212726 13383 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:24.228434 12634 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:24.232425 19132 tablet_replica.cc:333] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:20:24.241482 19132 raft_consensus.cc:2243] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:24.242594 19132 raft_consensus.cc:2272] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:24.240424 12523 catalog_manager.cc:5697] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 11 to 12, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 12 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:24.241511 12629 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:20:24.246196 13383 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:24.249023 19132 ts_tablet_manager.cc:1916] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:24.249756 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:24.252089 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:24.277839 12705 raft_consensus.cc:3055] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:24.278187 12705 raft_consensus.cc:740] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:24.278946 12705 consensus_queue.cc:260] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:24.280098 12705 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:24.254356 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:24.281852 12708 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:24.283034 19123 raft_consensus.cc:993] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2: : Instructing follower 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:24.283536 19123 raft_consensus.cc:1081] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Signalling peer 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:24.257071 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:24.285416 12632 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22"
dest_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:56838
I20260709 13:20:24.285899 12777 raft_consensus.cc:3055] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:24.286128 12632 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:24.286262 12777 raft_consensus.cc:740] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:24.286726 12632 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:24.294807 12632 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:24.302944 12632 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:24.304769 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:24.305583 12780 raft_consensus.cc:3055] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:24.306020 12780 raft_consensus.cc:740] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:24.307626 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:24.308182 12709 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:24.310578 12777 consensus_queue.cc:260] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:24.311693 12777 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:24.306916 12780 consensus_queue.cc:260] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:24.323501 12780 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:24.266741 12634 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:24.325063 12777 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:20:24.327512 12629 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:24.335023 12705 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 17.
I20260709 13:20:24.324251 12709 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:20:24.342227 12597 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:24.342733 12598 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:24.343360 19074 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:24.343607 19054 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:24.343875 19074 raft_consensus.cc:697] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:24.344112 19054 raft_consensus.cc:697] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:24.344780 19074 consensus_queue.cc:237] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:24.352768 12780 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:20:24.262651 13383 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:24.344874 19054 consensus_queue.cc:237] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:24.360843 13349 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:24.361804 19080 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:24.362347 19080 raft_consensus.cc:697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:24.363325 19080 consensus_queue.cc:237] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:24.335811 12708 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:20:24.406972 19132 ts_tablet_manager.cc:1929] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 20.17
I20260709 13:20:24.407912 19132 log.cc:1199] T d1362166bcbb48e88ac7b5862a23517e P 639ab2db265f4747bb7afb93e833f530: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-1-root/wals/d1362166bcbb48e88ac7b5862a23517e
I20260709 13:20:24.411239 12517 catalog_manager.cc:5028] TS 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): tablet d1362166bcbb48e88ac7b5862a23517e (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:20:24.415736 12523 catalog_manager.cc:5697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 16 to 17, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 17 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:24.422730 12629 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 17.
I20260709 13:20:24.422587 12525 catalog_manager.cc:5697] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 18 to 19, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 19 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:24.425231 12525 catalog_manager.cc:5697] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 18 to 19, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 19 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:24.614804 12629 raft_consensus.cc:1275] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:20:24.616793 19092 consensus_queue.cc:1048] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:24.648877 12705 raft_consensus.cc:1275] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:20:24.653262 19092 consensus_queue.cc:1048] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:24.810177 12777 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:20:24.811794 19091 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:20:24.814881 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 53664us
I20260709 13:20:24.824746 12709 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:20:24.833700 19074 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
W20260709 13:20:24.840718 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 57485us
I20260709 13:20:24.849869 12776 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:20:24.857282 19054 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:24.864348 12705 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:20:24.871611 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 46515.4us
I20260709 13:20:24.876142 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:24.873813 19074 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:20:24.962057 12629 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:20:24.969473 12709 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:20:24.994261 19092 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:25.013842 19092 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:20:25.018005 19151 log.cc:927] Time spent T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:20:25.021263 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 82722.9us
W20260709 13:20:25.139328 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 51392.7us
I20260709 13:20:25.331751 12634 consensus_queue.cc:237] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
W20260709 13:20:25.362653 12597 consensus_peers.cc:597] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4e4ae9d38ba4fd0a700998e4f4750be. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:25.367218 12708 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:20:25.369325 12777 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:20:25.371701 19054 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:25.372558 19155 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:25.391562 19155 raft_consensus.cc:2955] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Committing config change with OpId 20.13: config changed from index -1 to 13, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:25.406342 12777 raft_consensus.cc:2955] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index -1 to 13, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:25.421876 12776 consensus_queue.cc:237] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:25.422906 12523 catalog_manager.cc:5697] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index -1 to 13, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 20 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:25.450287 12746 consensus_peers.cc:597] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 49dad260ffe242d2b4e0976f119a19d7. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:25.444136 12708 raft_consensus.cc:2955] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index -1 to 13, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:25.452999 12705 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:20:25.455185 12634 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:20:25.456545 19156 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:25.458221 19157 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:25.503553 19157 raft_consensus.cc:2955] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Committing config change with OpId 15.10: config changed from index -1 to 10, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:25.506294 12632 raft_consensus.cc:2955] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index -1 to 10, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:25.543598 12705 raft_consensus.cc:2955] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index -1 to 10, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:25.567629 12777 consensus_queue.cc:237] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:25.598446 12528 catalog_manager.cc:5697] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index -1 to 10, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 15 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
W20260709 13:20:25.645030 12746 consensus_peers.cc:597] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 775128ed7b8d4283bf303c8d40247615. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:25.645923 12634 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:20:25.647464 19101 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:25.649194 12705 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:20:25.653054 19101 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:25.707758 19157 raft_consensus.cc:2955] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Committing config change with OpId 16.11: config changed from index -1 to 11, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:25.711470 12634 raft_consensus.cc:2955] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index -1 to 11, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:25.723327 12705 raft_consensus.cc:2955] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index -1 to 11, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:25.735973 12632 consensus_queue.cc:237] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:25.751881 12525 catalog_manager.cc:5697] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index -1 to 11, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 16 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:25.779919 12777 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:20:25.781723 19054 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:20:25.782629 12597 consensus_peers.cc:597] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 94e3333498524f14a17661cdb38ef144. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:25.789814 19074 raft_consensus.cc:2955] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Committing config change with OpId 19.12: config changed from index -1 to 12, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:25.805331 12709 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:20:25.804235 12777 raft_consensus.cc:2955] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Committing config change with OpId 19.12: config changed from index -1 to 12, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:25.806835 19054 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:25.814481 12709 raft_consensus.cc:2955] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Committing config change with OpId 19.12: config changed from index -1 to 12, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:25.832406 12525 catalog_manager.cc:5697] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index -1 to 12, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 19 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:25.878156 19174 ts_tablet_manager.cc:933] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:20:25.896863 19174 tablet_copy_client.cc:323] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.65:43603
I20260709 13:20:25.899432 12644 tablet_copy_service.cc:140] P 19cd864c053744bebd96093a7982537e: Received BeginTabletCopySession request for tablet c4e4ae9d38ba4fd0a700998e4f4750be from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:39590)
I20260709 13:20:25.900017 12644 tablet_copy_service.cc:161] P 19cd864c053744bebd96093a7982537e: Beginning new tablet copy session on tablet c4e4ae9d38ba4fd0a700998e4f4750be from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:39590: session id = 522fd2ba0e9f402691b5ef29af8579cd-c4e4ae9d38ba4fd0a700998e4f4750be
I20260709 13:20:25.910535 12644 tablet_copy_source_session.cc:215] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:25.920089 19174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4e4ae9d38ba4fd0a700998e4f4750be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:25.941220 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:25.971256 19174 tablet_copy_client.cc:806] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:25.971977 19174 tablet_copy_client.cc:670] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:25.978523 19177 ts_tablet_manager.cc:933] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:25.981530 19174 tablet_copy_client.cc:538] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:26.001997 19177 tablet_copy_client.cc:323] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.67:41181
I20260709 13:20:26.003875 12793 tablet_copy_service.cc:140] P 3d539f8d95d343d8b1696803f18b30d2: Received BeginTabletCopySession request for tablet 49dad260ffe242d2b4e0976f119a19d7 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:54346)
I20260709 13:20:26.004475 12793 tablet_copy_service.cc:161] P 3d539f8d95d343d8b1696803f18b30d2: Beginning new tablet copy session on tablet 49dad260ffe242d2b4e0976f119a19d7 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:54346: session id = 522fd2ba0e9f402691b5ef29af8579cd-49dad260ffe242d2b4e0976f119a19d7
I20260709 13:20:26.007390 19174 tablet_bootstrap.cc:492] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:20:26.033767 12793 tablet_copy_source_session.cc:215] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:26.040263 19177 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49dad260ffe242d2b4e0976f119a19d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:26.093775 12632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:26.094431 12632 raft_consensus.cc:606] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Received request to transfer leadership to TS 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:26.107009 12709 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "20db0f202ea546699789c2325d241b35"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:20:26.107733 12709 raft_consensus.cc:606] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:20:26.128786 12780 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:20:26.141214 12780 raft_consensus.cc:606] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:20:26.134433 19091 raft_consensus.cc:993] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e: : Instructing follower 639ab2db265f4747bb7afb93e833f530 to start an election
I20260709 13:20:26.142613 19091 raft_consensus.cc:1081] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Signalling peer 639ab2db265f4747bb7afb93e833f530 to start an election
I20260709 13:20:26.147873 12632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:26.148509 12632 raft_consensus.cc:606] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:26.151762 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:20:26.153860 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:26.172757 12705 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:42500
I20260709 13:20:26.173542 12705 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:26.174016 12705 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:26.179461 19180 ts_tablet_manager.cc:933] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:26.202183 19054 raft_consensus.cc:993] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e: : Instructing follower 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:20:26.202787 19074 raft_consensus.cc:1081] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Signalling peer 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:20:26.204972 12780 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:36488
I20260709 13:20:26.205552 12780 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:26.205902 12780 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:26.220783 12780 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:26.228894 12705 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:26.233212 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:26.242314 12632 raft_consensus.cc:3055] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:26.242791 12632 raft_consensus.cc:740] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:26.243712 12632 consensus_queue.cc:260] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:26.245325 12632 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:26.236218 12780 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:26.270076 12632 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:20:26.284945 19174 tablet_bootstrap.cc:492] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:20:26.295928 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 73038.7us
I20260709 13:20:26.269377 19180 tablet_copy_client.cc:323] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.67:41181
I20260709 13:20:26.341609 19174 tablet_bootstrap.cc:492] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:20:26.342640 19174 ts_tablet_manager.cc:1403] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.335s	user 0.053s	sys 0.000s
I20260709 13:20:26.346549 19174 raft_consensus.cc:359] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:26.379232 12747 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:26.384112 12705 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:26.387110 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:26.387809 12777 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:26.395226 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: true dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:26.396010 13385 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:26.347680 19174 raft_consensus.cc:740] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:20:26.405176 12777 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:20:26.412727 13385 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:20:26.401501 19101 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:26.426084 19174 consensus_queue.cc:260] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:26.430866 19181 ts_tablet_manager.cc:933] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
W20260709 13:20:26.501305 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 87722.7us
I20260709 13:20:26.510008 12777 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:26.496228 19101 raft_consensus.cc:697] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:26.562013 12793 tablet_copy_service.cc:140] P 3d539f8d95d343d8b1696803f18b30d2: Received BeginTabletCopySession request for tablet 775128ed7b8d4283bf303c8d40247615 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:54346)
I20260709 13:20:26.565338 19174 ts_tablet_manager.cc:1434] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.222s	user 0.005s	sys 0.004s
I20260709 13:20:26.593825 12793 tablet_copy_service.cc:161] P 3d539f8d95d343d8b1696803f18b30d2: Beginning new tablet copy session on tablet 775128ed7b8d4283bf303c8d40247615 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:54346: session id = 522fd2ba0e9f402691b5ef29af8579cd-775128ed7b8d4283bf303c8d40247615
I20260709 13:20:26.595613 19091 consensus_queue.cc:1059] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:20:26.694455 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:26.695170 12629 raft_consensus.cc:3055] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:26.695449 12629 raft_consensus.cc:740] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:26.699908 12629 consensus_queue.cc:260] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:26.701234 12629 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:26.689738 19181 tablet_copy_client.cc:323] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.65:43603
I20260709 13:20:26.712108 19177 tablet_copy_client.cc:806] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:26.719856 12629 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
W20260709 13:20:26.751961 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 335977us
I20260709 13:20:26.733819 19156 raft_consensus.cc:993] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2: : Instructing follower 19cd864c053744bebd96093a7982537e to start an election
W20260709 13:20:26.755000 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 78473.6us
I20260709 13:20:26.683408 19101 consensus_queue.cc:237] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:26.760638 19074 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:20:26.755910 12673 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:26.774669 19185 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:26.775305 19185 raft_consensus.cc:697] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:26.777064 19211 raft_consensus.cc:1081] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Signalling peer 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:26.765147 19177 tablet_copy_client.cc:670] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:26.790249 12644 tablet_copy_service.cc:342] P 19cd864c053744bebd96093a7982537e: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-c4e4ae9d38ba4fd0a700998e4f4750be received from {username='slave'} at 127.0.0.1:39590
I20260709 13:20:26.790781 12644 tablet_copy_service.cc:434] P 19cd864c053744bebd96093a7982537e: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-c4e4ae9d38ba4fd0a700998e4f4750be on tablet c4e4ae9d38ba4fd0a700998e4f4750be with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:26.776276 19185 consensus_queue.cc:237] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:26.803102 19208 raft_consensus.cc:993] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530: : Instructing follower 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:26.803623 19208 raft_consensus.cc:1081] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Signalling peer 19cd864c053744bebd96093a7982537e to start an election
W20260709 13:20:26.809257 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 49966.4us
I20260709 13:20:26.842413 12632 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "20db0f202ea546699789c2325d241b35"
dest_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:56832
I20260709 13:20:26.843212 12632 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:26.843679 12632 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:26.852622 12644 tablet_copy_service.cc:140] P 19cd864c053744bebd96093a7982537e: Received BeginTabletCopySession request for tablet 94e3333498524f14a17661cdb38ef144 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:39590)
I20260709 13:20:26.868692 12644 tablet_copy_service.cc:161] P 19cd864c053744bebd96093a7982537e: Beginning new tablet copy session on tablet 94e3333498524f14a17661cdb38ef144 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:39590: session id = 522fd2ba0e9f402691b5ef29af8579cd-94e3333498524f14a17661cdb38ef144
I20260709 13:20:26.860234 12793 tablet_copy_source_session.cc:215] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:26.897986 12633 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615"
dest_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:56838
I20260709 13:20:26.898619 12633 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:26.898972 12633 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:26.907451 12632 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:26.915726 12632 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:26.921669 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:26.922370 12704 raft_consensus.cc:3055] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:26.922672 12704 raft_consensus.cc:740] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:26.923398 12704 consensus_queue.cc:260] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:26.924511 12704 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:26.926570 12705 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:26.932104 12704 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:20:26.944609 12644 tablet_copy_source_session.cc:215] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:26.952952 19181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94e3333498524f14a17661cdb38ef144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:26.956897 13383 raft_consensus.cc:1217] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEARNER]: Deduplicated request from leader. Original: 20.12->[20.13-20.13]   Dedup: 20.13->[]
I20260709 13:20:26.971369 19180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 775128ed7b8d4283bf303c8d40247615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:26.979439 12598 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:26.983798 19213 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:26.984350 19213 raft_consensus.cc:697] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:26.985149 19213 consensus_queue.cc:237] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:26.989337 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:26.990603 12780 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:26.996240 19177 tablet_copy_client.cc:538] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:27.011317 12530 catalog_manager.cc:5697] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 14 to 15, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 15 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:27.012852 12523 catalog_manager.cc:5697] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 17 to 18, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 18 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:27.033094 19177 tablet_bootstrap.cc:492] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:20:26.989694 12633 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:27.047138 12780 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:20:27.133097 12627 raft_consensus.cc:1240] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:27.133709 12776 raft_consensus.cc:1240] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: []
W20260709 13:20:27.049717 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 58757.7us
I20260709 13:20:27.145695 19181 tablet_copy_client.cc:806] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:27.146473 19181 tablet_copy_client.cc:670] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:27.158751 19181 tablet_copy_client.cc:538] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:27.170805 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:27.171386 19181 tablet_bootstrap.cc:492] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:20:27.173959 12525 catalog_manager.cc:5697] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 17 to 18, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 18 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:27.186512 12633 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:27.194914 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:27.195544 12782 raft_consensus.cc:3055] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:27.195842 12782 raft_consensus.cc:740] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:27.196913 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:27.197975 12700 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:27.196630 12782 consensus_queue.cc:260] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:27.200269 12782 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:27.200368 19180 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:51768 (request call id 1248) took 1055 ms. Trace:
I20260709 13:20:27.200841 19180 rpcz_store.cc:276] 0709 13:20:26.145406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.145576 (+   170us) service_pool.cc:224] Handling call
0709 13:20:26.179845 (+ 34269us) ts_tablet_manager.cc:934] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
0709 13:20:26.969611 (+789766us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:20:27.073357 (+103746us) tablet_metadata.cc:665] Metadata flushed
0709 13:20:27.073470 (+   113us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:20:27.199652 (+126182us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:20:27.199951 (+   299us) tablet_replica.cc:206] Creating consensus instance
0709 13:20:27.200226 (+   275us) ts_tablet_manager.cc:989] Replica registered.
0709 13:20:27.200286 (+    60us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":3058,"tablet-copy.queue_time_us":33225,"thread_start_us":116466,"threads_started":2}
I20260709 13:20:27.203591 19180 tablet_copy_client.cc:806] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:27.204195 19180 tablet_copy_client.cc:670] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:27.287658 19181 tablet_bootstrap.cc:700] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent opening tablet: real 0.111s	user 0.006s	sys 0.001s
I20260709 13:20:27.290011 12700 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:20:27.299252 12598 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:27.301369 19220 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:27.302095 19220 raft_consensus.cc:697] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:27.303879 12782 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:20:27.303237 19220 consensus_queue.cc:237] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
W20260709 13:20:27.311409 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:27.324545 19180 tablet_copy_client.cc:538] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:27.337320 19180 tablet_bootstrap.cc:492] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:20:27.349145 12633 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:20:27.355562 19141 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:27.397044 19177 tablet_bootstrap.cc:492] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=1 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:27.410388 19177 tablet_bootstrap.cc:492] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:20:27.413205 19177 ts_tablet_manager.cc:1403] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.380s	user 0.058s	sys 0.001s
I20260709 13:20:27.414934 12700 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:20:27.416383 19205 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:27.421837 19177 raft_consensus.cc:359] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:27.423205 19177 raft_consensus.cc:740] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:20:27.424290 19177 consensus_queue.cc:260] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:27.435897 12777 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:20:27.441949 12633 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:20:27.448064 19211 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:27.473441 19177 ts_tablet_manager.cc:1434] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.060s	user 0.007s	sys 0.000s
I20260709 13:20:27.470518 19141 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:20:27.482345 12780 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:20:27.486959 19220 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:27.556968 13385 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:20:27.567811 13377 raft_consensus.cc:1217] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 LEARNER]: Deduplicated request from leader. Original: 15.9->[15.10-15.10]   Dedup: 15.10->[]
I20260709 13:20:27.576622 12792 tablet_copy_service.cc:342] P 3d539f8d95d343d8b1696803f18b30d2: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-49dad260ffe242d2b4e0976f119a19d7 received from {username='slave'} at 127.0.0.1:54346
I20260709 13:20:27.621685 19193 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:20:27.680756 12523 catalog_manager.cc:5697] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 16 to 17, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 17 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:27.702953 12792 tablet_copy_service.cc:434] P 3d539f8d95d343d8b1696803f18b30d2: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-49dad260ffe242d2b4e0976f119a19d7 on tablet 49dad260ffe242d2b4e0976f119a19d7 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:27.788686 19237 raft_consensus.cc:1064] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
W20260709 13:20:27.883286 19181 log.cc:927] Time spent T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.079s	user 0.005s	sys 0.000s
W20260709 13:20:27.928943 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 162998us
W20260709 13:20:27.974202 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 56508.1us
W20260709 13:20:27.986068 19243 log.cc:927] Time spent T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:20:28.025223 19237 consensus_queue.cc:237] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 11, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:28.163163 19187 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.164059 19187 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:28.237390 12707 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:20:28.249132 19187 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:28.269291 19186 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:28.277065 19186 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:28.283131 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:28.290220 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:28.331427 19181 log.cc:927] Time spent T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:20:28.355811 12780 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:20:28.316283 19186 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:28.370795 19181 tablet_bootstrap.cc:492] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=1 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:28.372315 19181 tablet_bootstrap.cc:492] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:20:28.373469 19181 ts_tablet_manager.cc:1403] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 1.202s	user 0.052s	sys 0.020s
W20260709 13:20:28.375913 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 74846.3us
I20260709 13:20:28.383216 19074 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:28.392885 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:28.399636 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:28.417605 19239 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:20:28.419169 12597 consensus_peers.cc:597] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:28.438177 13348 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:28.479202 12782 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:20:28.490154 19186 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:20:28.491611 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 59860.8us
I20260709 13:20:28.492482 13348 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:28.493485 19186 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:28.494549 19181 raft_consensus.cc:359] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:28.495522 19181 raft_consensus.cc:740] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:20:28.496207 19181 consensus_queue.cc:260] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:28.538587 13376 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEARNER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:20:28.545684 19181 ts_tablet_manager.cc:1434] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.172s	user 0.001s	sys 0.009s
I20260709 13:20:28.555351 19237 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:28.555907 12644 tablet_copy_service.cc:342] P 19cd864c053744bebd96093a7982537e: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-94e3333498524f14a17661cdb38ef144 received from {username='slave'} at 127.0.0.1:39590
I20260709 13:20:28.568898 12644 tablet_copy_service.cc:434] P 19cd864c053744bebd96093a7982537e: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-94e3333498524f14a17661cdb38ef144 on tablet 94e3333498524f14a17661cdb38ef144 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:28.556716 19157 raft_consensus.cc:1064] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:20:28.582659 19245 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:28.572974 19157 consensus_queue.cc:237] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 8, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:28.600767 19180 tablet_bootstrap.cc:492] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=1 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:28.609925 12705 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:20:28.647420 19180 tablet_bootstrap.cc:492] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:20:28.666435 19180 ts_tablet_manager.cc:1403] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 1.329s	user 0.045s	sys 0.027s
I20260709 13:20:28.698683 19239 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:28.706370 19180 raft_consensus.cc:359] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:28.709450 19180 raft_consensus.cc:740] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:20:28.712303 19180 consensus_queue.cc:260] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:28.720912 19180 ts_tablet_manager.cc:1434] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.054s	user 0.009s	sys 0.000s
I20260709 13:20:28.725281 12792 tablet_copy_service.cc:342] P 3d539f8d95d343d8b1696803f18b30d2: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-775128ed7b8d4283bf303c8d40247615 received from {username='slave'} at 127.0.0.1:54346
I20260709 13:20:28.725953 12792 tablet_copy_service.cc:434] P 3d539f8d95d343d8b1696803f18b30d2: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-775128ed7b8d4283bf303c8d40247615 on tablet 775128ed7b8d4283bf303c8d40247615 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:28.744346 12708 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:20:28.744390 13376 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 LEARNER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:20:28.750322 19248 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:28.760425 12627 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:20:28.770664 12781 raft_consensus.cc:2955] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 13 to 14, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:28.764425 19237 raft_consensus.cc:2955] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Committing config change with OpId 20.14: config changed from index 13 to 14, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:28.759986 19157 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:20:28.795159 19246 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:28.828637 13378 raft_consensus.cc:1217] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 LEARNER]: Deduplicated request from leader. Original: 19.11->[19.12-19.12]   Dedup: 19.12->[]
I20260709 13:20:28.843705 19157 raft_consensus.cc:2955] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Committing config change with OpId 15.11: config changed from index 10 to 11, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:28.859799 13383 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEARNER]: Advancing to term 17
I20260709 13:20:28.859077 13378 raft_consensus.cc:2955] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 13 to 14, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:28.867089 12703 raft_consensus.cc:2955] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:28.894606 13376 raft_consensus.cc:2955] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:28.924861 12530 catalog_manager.cc:5697] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index 13 to 14, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 20 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:28.942489 12525 catalog_manager.cc:5697] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 10 to 11, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 15 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:20:28.997507 19273 log.cc:927] Time spent T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.103s	user 0.012s	sys 0.005s
W20260709 13:20:28.998818 19268 log.cc:927] Time spent T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.092s	user 0.004s	sys 0.016s
W20260709 13:20:29.000167 19280 log.cc:927] Time spent T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.082s	user 0.001s	sys 0.005s
I20260709 13:20:28.980208 12708 raft_consensus.cc:2955] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 13 to 14, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:20:29.002971 19279 log.cc:927] Time spent T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.071s	user 0.012s	sys 0.000s
I20260709 13:20:29.020576 12627 raft_consensus.cc:2955] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:20:29.024957 19278 log.cc:927] Time spent T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.123s	user 0.005s	sys 0.018s
I20260709 13:20:29.103374 12782 consensus_queue.cc:237] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:29.111812 13383 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:29.114070 12703 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:29.116557 12627 consensus_queue.cc:237] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:29.118225 19246 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:29.121814 19157 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:29.126808 13383 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:20:29.130113 19215 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:20:29.129796 19246 raft_consensus.cc:2955] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:29.133217 12709 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:20:29.143096 19220 raft_consensus.cc:2955] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Committing config change with OpId 20.15: config changed from index 14 to 15, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:29.147698 13383 raft_consensus.cc:2955] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 14 to 15, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:29.149103 12708 raft_consensus.cc:2955] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:29.156364 13376 raft_consensus.cc:2955] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:29.160104 19237 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:20:29.170024 12703 raft_consensus.cc:2955] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 14 to 15, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:29.185504 12516 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 49dad260ffe242d2b4e0976f119a19d7 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:20:29.186571 12530 catalog_manager.cc:5697] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 14 to 15, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New cstate: current_term: 20 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:29.197445 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c4e4ae9d38ba4fd0a700998e4f4750be with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:20:29.211648 12773 tablet_service.cc:1558] Processing DeleteTablet for tablet c4e4ae9d38ba4fd0a700998e4f4750be with delete_type TABLET_DATA_TOMBSTONED (TS 3d539f8d95d343d8b1696803f18b30d2 not found in new config with opid_index 15) from {username='slave'} at 127.0.0.1:50664
I20260709 13:20:29.235008 12531 catalog_manager.cc:5697] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index 11 to 12, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New cstate: current_term: 15 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:29.252476 19215 raft_consensus.cc:1064] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:20:29.268296 19215 consensus_queue.cc:237] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 10, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:29.307096 19284 tablet_replica.cc:333] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:20:29.371480 19284 raft_consensus.cc:2243] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:29.372409 19284 raft_consensus.cc:2272] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:29.384049 12624 tablet_service.cc:1558] Processing DeleteTablet for tablet 49dad260ffe242d2b4e0976f119a19d7 with delete_type TABLET_DATA_TOMBSTONED (TS 19cd864c053744bebd96093a7982537e not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:56830
I20260709 13:20:29.404953 19284 ts_tablet_manager.cc:1916] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:29.453006 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:20:29.463558 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 77253.1us
I20260709 13:20:29.505199 19288 tablet_replica.cc:333] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:20:29.512310 19288 raft_consensus.cc:2243] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Raft consensus shutting down.
W20260709 13:20:29.523973 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 65252us
I20260709 13:20:29.596069 19288 raft_consensus.cc:2272] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:29.658380 19284 ts_tablet_manager.cc:1929] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 20.14
I20260709 13:20:29.667379 19220 raft_consensus.cc:1064] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:20:29.678037 19288 ts_tablet_manager.cc:1916] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:29.679801 13376 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 LEARNER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:20:29.685748 12703 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:20:29.746659 12782 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:20:29.717790 19284 log.cc:1199] T c4e4ae9d38ba4fd0a700998e4f4750be P 3d539f8d95d343d8b1696803f18b30d2: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-2-root/wals/c4e4ae9d38ba4fd0a700998e4f4750be
I20260709 13:20:29.771281 12516 catalog_manager.cc:5028] TS 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): tablet c4e4ae9d38ba4fd0a700998e4f4750be (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
W20260709 13:20:29.823724 12516 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 54818.5us
W20260709 13:20:29.824771 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 53850.9us
W20260709 13:20:29.850417 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 163403us
I20260709 13:20:29.769981 19238 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 13:20:29.869796 19220 consensus_queue.cc:237] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 11, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:29.874485 19305 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:29.880246 19245 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:29.974211 13376 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 LEARNER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:20:29.981042 19296 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 13:20:30.010999 12782 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:20:30.015302 19311 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:30.055150 13383 raft_consensus.cc:2955] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 12 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:30.055562 19311 raft_consensus.cc:2955] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Committing config change with OpId 19.13: config changed from index 12 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:30.086254 12530 catalog_manager.cc:5697] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 12 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 19 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:30.053416 19288 ts_tablet_manager.cc:1929] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.11
I20260709 13:20:30.096910 19288 log.cc:1199] T 49dad260ffe242d2b4e0976f119a19d7 P 19cd864c053744bebd96093a7982537e: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-0-root/wals/49dad260ffe242d2b4e0976f119a19d7
I20260709 13:20:30.125886 12517 catalog_manager.cc:5028] TS 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): tablet 49dad260ffe242d2b4e0976f119a19d7 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:20:30.156989 19296 raft_consensus.cc:2955] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Committing config change with OpId 17.13: config changed from index 11 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:30.166540 12778 raft_consensus.cc:2955] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 11 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:30.186757 12708 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:20:30.214476 19305 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:20:30.228379 19316 log.cc:927] Time spent T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.083s	user 0.003s	sys 0.000s
I20260709 13:20:30.242311 12531 catalog_manager.cc:5697] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index 11 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 17 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:30.246315 13383 raft_consensus.cc:2955] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 11 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:30.266582 12704 raft_consensus.cc:2955] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 12 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:30.274636 12782 raft_consensus.cc:2955] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 12 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:30.319244 12708 raft_consensus.cc:2955] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 11 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:30.318172 12629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:30.324777 12629 raft_consensus.cc:606] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 23 LEADER]: Received request to transfer leadership to TS 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:30.347922 12704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:20:30.348605 12704 raft_consensus.cc:606] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:20:30.362751 12778 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:20:30.363245 12778 raft_consensus.cc:606] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 LEADER]: Received request to transfer leadership to TS 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:30.378651 12629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:30.379405 12629 raft_consensus.cc:606] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 22 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:30.381903 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:20:30.383665 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:30.479240 12634 consensus_queue.cc:237] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:30.483724 12626 consensus_queue.cc:237] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:30.489128 13383 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:20:30.490789 19311 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:30.491079 13383 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:20:30.492705 19311 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:30.495410 12782 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:20:30.499410 19311 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:30.508936 19215 raft_consensus.cc:2955] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Committing config change with OpId 19.14: config changed from index 13 to 14, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:30.518008 12782 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:20:30.536989 19307 raft_consensus.cc:993] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2: : Instructing follower 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:20:30.541226 19247 raft_consensus.cc:1081] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 LEADER]: Signalling peer 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:20:30.540555 13385 raft_consensus.cc:2955] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Committing config change with OpId 17.14: config changed from index 13 to 14, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:30.513852 19237 raft_consensus.cc:2955] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Committing config change with OpId 17.14: config changed from index 13 to 14, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:30.539909 13377 raft_consensus.cc:2955] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 13 to 14, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:30.553452 13376 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82"
dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:51768
I20260709 13:20:30.561795 13376 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:30.562191 13376 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:30.550582 19305 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:30.566404 12778 raft_consensus.cc:2955] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Committing config change with OpId 17.14: config changed from index 13 to 14, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:30.574436 13376 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:30.578099 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:30.578696 12780 raft_consensus.cc:3055] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:20:30.578994 12780 raft_consensus.cc:740] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:30.579763 12780 consensus_queue.cc:260] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:30.580825 12780 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:30.592000 12780 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 23.
I20260709 13:20:30.595842 12782 raft_consensus.cc:2955] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 13 to 14, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:30.603266 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:30.614452 13376 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:30.617488 13348 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:30.625613 12704 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:30.639451 12704 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 23.
I20260709 13:20:30.648384 19186 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:20:30.657912 12528 catalog_manager.cc:5697] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 13 to 14, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New cstate: current_term: 17 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:30.678615 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 94e3333498524f14a17661cdb38ef144 with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:20:30.680418 19312 raft_consensus.cc:993] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e: : Instructing follower 639ab2db265f4747bb7afb93e833f530 to start an election
I20260709 13:20:30.680956 19311 raft_consensus.cc:1081] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 23 LEADER]: Signalling peer 639ab2db265f4747bb7afb93e833f530 to start an election
I20260709 13:20:30.683058 12707 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:42500
I20260709 13:20:30.683563 12707 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:30.683905 12707 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:30.692041 12695 tablet_service.cc:1558] Processing DeleteTablet for tablet 775128ed7b8d4283bf303c8d40247615 with delete_type TABLET_DATA_TOMBSTONED (TS 639ab2db265f4747bb7afb93e833f530 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:49378
I20260709 13:20:30.665277 19186 raft_consensus.cc:697] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:20:30.708890 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 61170.6us
I20260709 13:20:30.713250 19186 consensus_queue.cc:237] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:30.717373 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 775128ed7b8d4283bf303c8d40247615 with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:20:30.704576 12528 catalog_manager.cc:5697] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index 13 to 14, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New cstate: current_term: 19 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:30.722692 12707 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:30.729321 19289 raft_consensus.cc:993] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530: : Instructing follower 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:30.729979 19291 raft_consensus.cc:1081] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Signalling peer 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:30.731799 12629 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce"
dest_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:56832
I20260709 13:20:30.732321 12629 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:30.732613 12629 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:30.755193 12692 tablet_service.cc:1558] Processing DeleteTablet for tablet 94e3333498524f14a17661cdb38ef144 with delete_type TABLET_DATA_TOMBSTONED (TS 639ab2db265f4747bb7afb93e833f530 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:49378
I20260709 13:20:30.769829 19311 raft_consensus.cc:993] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e: : Instructing follower 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:20:30.770444 19311 raft_consensus.cc:1081] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 22 LEADER]: Signalling peer 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:20:30.777801 12782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:36488
I20260709 13:20:30.778641 12782 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:30.779115 12782 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:30.789592 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:30.790228 12634 raft_consensus.cc:3055] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:20:30.790516 12634 raft_consensus.cc:740] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:30.791276 12634 consensus_queue.cc:260] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:30.792446 12634 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:30.798157 19326 tablet_replica.cc:333] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:20:30.800464 12634 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 24.
I20260709 13:20:30.802309 12673 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:30.803119 19290 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:20:30.799495 19326 raft_consensus.cc:2243] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:30.803679 19290 raft_consensus.cc:697] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 24 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:30.804986 19326 raft_consensus.cc:2272] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:30.804615 19290 consensus_queue.cc:237] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:30.806962 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: true dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:30.807905 13377 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:30.806389 12528 catalog_manager.cc:5697] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 22 to 23, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 23 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:30.813846 12707 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:30.815586 12782 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:30.817166 12782 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:30.787673 12629 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:30.831197 12629 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:30.832521 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:30.816560 13377 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 24.
I20260709 13:20:30.819591 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:30.833273 12782 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:30.833491 12634 raft_consensus.cc:3055] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:20:30.833930 12634 raft_consensus.cc:740] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:30.834677 12634 consensus_queue.cc:260] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:30.832619 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:30.819267 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:30.836948 12705 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:30.838877 12707 raft_consensus.cc:3055] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:30.839188 12707 raft_consensus.cc:740] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:30.841755 12634 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:30.848400 12782 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 20.
I20260709 13:20:30.848424 12707 consensus_queue.cc:260] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:30.860839 12597 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:30.861747 19312 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:30.862926 19312 raft_consensus.cc:697] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:30.866546 19312 consensus_queue.cc:237] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:30.869717 12707 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:30.865285 12634 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 23.
I20260709 13:20:30.877501 12705 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 23.
I20260709 13:20:30.880754 19326 ts_tablet_manager.cc:1916] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:30.890861 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:30.894061 19329 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:20:30.895262 19329 raft_consensus.cc:697] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 23 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:30.896134 19329 consensus_queue.cc:237] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:30.904075 12707 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 20.
I20260709 13:20:30.906057 12528 catalog_manager.cc:5697] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 23 to 24, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 24 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:30.949260 12528 catalog_manager.cc:5697] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 19 to 20, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 20 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:31.023205 19326 ts_tablet_manager.cc:1929] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 17.13
I20260709 13:20:31.028025 19326 log.cc:1199] T 775128ed7b8d4283bf303c8d40247615 P 639ab2db265f4747bb7afb93e833f530: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-1-root/wals/775128ed7b8d4283bf303c8d40247615
I20260709 13:20:31.030278 12517 catalog_manager.cc:5028] TS 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): tablet 775128ed7b8d4283bf303c8d40247615 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:20:31.031944 19326 tablet_replica.cc:333] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:20:31.039283 19326 raft_consensus.cc:2243] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:31.040311 19326 raft_consensus.cc:2272] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:31.048434 19326 ts_tablet_manager.cc:1916] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:31.073220 19326 ts_tablet_manager.cc:1929] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 19.13
I20260709 13:20:31.073866 19326 log.cc:1199] T 94e3333498524f14a17661cdb38ef144 P 639ab2db265f4747bb7afb93e833f530: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-1-root/wals/94e3333498524f14a17661cdb38ef144
I20260709 13:20:31.075765 12517 catalog_manager.cc:5028] TS 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): tablet 94e3333498524f14a17661cdb38ef144 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:20:31.060249 12528 catalog_manager.cc:5697] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 22 to 23, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 23 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:31.165623 12778 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:20:31.176100 19323 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:20:31.278913 12707 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:20:31.287762 19323 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:20:31.369977 13377 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:20:31.371477 19332 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:20:31.384707 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:31.385905 12629 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:20:31.387470 19291 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:20:31.434055 12780 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:20:31.464335 19237 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:31.478056 12704 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:20:31.480064 19237 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:31.556507 12634 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:20:31.573885 19214 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:20:31.581344 12707 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:20:31.598629 19214 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:20:31.678551 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 53240.7us
W20260709 13:20:31.760859 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 125668us
W20260709 13:20:31.811156 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 56380.9us
W20260709 13:20:31.830534 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 72519.8us
W20260709 13:20:31.824590 19340 log.cc:927] Time spent T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
W20260709 13:20:31.929693 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 67996.6us
W20260709 13:20:32.026410 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 50394.4us
W20260709 13:20:32.236258 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 152555us
W20260709 13:20:32.301707 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 215134us
I20260709 13:20:32.460029 12707 consensus_queue.cc:237] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:32.490924 12708 consensus_queue.cc:237] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
W20260709 13:20:32.585700 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.598258 12626 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:20:32.608686 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.606005 19348 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:20:32.639804 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:32.668563 12778 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:20:32.672374 12781 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:20:32.683100 19291 raft_consensus.cc:2955] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Committing config change with OpId 15.10: config changed from index -1 to 10, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:32.686357 12630 raft_consensus.cc:2955] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index -1 to 10, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:32.718138 19347 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:32.722322 19347 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:32.728863 13377 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:20:32.732921 12705 consensus_queue.cc:237] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
W20260709 13:20:32.742965 12672 consensus_peers.cc:597] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 199931c4ffc44fee980d8bb49cf37f00. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:32.739714 12528 catalog_manager.cc:5697] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index -1 to 10, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 15 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:32.765214 19347 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:32.782356 12780 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:20:32.783463 12630 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:20:32.791534 12781 raft_consensus.cc:2955] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index -1 to 10, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:32.801874 12707 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:20:32.802575 12707 raft_consensus.cc:606] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:20:32.789469 19252 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 13:20:32.817876 19289 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 13:20:32.833103 12780 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:20:32.833819 12780 raft_consensus.cc:606] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:20:32.859016 12630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fa34731f37084c608660f0088a2dba09"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:32.859760 12630 raft_consensus.cc:606] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:32.868541 19360 raft_consensus.cc:993] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: : Instructing follower 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:32.869143 19394 raft_consensus.cc:1081] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Signalling peer 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:32.876820 12630 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e"
dest_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:56838
I20260709 13:20:32.877406 12630 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:32.877728 12630 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:32.889585 12630 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:32.892086 12630 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:32.894362 12672 consensus_peers.cc:597] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 18b2f67fae7f4963a4b4c562003df2fe. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:32.894830 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 57853.6us
I20260709 13:20:32.895952 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: true dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:32.896667 13380 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:32.902788 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:32.903426 12780 raft_consensus.cc:3055] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:20:32.903730 12780 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:32.904647 12780 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 20.18, Last appended by leader: 18, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:32.905753 12780 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:20:32.928115 19404 log.cc:927] Time spent T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:20:32.937367 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:20:32.939255 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:32.948540 12780 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 21.
I20260709 13:20:32.951371 13380 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 21.
I20260709 13:20:32.950912 12597 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:32.969892 19361 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:32.989707 19361 raft_consensus.cc:697] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 21 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:33.001647 19361 consensus_queue.cc:237] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 20.18, Last appended by leader: 18, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:33.010047 19252 raft_consensus.cc:2955] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 20 LEADER]: Committing config change with OpId 20.13: config changed from index -1 to 13, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:33.021234 12633 raft_consensus.cc:2955] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index -1 to 13, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:33.022974 19289 raft_consensus.cc:2955] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Committing config change with OpId 19.13: config changed from index 5 to 13, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:33.025872 13378 raft_consensus.cc:2955] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 5 to 13, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:33.027702 12778 raft_consensus.cc:2955] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 5 to 13, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:20:33.057075 12579 auto_rebalancer.cc:629] Failed to schedule move for tablet c6cd31a55742463db9f0f0060bb60a77: Remote error: Service unavailable: leader transfer in progress
W20260709 13:20:33.061951 12579 auto_rebalancer.cc:629] Failed to schedule move for tablet fa34731f37084c608660f0088a2dba09: Remote error: Service unavailable: leader transfer in progress
I20260709 13:20:33.063431 12530 catalog_manager.cc:5697] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 5 to 13, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New cstate: current_term: 19 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:20:33.061965 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 55382.5us
I20260709 13:20:33.079275 12776 raft_consensus.cc:2955] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index -1 to 13, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:33.124620 19291 raft_consensus.cc:993] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530: : Instructing follower 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:33.125692 19347 raft_consensus.cc:1081] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Signalling peer 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:33.136345 12633 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77"
dest_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:56832
I20260709 13:20:33.136917 12633 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:33.137259 12633 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:33.163255 12624 tablet_service.cc:1558] Processing DeleteTablet for tablet 4c13fa2aff324f65ad0bf6191ea68fab with delete_type TABLET_DATA_TOMBSTONED (TS 19cd864c053744bebd96093a7982537e not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:56830
I20260709 13:20:33.168329 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4c13fa2aff324f65ad0bf6191ea68fab with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 13)
I20260709 13:20:33.170220 19413 ts_tablet_manager.cc:933] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:33.174822 12633 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:33.178548 12633 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:33.180045 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:33.180727 12705 raft_consensus.cc:3055] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:33.181092 12705 raft_consensus.cc:740] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:33.181932 12705 consensus_queue.cc:260] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:33.183060 12705 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:33.184849 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:33.185595 12778 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:33.195051 19385 raft_consensus.cc:993] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: : Instructing follower 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:20:33.194389 12531 catalog_manager.cc:5697] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index -1 to 13, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 20 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:33.195710 19353 raft_consensus.cc:1081] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Signalling peer 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:20:33.200177 12780 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fa34731f37084c608660f0088a2dba09"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:36488
I20260709 13:20:33.200738 12780 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:33.201114 12780 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:33.208559 12780 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:33.213063 12780 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:33.215677 12525 catalog_manager.cc:5697] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 20 to 21, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 21 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:33.223584 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:33.224371 12707 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:33.225607 12705 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:20:33.228066 19413 tablet_copy_client.cc:323] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.66:45173
I20260709 13:20:33.229797 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:33.230471 12626 raft_consensus.cc:3055] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:33.230860 12626 raft_consensus.cc:740] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:33.232146 12626 consensus_queue.cc:260] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:33.230700 12718 tablet_copy_service.cc:140] P 639ab2db265f4747bb7afb93e833f530: Received BeginTabletCopySession request for tablet 199931c4ffc44fee980d8bb49cf37f00 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:38496)
I20260709 13:20:33.234117 12718 tablet_copy_service.cc:161] P 639ab2db265f4747bb7afb93e833f530: Beginning new tablet copy session on tablet 199931c4ffc44fee980d8bb49cf37f00 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:38496: session id = 522fd2ba0e9f402691b5ef29af8579cd-199931c4ffc44fee980d8bb49cf37f00
I20260709 13:20:33.238255 12598 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:33.239140 19353 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:33.239627 19353 raft_consensus.cc:697] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:33.240568 19353 consensus_queue.cc:237] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:33.244483 12778 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:20:33.233358 12626 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:33.310763 12707 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:20:33.314596 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:33.330634 12525 catalog_manager.cc:5697] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 17 to 18, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 18 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:33.339284 19359 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:33.339923 19359 raft_consensus.cc:697] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:33.340966 19359 consensus_queue.cc:237] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:33.349824 12626 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:20:33.375216 12718 tablet_copy_source_session.cc:215] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:33.382133 19414 tablet_replica.cc:333] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:20:33.382521 19413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 199931c4ffc44fee980d8bb49cf37f00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:33.383219 19414 raft_consensus.cc:2243] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:33.384219 19414 raft_consensus.cc:2272] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:33.398818 19414 ts_tablet_manager.cc:1916] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:33.456606 19418 ts_tablet_manager.cc:933] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:33.471340 12525 catalog_manager.cc:5697] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 18 to 19, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 19 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:33.492542 19414 ts_tablet_manager.cc:1929] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 19.12
I20260709 13:20:33.493165 19414 log.cc:1199] T 4c13fa2aff324f65ad0bf6191ea68fab P 19cd864c053744bebd96093a7982537e: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-0-root/wals/4c13fa2aff324f65ad0bf6191ea68fab
I20260709 13:20:33.495925 19413 tablet_copy_client.cc:806] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:33.496531 19413 tablet_copy_client.cc:670] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:33.501926 12517 catalog_manager.cc:5028] TS 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): tablet 4c13fa2aff324f65ad0bf6191ea68fab (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:20:33.509145 19418 tablet_copy_client.cc:323] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.66:45173
I20260709 13:20:33.515445 12718 tablet_copy_service.cc:140] P 639ab2db265f4747bb7afb93e833f530: Received BeginTabletCopySession request for tablet 18b2f67fae7f4963a4b4c562003df2fe from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:38496)
I20260709 13:20:33.515967 19413 tablet_copy_client.cc:538] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:33.516005 12718 tablet_copy_service.cc:161] P 639ab2db265f4747bb7afb93e833f530: Beginning new tablet copy session on tablet 18b2f67fae7f4963a4b4c562003df2fe from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:38496: session id = 522fd2ba0e9f402691b5ef29af8579cd-18b2f67fae7f4963a4b4c562003df2fe
I20260709 13:20:33.522627 12718 tablet_copy_source_session.cc:215] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:33.530419 19418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18b2f67fae7f4963a4b4c562003df2fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:33.563653 19413 tablet_bootstrap.cc:492] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:20:33.570855 13378 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 20 index: 18. Preceding OpId from leader: term: 21 index: 19. (index mismatch)
I20260709 13:20:33.586016 19385 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:20:33.610572 19418 tablet_copy_client.cc:806] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:33.611265 19418 tablet_copy_client.cc:670] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:33.615446 12780 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 20 index: 18. Preceding OpId from leader: term: 21 index: 19. (index mismatch)
I20260709 13:20:33.620555 19365 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:20:33.634534 19418 tablet_copy_client.cc:538] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:33.689168 19418 tablet_bootstrap.cc:492] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:20:33.735015 12707 raft_consensus.cc:1275] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:20:33.740523 19359 consensus_queue.cc:1048] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:33.760408 12778 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:20:33.766885 19361 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:33.820751 12708 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:20:33.825651 19353 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:33.833051 12626 raft_consensus.cc:1275] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:20:33.836198 19408 consensus_queue.cc:1048] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:33.940241 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:34.020253 19413 tablet_bootstrap.cc:492] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=1 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:34.021260 19413 tablet_bootstrap.cc:492] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:20:34.045104 19413 ts_tablet_manager.cc:1403] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.494s	user 0.035s	sys 0.032s
I20260709 13:20:34.053010 19413 raft_consensus.cc:359] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:34.061450 19413 raft_consensus.cc:740] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:20:34.062202 19413 consensus_queue.cc:260] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:34.069453 19413 ts_tablet_manager.cc:1434] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:20:34.072288 12718 tablet_copy_service.cc:342] P 639ab2db265f4747bb7afb93e833f530: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-199931c4ffc44fee980d8bb49cf37f00 received from {username='slave'} at 127.0.0.1:38496
I20260709 13:20:34.072778 12718 tablet_copy_service.cc:434] P 639ab2db265f4747bb7afb93e833f530: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-199931c4ffc44fee980d8bb49cf37f00 on tablet 199931c4ffc44fee980d8bb49cf37f00 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:34.073942 19418 tablet_bootstrap.cc:492] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=2 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:34.075009 19418 tablet_bootstrap.cc:492] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:20:34.075778 19418 ts_tablet_manager.cc:1403] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.387s	user 0.042s	sys 0.017s
I20260709 13:20:34.110836 19418 raft_consensus.cc:359] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:34.123796 19418 raft_consensus.cc:740] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:20:34.124471 19418 consensus_queue.cc:260] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:34.130324 19418 ts_tablet_manager.cc:1434] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.003s
I20260709 13:20:34.137107 12718 tablet_copy_service.cc:342] P 639ab2db265f4747bb7afb93e833f530: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-18b2f67fae7f4963a4b4c562003df2fe received from {username='slave'} at 127.0.0.1:38496
I20260709 13:20:34.137547 12718 tablet_copy_service.cc:434] P 639ab2db265f4747bb7afb93e833f530: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-18b2f67fae7f4963a4b4c562003df2fe on tablet 18b2f67fae7f4963a4b4c562003df2fe with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:34.290635 13378 raft_consensus.cc:1217] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 LEARNER]: Deduplicated request from leader. Original: 15.9->[15.10-15.10]   Dedup: 15.10->[]
I20260709 13:20:34.484521 13378 raft_consensus.cc:1217] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEARNER]: Deduplicated request from leader. Original: 20.12->[20.13-20.13]   Dedup: 20.13->[]
I20260709 13:20:34.887509 19347 raft_consensus.cc:1064] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:20:34.891918 19347 consensus_queue.cc:237] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 11, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:34.912618 19432 raft_consensus.cc:1064] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:20:34.915751 19432 consensus_queue.cc:237] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 8, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:34.919811 13376 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEARNER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:20:34.921286 12633 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:20:34.921296 19252 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:34.921414 12778 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:20:34.924889 19290 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:34.925820 19252 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:34.970243 19290 raft_consensus.cc:2955] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 20 LEADER]: Committing config change with OpId 20.14: config changed from index 13 to 14, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:20:34.986815 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 62556.1us
I20260709 13:20:34.995028 12776 raft_consensus.cc:2955] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 13 to 14, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:35.013903 12778 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:20:35.019295 13376 raft_consensus.cc:2955] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 13 to 14, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:35.026185 13378 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 LEARNER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:20:35.055639 12530 catalog_manager.cc:5697] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index 13 to 14, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 20 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:35.063165 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:35.071966 12630 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:20:35.081118 12633 raft_consensus.cc:2955] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 13 to 14, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:35.047425 19434 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:20:35.130365 19252 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:35.259385 19347 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:20:35.279235 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 173369us
W20260709 13:20:35.267591 19441 log.cc:927] Time spent T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 13:20:35.376264 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 64399.9us
W20260709 13:20:35.470088 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 162391us
W20260709 13:20:35.534752 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 229482us
W20260709 13:20:35.643041 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 167610us
W20260709 13:20:35.706640 19460 log.cc:927] Time spent T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:20:35.805299 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 61488.9us
W20260709 13:20:35.853173 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 202815us
W20260709 13:20:35.899982 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 56032.9us
W20260709 13:20:36.020488 12672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.052s	sys 0.094s
W20260709 13:20:36.020915 12672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.052s	sys 0.094s
W20260709 13:20:36.021190 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 512999us
I20260709 13:20:36.023327 19482 raft_consensus.cc:2955] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Committing config change with OpId 15.11: config changed from index 10 to 11, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:20:36.037744 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 496759us
W20260709 13:20:36.053730 12749 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.005s	sys 0.018s
W20260709 13:20:36.054177 12749 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.005s	sys 0.018s
W20260709 13:20:36.055573 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 44571.2us
I20260709 13:20:36.063154 19484 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:36.063154 12530 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54814 (request call id 273) took 1027 ms. Trace:
I20260709 13:20:36.064047 19484 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:36.066694 19484 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:36.067930 12781 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "866523bf2d2f442f971d504267ed200f"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:20:36.068722 12781 raft_consensus.cc:606] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Received request to transfer leadership to TS 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:36.071430 12626 raft_consensus.cc:2955] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:36.084249 12703 consensus_queue.cc:237] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:36.090508 19480 raft_consensus.cc:993] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2: : Instructing follower 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:20:36.091274 19440 raft_consensus.cc:1081] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Signalling peer 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:20:36.064051 12530 rpcz_store.cc:276] 0709 13:20:35.036089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:35.036247 (+   158us) service_pool.cc:224] Handling call
0709 13:20:36.063077 (+1026830us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:35.113941 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:35.114550 (+   609us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:35.114597 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:20:35.114616 (+    19us) tablet.cc:662] Decoding operations
0709 13:20:35.114957 (+   341us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:35.115034 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:35.115066 (+    32us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:20:35.115248 (+   182us) tablet.cc:701] Row locks acquired
0709 13:20:35.115266 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:20:35.115402 (+   136us) write_op.cc:270] Start()
0709 13:20:35.115518 (+   116us) write_op.cc:276] Timestamp: P: 1783603235115360 usec, L: 0
0709 13:20:35.115541 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:20:35.116369 (+   828us) log.cc:844] Serialized 595 byte log entry
0709 13:20:35.730057 (+613688us) spinlock_profiling.cc:243] Waited 9.51 ms on lock 0x7b18000f05e0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c53f3 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:20:36.011778 (+281721us) op_driver.cc:464] REPLICATION: finished
0709 13:20:36.026294 (+ 14516us) write_op.cc:301] APPLY: starting
0709 13:20:36.026386 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:36.027717 (+  1331us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:36.027747 (+    30us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:36.028519 (+   772us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:36.028745 (+   226us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:20:36.028802 (+    57us) write_op.cc:312] APPLY: finished
0709 13:20:36.029012 (+   210us) log.cc:844] Serialized 25 byte log entry
0709 13:20:36.058472 (+ 29460us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:36.058562 (+    90us) write_op.cc:454] Released schema lock
0709 13:20:36.058605 (+    43us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":260352,"child_traces":[["op",{"apply.queue_time_us":14244,"apply.run_cpu_time_us":3503,"apply.run_wall_time_us":37554,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8311,"num_ops":1,"prepare.queue_time_us":53798,"prepare.run_cpu_time_us":78772,"prepare.run_wall_time_us":132419,"raft.queue_time_us":276422,"raft.run_cpu_time_us":1112,"raft.run_wall_time_us":30346,"replication_time_us":896016,"spinlock_wait_cycles":21874432,"thread_start_us":344869,"threads_started":4,"wal-append.queue_time_us":142532}]]}
I20260709 13:20:36.110700 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:36.111100 12778 raft_consensus.cc:2955] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:36.112398 12630 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:20:36.113291 13385 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "866523bf2d2f442f971d504267ed200f"
dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:51768
I20260709 13:20:36.113837 13385 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:36.113881 19450 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:36.114238 13385 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:36.121028 19475 raft_consensus.cc:2955] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 20 LEADER]: Committing config change with OpId 20.15: config changed from index 14 to 15, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:36.136916 12633 raft_consensus.cc:2955] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 14 to 15, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:36.153743 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:36.156495 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 18b2f67fae7f4963a4b4c562003df2fe with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:20:36.159389 13385 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:36.161280 13385 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:36.190763 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 60290.6us
I20260709 13:20:36.195868 19408 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:36.196482 19408 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:36.195748 13385 raft_consensus.cc:2955] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:36.205189 13376 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:20:36.205943 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:36.206625 12707 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:36.207393 19408 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:20:36.208621 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 84839.2us
I20260709 13:20:36.218693 12530 catalog_manager.cc:5697] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index 10 to 11, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 15 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:36.220810 12707 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:20:36.232067 19450 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:36.240557 12633 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:36.231266 12747 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:36.242344 12633 raft_consensus.cc:606] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:36.249692 19408 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:36.255267 13349 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:36.260878 19459 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:36.275435 19459 raft_consensus.cc:697] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:36.276479 19459 consensus_queue.cc:237] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:36.277809 12633 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:36.278398 12633 raft_consensus.cc:606] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Received request to transfer leadership to TS 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:36.279729 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:20:36.281462 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:36.291810 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:36.291759 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:36.294066 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:36.294670 12782 raft_consensus.cc:3055] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:36.294955 12782 raft_consensus.cc:740] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:36.295641 12782 consensus_queue.cc:260] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:36.296890 12782 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:36.302291 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:36.304661 12782 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:20:36.306051 19466 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:36.315709 12707 consensus_queue.cc:237] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:36.322021 13376 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:36.332989 13385 raft_consensus.cc:2955] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 14 to 15, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:36.334205 19434 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:36.342724 12782 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:36.349898 19439 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:36.369729 19482 raft_consensus.cc:2955] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:36.415695 12778 raft_consensus.cc:2955] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:36.377372 12525 catalog_manager.cc:5697] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 14 to 15, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New cstate: current_term: 20 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:36.777441 12773 tablet_service.cc:1558] Processing DeleteTablet for tablet 18b2f67fae7f4963a4b4c562003df2fe with delete_type TABLET_DATA_TOMBSTONED (TS 3d539f8d95d343d8b1696803f18b30d2 not found in new config with opid_index 15) from {username='slave'} at 127.0.0.1:50664
W20260709 13:20:36.812114 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 350066us
I20260709 13:20:36.818527 19479 raft_consensus.cc:993] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e: : Instructing follower 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:20:36.819468 19479 raft_consensus.cc:1081] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Signalling peer 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:20:36.847859 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 199931c4ffc44fee980d8bb49cf37f00 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:20:36.853107 13378 raft_consensus.cc:2955] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:36.860793 12782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:36488
I20260709 13:20:36.861462 12782 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:36.861873 12782 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:36.862519 12528 catalog_manager.cc:5697] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index 11 to 12, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New cstate: current_term: 15 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:36.869324 12781 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:20:36.889045 12531 catalog_manager.cc:5697] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 15 to 16, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 16 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:36.928391 12782 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:36.950719 12782 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:36.953758 12707 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:20:36.964726 19492 tablet_replica.cc:333] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:20:36.966020 19492 raft_consensus.cc:2243] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:36.967133 19492 raft_consensus.cc:2272] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:36.978340 12624 tablet_service.cc:1558] Processing DeleteTablet for tablet 199931c4ffc44fee980d8bb49cf37f00 with delete_type TABLET_DATA_TOMBSTONED (TS 19cd864c053744bebd96093a7982537e not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:56830
I20260709 13:20:37.023592 19492 ts_tablet_manager.cc:1916] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:20:37.042536 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 133048us
I20260709 13:20:37.044152 19459 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:37.049110 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:37.049979 12626 raft_consensus.cc:3055] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:37.050417 12626 raft_consensus.cc:740] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:37.051616 12626 consensus_queue.cc:260] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:37.053141 12626 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:37.072508 19459 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:37.095908 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:37.096655 12709 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:37.097303 19481 raft_consensus.cc:993] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e: : Instructing follower 639ab2db265f4747bb7afb93e833f530 to start an election
I20260709 13:20:37.097936 19479 raft_consensus.cc:1081] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Signalling peer 639ab2db265f4747bb7afb93e833f530 to start an election
I20260709 13:20:37.107125 12709 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:20:37.153208 12708 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:42500
I20260709 13:20:37.153838 12708 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:37.154372 12708 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:37.161698 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 67631.5us
I20260709 13:20:37.202101 12708 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:37.204648 12708 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:37.206403 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:37.207064 12781 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:37.215047 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:37.215734 12633 raft_consensus.cc:3055] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:37.216055 12633 raft_consensus.cc:740] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:37.217013 12633 consensus_queue.cc:260] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:37.218381 12633 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:37.221752 12781 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:20:37.222957 12672 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:37.232748 19439 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:37.233345 19439 raft_consensus.cc:697] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:37.235100 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:37.235369 19439 consensus_queue.cc:237] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:37.237341 19493 tablet_replica.cc:333] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:20:37.239665 19493 raft_consensus.cc:2243] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:37.238111 12626 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:20:37.253965 19480 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:37.259428 12633 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:20:37.268325 19480 raft_consensus.cc:697] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:37.270195 19480 consensus_queue.cc:237] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:37.274014 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 50840.7us
I20260709 13:20:37.286885 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:20:37.296427 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 114898us
I20260709 13:20:37.282765 19493 raft_consensus.cc:2272] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:37.339569 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 44883.7us
I20260709 13:20:37.392354 19493 ts_tablet_manager.cc:1916] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:37.415490 19492 ts_tablet_manager.cc:1929] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 20.14
I20260709 13:20:37.416026 19492 log.cc:1199] T 18b2f67fae7f4963a4b4c562003df2fe P 3d539f8d95d343d8b1696803f18b30d2: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-2-root/wals/18b2f67fae7f4963a4b4c562003df2fe
W20260709 13:20:37.401124 19504 log.cc:927] Time spent T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:20:37.452924 12516 catalog_manager.cc:5028] TS 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): tablet 18b2f67fae7f4963a4b4c562003df2fe (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:20:37.488018 12528 catalog_manager.cc:5697] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 14 to 15, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 15 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
W20260709 13:20:37.501710 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 44751.6us
W20260709 13:20:37.552114 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 85439.3us
W20260709 13:20:37.557387 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 44079.3us
W20260709 13:20:37.705200 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 144638us
W20260709 13:20:37.712822 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 143423us
W20260709 13:20:37.718130 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 160022us
I20260709 13:20:37.819890 12705 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:20:37.825958 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 265346us
W20260709 13:20:37.828892 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 264316us
I20260709 13:20:37.850899 19529 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
W20260709 13:20:37.906176 12597 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.001s	sys 0.017s
W20260709 13:20:37.907035 12597 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.001s	sys 0.017s
W20260709 13:20:37.907544 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 504138us
I20260709 13:20:37.947233 19493 ts_tablet_manager.cc:1929] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.11
I20260709 13:20:37.959161 19493 log.cc:1199] T 199931c4ffc44fee980d8bb49cf37f00 P 19cd864c053744bebd96093a7982537e: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-0-root/wals/199931c4ffc44fee980d8bb49cf37f00
I20260709 13:20:38.001722 19531 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:38.003221 19531 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:38.076889 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 54852.8us
I20260709 13:20:38.078090 12525 catalog_manager.cc:5697] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 13 to 14, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 14 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:38.081635 19531 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:38.057334 12517 catalog_manager.cc:5028] TS 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): tablet 199931c4ffc44fee980d8bb49cf37f00 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
W20260709 13:20:38.101353 12517 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 44231us
I20260709 13:20:38.149371 12626 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:20:38.170888 12627 raft_consensus.cc:1275] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:20:38.172566 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:38.193320 19518 consensus_queue.cc:1048] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:38.197117 19521 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
W20260709 13:20:38.217739 12804 debug-util.cc:398] Leaking SignalData structure 0x7b0800394880 after lost signal to thread 11520
W20260709 13:20:38.220274 12804 debug-util.cc:398] Leaking SignalData structure 0x7b080021bce0 after lost signal to thread 12566
W20260709 13:20:38.223004 12804 debug-util.cc:398] Leaking SignalData structure 0x7b08000610a0 after lost signal to thread 12658
I20260709 13:20:38.249531 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:38.249761 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:38.250036 12775 raft_consensus.cc:1275] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:20:38.308458 12804 debug-util.cc:398] Leaking SignalData structure 0x7b08002b1460 after lost signal to thread 12733
W20260709 13:20:38.310325 12804 debug-util.cc:398] Leaking SignalData structure 0x7b08003564a0 after lost signal to thread 12807
W20260709 13:20:38.312291 12804 debug-util.cc:398] Leaking SignalData structure 0x7b08002fa000 after lost signal to thread 13410
I20260709 13:20:38.368101 19530 consensus_queue.cc:1048] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:38.395856 12672 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:38.396926 19530 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:38.808343 12633 consensus_queue.cc:237] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
W20260709 13:20:38.905894 12597 consensus_peers.cc:597] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6146f826f1904f6f8b10ad518d454c22. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:38.906174 12780 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:20:38.907966 19499 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:38.923962 12704 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:20:38.941259 19519 raft_consensus.cc:2955] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Committing config change with OpId 19.12: config changed from index -1 to 12, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:38.944975 12780 raft_consensus.cc:2955] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Committing config change with OpId 19.12: config changed from index -1 to 12, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
W20260709 13:20:38.973274 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 47401us
I20260709 13:20:39.001140 12531 catalog_manager.cc:5697] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index -1 to 12, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 19 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
W20260709 13:20:39.022984 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 47886.7us
I20260709 13:20:39.053169 19499 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
I20260709 13:20:39.098541 12627 consensus_queue.cc:237] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:39.121193 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:39.160053 12776 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:20:39.162007 19499 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:20:39.167330 12597 consensus_peers.cc:597] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e2a417d650a44658bfd5e2f5305185ac. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:39.178790 12703 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:20:39.213807 19353 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:39.216926 12705 raft_consensus.cc:2955] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Committing config change with OpId 19.12: config changed from index -1 to 12, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:39.241349 19353 raft_consensus.cc:2955] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Committing config change with OpId 18.12: config changed from index -1 to 12, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:39.246448 12781 raft_consensus.cc:2955] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index -1 to 12, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:39.295531 12531 catalog_manager.cc:5697] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index -1 to 12, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 18 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:39.302078 12703 raft_consensus.cc:2955] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index -1 to 12, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:39.326653 12630 consensus_queue.cc:237] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
W20260709 13:20:39.351850 12597 consensus_peers.cc:597] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ca762dedae144be828acb659b8d9e4b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:39.352458 12781 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:20:39.353988 12705 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:20:39.354876 19481 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:39.359426 19481 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:39.407632 19519 raft_consensus.cc:2955] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:39.413342 12781 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:20:39.414361 12781 raft_consensus.cc:606] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:20:39.418870 12704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:20:39.417538 12780 raft_consensus.cc:2955] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:39.419905 12704 raft_consensus.cc:606] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 20 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:39.422068 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:20:39.426800 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:39.453720 12531 catalog_manager.cc:5697] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index -1 to 10, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 13 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:39.472761 12780 consensus_queue.cc:237] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:39.513094 19482 raft_consensus.cc:993] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530: : Instructing follower 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:20:39.513737 19482 raft_consensus.cc:1081] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 20 LEADER]: Signalling peer 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:20:39.518772 12776 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:36478
I20260709 13:20:39.519537 12776 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:39.519986 12776 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:39.518920 12705 raft_consensus.cc:2955] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:39.530200 19534 raft_consensus.cc:993] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2: : Instructing follower 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:39.538095 19521 raft_consensus.cc:1081] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Signalling peer 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:39.536274 12704 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:20:39.544806 19591 ts_tablet_manager.cc:933] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
W20260709 13:20:39.548568 12746 consensus_peers.cc:597] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5fd9f4e830ed4a178665ad4e4ecc4452. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:39.552788 12627 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac"
dest_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:56838
I20260709 13:20:39.553898 12627 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:39.554358 12627 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:39.554667 12630 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:20:39.568817 19514 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:39.582990 12776 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:39.587638 12627 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:39.590764 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:39.591442 12781 raft_consensus.cc:3055] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:39.591800 12781 raft_consensus.cc:740] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:39.592680 12781 consensus_queue.cc:260] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:39.585300 19521 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 13:20:39.604799 12781 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:39.609243 12627 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:39.611569 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: true dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:39.612221 13376 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:39.623040 12781 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:20:39.624682 19591 tablet_copy_client.cc:323] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.65:43603
I20260709 13:20:39.624894 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:39.625980 19547 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:39.626629 19547 raft_consensus.cc:697] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:39.627256 12644 tablet_copy_service.cc:140] P 19cd864c053744bebd96093a7982537e: Received BeginTabletCopySession request for tablet 6146f826f1904f6f8b10ad518d454c22 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:39590)
I20260709 13:20:39.628129 12644 tablet_copy_service.cc:161] P 19cd864c053744bebd96093a7982537e: Beginning new tablet copy session on tablet 6146f826f1904f6f8b10ad518d454c22 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:39590: session id = 522fd2ba0e9f402691b5ef29af8579cd-6146f826f1904f6f8b10ad518d454c22
I20260709 13:20:39.627766 19547 consensus_queue.cc:237] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:39.641711 13376 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:20:39.656309 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: true dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:39.657583 13376 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:39.661674 12776 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:39.662042 19553 raft_consensus.cc:2955] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Committing config change with OpId 18.12: config changed from index -1 to 12, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:39.670212 13376 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 21.
I20260709 13:20:39.671950 12746 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:39.674015 19516 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:39.674993 19516 raft_consensus.cc:697] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:39.675989 19516 consensus_queue.cc:237] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:39.678321 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:39.678596 12644 tablet_copy_source_session.cc:215] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:39.678930 12704 raft_consensus.cc:3055] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:20:39.679280 12704 raft_consensus.cc:740] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:39.682466 12704 consensus_queue.cc:260] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:39.682008 12705 raft_consensus.cc:2955] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index -1 to 12, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:39.684088 12704 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:39.687657 19591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6146f826f1904f6f8b10ad518d454c22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:39.719339 12530 catalog_manager.cc:5697] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 12 to 13, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 13 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:39.731616 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 57720.5us
I20260709 13:20:39.738677 12704 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 21.
I20260709 13:20:39.759493 12531 catalog_manager.cc:5697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index -1 to 12, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 18 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
W20260709 13:20:39.774219 19595 log.cc:927] Time spent T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:20:39.792865 12633 raft_consensus.cc:2955] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index -1 to 12, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:39.812798 19591 tablet_copy_client.cc:806] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:39.813669 19591 tablet_copy_client.cc:670] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:39.814478 12531 catalog_manager.cc:5697] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 20 to 21, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 21 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:39.817291 19598 ts_tablet_manager.cc:933] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:20:39.822309 19591 tablet_copy_client.cc:538] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:39.878270 19598 tablet_copy_client.cc:323] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.65:43603
I20260709 13:20:39.879796 19591 tablet_bootstrap.cc:492] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:20:39.880012 12644 tablet_copy_service.cc:140] P 19cd864c053744bebd96093a7982537e: Received BeginTabletCopySession request for tablet e2a417d650a44658bfd5e2f5305185ac from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:39590)
I20260709 13:20:39.880558 12644 tablet_copy_service.cc:161] P 19cd864c053744bebd96093a7982537e: Beginning new tablet copy session on tablet e2a417d650a44658bfd5e2f5305185ac from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:39590: session id = 522fd2ba0e9f402691b5ef29af8579cd-e2a417d650a44658bfd5e2f5305185ac
I20260709 13:20:39.891235 12644 tablet_copy_source_session.cc:215] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:39.902200 19598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2a417d650a44658bfd5e2f5305185ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:40.018710 19600 ts_tablet_manager.cc:933] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:20:40.035840 13383 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:20:40.039269 19547 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:40.042030 19600 tablet_copy_client.cc:323] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.65:43603
I20260709 13:20:40.054701 12644 tablet_copy_service.cc:140] P 19cd864c053744bebd96093a7982537e: Received BeginTabletCopySession request for tablet 3ca762dedae144be828acb659b8d9e4b from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:39590)
I20260709 13:20:40.064606 12644 tablet_copy_service.cc:161] P 19cd864c053744bebd96093a7982537e: Beginning new tablet copy session on tablet 3ca762dedae144be828acb659b8d9e4b from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:39590: session id = 522fd2ba0e9f402691b5ef29af8579cd-3ca762dedae144be828acb659b8d9e4b
I20260709 13:20:40.085173 19598 tablet_copy_client.cc:806] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:40.085759 12704 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:20:40.086017 19598 tablet_copy_client.cc:670] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:40.092121 19534 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:20:40.093454 12644 tablet_copy_source_session.cc:215] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:40.109606 19598 tablet_copy_client.cc:538] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:40.135426 19600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ca762dedae144be828acb659b8d9e4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:40.172492 19591 tablet_bootstrap.cc:492] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=1 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:40.173625 19591 tablet_bootstrap.cc:492] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:20:40.174417 19591 ts_tablet_manager.cc:1403] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.295s	user 0.051s	sys 0.003s
I20260709 13:20:40.181550 12781 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:20:40.204358 19606 ts_tablet_manager.cc:933] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:40.214813 19519 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:40.231300 13383 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:20:40.236588 19521 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:20:40.261750 19598 tablet_bootstrap.cc:492] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:20:40.267763 19606 tablet_copy_client.cc:323] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.67:41181
I20260709 13:20:40.279649 12792 tablet_copy_service.cc:140] P 3d539f8d95d343d8b1696803f18b30d2: Received BeginTabletCopySession request for tablet 5fd9f4e830ed4a178665ad4e4ecc4452 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:54346)
I20260709 13:20:40.292054 19591 raft_consensus.cc:359] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:40.293118 12792 tablet_copy_service.cc:161] P 3d539f8d95d343d8b1696803f18b30d2: Beginning new tablet copy session on tablet 5fd9f4e830ed4a178665ad4e4ecc4452 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:54346: session id = 522fd2ba0e9f402691b5ef29af8579cd-5fd9f4e830ed4a178665ad4e4ecc4452
I20260709 13:20:40.304066 19591 raft_consensus.cc:740] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:20:40.349864 19591 consensus_queue.cc:260] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:40.373875 19591 ts_tablet_manager.cc:1434] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.199s	user 0.011s	sys 0.000s
I20260709 13:20:40.382157 12644 tablet_copy_service.cc:342] P 19cd864c053744bebd96093a7982537e: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-6146f826f1904f6f8b10ad518d454c22 received from {username='slave'} at 127.0.0.1:39590
I20260709 13:20:40.382732 12644 tablet_copy_service.cc:434] P 19cd864c053744bebd96093a7982537e: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-6146f826f1904f6f8b10ad518d454c22 on tablet 6146f826f1904f6f8b10ad518d454c22 with peer 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:20:40.395706 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 75556.2us
I20260709 13:20:40.429152 13383 raft_consensus.cc:1217] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 LEARNER]: Deduplicated request from leader. Original: 19.11->[19.12-19.12]   Dedup: 19.12->[]
W20260709 13:20:40.457885 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 51691.6us
I20260709 13:20:40.426066 19600 tablet_copy_client.cc:806] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
W20260709 13:20:40.458774 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 101712us
I20260709 13:20:40.472352 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:40.466995 19600 tablet_copy_client.cc:670] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
W20260709 13:20:40.549580 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 76888.6us
W20260709 13:20:40.562354 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 75842.8us
W20260709 13:20:40.688639 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 111769us
I20260709 13:20:40.712395 12792 tablet_copy_source_session.cc:215] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:20:40.821764 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 132447us
I20260709 13:20:40.895629 19606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fd9f4e830ed4a178665ad4e4ecc4452. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:20:40.903657 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 360950us
I20260709 13:20:40.956267 19623 raft_consensus.cc:1064] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:20:40.958982 19623 consensus_queue.cc:237] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 10, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:41.037941 19600 tablet_copy_client.cc:538] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:41.077234 12781 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:20:41.080447 19519 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:41.080787 12703 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:20:41.093415 13385 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 LEARNER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:20:41.096909 19600 tablet_bootstrap.cc:492] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:20:41.103217 19631 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:41.107914 19606 tablet_copy_client.cc:806] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:41.115942 19614 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:41.118301 19606 tablet_copy_client.cc:670] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:41.136583 19606 tablet_copy_client.cc:538] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:41.152597 19606 tablet_bootstrap.cc:492] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:20:41.194049 19598 tablet_bootstrap.cc:492] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:41.209538 19631 raft_consensus.cc:2955] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Committing config change with OpId 19.13: config changed from index 12 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:41.215989 12781 raft_consensus.cc:2955] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 12 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:41.221993 19598 tablet_bootstrap.cc:492] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:20:41.222940 19598 ts_tablet_manager.cc:1403] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.997s	user 0.056s	sys 0.002s
I20260709 13:20:41.272418 12525 catalog_manager.cc:5697] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index 12 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 19 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:41.275624 12703 raft_consensus.cc:2955] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 12 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:41.286092 19598 raft_consensus.cc:359] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 18 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:41.302253 19598 raft_consensus.cc:740] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 18 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:20:41.304843 19598 consensus_queue.cc:260] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:41.335183 19598 ts_tablet_manager.cc:1434] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.112s	user 0.007s	sys 0.000s
I20260709 13:20:41.341483 13385 raft_consensus.cc:2955] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 12 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:41.374809 12644 tablet_copy_service.cc:342] P 19cd864c053744bebd96093a7982537e: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-e2a417d650a44658bfd5e2f5305185ac received from {username='slave'} at 127.0.0.1:39590
I20260709 13:20:41.375308 12644 tablet_copy_service.cc:434] P 19cd864c053744bebd96093a7982537e: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-e2a417d650a44658bfd5e2f5305185ac on tablet e2a417d650a44658bfd5e2f5305185ac with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:41.589479 12627 consensus_queue.cc:237] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:41.597460 13385 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:20:41.598881 19614 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:41.602429 12703 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:20:41.604450 19614 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:41.607554 19614 raft_consensus.cc:2955] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Committing config change with OpId 19.14: config changed from index 13 to 14, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:41.616379 13385 raft_consensus.cc:2955] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 13 to 14, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:41.624145 12703 raft_consensus.cc:2955] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 13 to 14, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:41.626204 19600 tablet_bootstrap.cc:492] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:41.627215 19600 tablet_bootstrap.cc:492] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:20:41.627911 19600 ts_tablet_manager.cc:1403] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.531s	user 0.049s	sys 0.006s
I20260709 13:20:41.630409 19606 tablet_bootstrap.cc:492] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=2 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:41.635542 19606 tablet_bootstrap.cc:492] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:20:41.636368 19606 ts_tablet_manager.cc:1403] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.484s	user 0.040s	sys 0.016s
I20260709 13:20:41.646354 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6146f826f1904f6f8b10ad518d454c22 with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:20:41.647236 19600 raft_consensus.cc:359] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 13 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:41.648128 19600 raft_consensus.cc:740] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 13 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:20:41.648836 19600 consensus_queue.cc:260] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:41.651378 19600 ts_tablet_manager.cc:1434] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:20:41.661412 12531 catalog_manager.cc:5697] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 13 to 14, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New cstate: current_term: 19 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:41.690620 12644 tablet_copy_service.cc:342] P 19cd864c053744bebd96093a7982537e: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-3ca762dedae144be828acb659b8d9e4b received from {username='slave'} at 127.0.0.1:39590
I20260709 13:20:41.691105 12644 tablet_copy_service.cc:434] P 19cd864c053744bebd96093a7982537e: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-3ca762dedae144be828acb659b8d9e4b on tablet 3ca762dedae144be828acb659b8d9e4b with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:41.694238 12773 tablet_service.cc:1558] Processing DeleteTablet for tablet 6146f826f1904f6f8b10ad518d454c22 with delete_type TABLET_DATA_TOMBSTONED (TS 3d539f8d95d343d8b1696803f18b30d2 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:50664
I20260709 13:20:41.694069 19606 raft_consensus.cc:359] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:41.695086 19606 raft_consensus.cc:740] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:20:41.695775 19606 consensus_queue.cc:260] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:41.698407 19606 ts_tablet_manager.cc:1434] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.001s
I20260709 13:20:41.710986 19650 tablet_replica.cc:333] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:20:41.713864 19650 raft_consensus.cc:2243] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:41.714918 19650 raft_consensus.cc:2272] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:41.717830 12792 tablet_copy_service.cc:342] P 3d539f8d95d343d8b1696803f18b30d2: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-5fd9f4e830ed4a178665ad4e4ecc4452 received from {username='slave'} at 127.0.0.1:54346
I20260709 13:20:41.718458 12792 tablet_copy_service.cc:434] P 3d539f8d95d343d8b1696803f18b30d2: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-5fd9f4e830ed4a178665ad4e4ecc4452 on tablet 5fd9f4e830ed4a178665ad4e4ecc4452 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:41.730513 19650 ts_tablet_manager.cc:1916] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:41.760217 19650 ts_tablet_manager.cc:1929] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 19.13
I20260709 13:20:41.760728 19650 log.cc:1199] T 6146f826f1904f6f8b10ad518d454c22 P 3d539f8d95d343d8b1696803f18b30d2: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-2-root/wals/6146f826f1904f6f8b10ad518d454c22
I20260709 13:20:41.774365 12516 catalog_manager.cc:5028] TS 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): tablet 6146f826f1904f6f8b10ad518d454c22 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:20:41.819024 13385 raft_consensus.cc:1217] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 18 LEARNER]: Deduplicated request from leader. Original: 18.11->[18.12-18.12]   Dedup: 18.12->[]
I20260709 13:20:41.880198 13385 raft_consensus.cc:1217] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 LEARNER]: Deduplicated request from leader. Original: 18.11->[18.12-18.12]   Dedup: 18.12->[]
I20260709 13:20:41.946218 13385 raft_consensus.cc:1217] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 13 LEARNER]: Deduplicated request from leader. Original: 13.9->[13.10-13.10]   Dedup: 13.10->[]
I20260709 13:20:42.042327 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:42.247136 12776 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:20:42.247793 12776 raft_consensus.cc:606] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Received request to transfer leadership to TS 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:42.253424 12630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:33786
I20260709 13:20:42.254104 12630 raft_consensus.cc:606] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 21 LEADER]: Received request to transfer leadership to TS 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:42.256239 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:20:42.257941 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:42.266474 19516 raft_consensus.cc:993] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2: : Instructing follower 639ab2db265f4747bb7afb93e833f530 to start an election
I20260709 13:20:42.267072 19562 raft_consensus.cc:1081] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Signalling peer 639ab2db265f4747bb7afb93e833f530 to start an election
I20260709 13:20:42.269101 12703 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:42490
I20260709 13:20:42.269681 12703 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:42.270032 12703 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:42.282106 12703 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:42.286969 12703 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:42.292639 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: true dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:42.293185 13385 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:42.302374 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:42.303193 12776 raft_consensus.cc:3055] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:42.303751 12776 raft_consensus.cc:740] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:42.304659 12776 consensus_queue.cc:260] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:42.325196 19614 raft_consensus.cc:993] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: : Instructing follower 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:20:42.333774 19614 raft_consensus.cc:1081] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 21 LEADER]: Signalling peer 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:20:42.335927 13376 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e"
dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:51754
I20260709 13:20:42.336535 13376 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:42.336869 13376 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:42.338532 12776 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:42.344247 13376 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:42.359266 13376 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:42.359527 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 19 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:42.360210 12627 raft_consensus.cc:3055] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:20:42.360505 12627 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:42.361346 12627 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 21.19, Last appended by leader: 19, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:42.361919 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 19 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:42.362493 12627 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:42.362478 12778 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:42.381313 19617 raft_consensus.cc:1064] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:20:42.384090 19617 consensus_queue.cc:237] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 10, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:42.390133 12627 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 22.
I20260709 13:20:42.392836 13385 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
I20260709 13:20:42.399791 13349 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:42.400801 19653 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:20:42.401357 19653 raft_consensus.cc:697] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 22 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:42.394511 12778 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 22.
I20260709 13:20:42.402228 19653 consensus_queue.cc:237] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 21.19, Last appended by leader: 19, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:42.403460 12776 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
I20260709 13:20:42.408382 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:42.410466 19628 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:42.410995 19628 raft_consensus.cc:697] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:42.411885 19628 consensus_queue.cc:237] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:42.433708 19631 raft_consensus.cc:1064] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:20:42.446002 19614 raft_consensus.cc:1064] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:20:42.449733 19614 consensus_queue.cc:237] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:42.436457 19631 consensus_queue.cc:237] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 10, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:42.461771 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 51510.1us
I20260709 13:20:42.495132 13385 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 18 LEARNER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:20:42.496069 12776 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:20:42.499399 19519 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:42.500579 12708 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:20:42.501456 19519 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
W20260709 13:20:42.529250 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 46623.2us
I20260709 13:20:42.550860 13376 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 LEARNER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:20:42.552424 19514 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:42.554105 19519 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:42.568077 12704 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:20:42.570554 12630 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:20:42.576061 19514 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:42.585011 19617 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 13:20:42.611249 12530 catalog_manager.cc:5697] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 21 to 22, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 22 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:42.616701 12531 catalog_manager.cc:5697] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 18 to 19, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 19 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:42.693148 19623 raft_consensus.cc:2955] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Committing config change with OpId 18.13: config changed from index 12 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:42.694387 13385 raft_consensus.cc:2955] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 12 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:42.716449 19514 raft_consensus.cc:2955] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Committing config change with OpId 18.13: config changed from index 12 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:42.718165 12704 raft_consensus.cc:2955] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 12 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:20:42.796536 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 64281.5us
I20260709 13:20:42.802181 12704 raft_consensus.cc:2955] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 12 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:42.803774 13385 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 13 LEARNER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:42.823798 12778 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:20:42.824868 12707 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:42.827132 19651 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:42.827661 12627 raft_consensus.cc:2955] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 12 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:42.830325 19614 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:42.833212 19631 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:20:42.846048 12780 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:42.865605 13376 raft_consensus.cc:2955] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 12 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:42.895574 13383 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:20:42.908769 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 60846us
I20260709 13:20:42.912743 19623 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:42.916589 12627 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 21 index: 19. Preceding OpId from leader: term: 22 index: 20. (index mismatch)
I20260709 13:20:42.922461 19651 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:20:42.925889 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 69307us
I20260709 13:20:42.940040 19546 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.003s
I20260709 13:20:42.949851 12780 raft_consensus.cc:2955] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 12 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:42.951813 12528 catalog_manager.cc:5697] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index 12 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 18 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:42.980790 12774 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 21 index: 19. Preceding OpId from leader: term: 22 index: 20. (index mismatch)
I20260709 13:20:42.983265 19632 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.004s
I20260709 13:20:42.993136 19614 raft_consensus.cc:2955] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Committing config change with OpId 13.11: config changed from index 10 to 11, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:43.008934 12707 raft_consensus.cc:2955] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:20:43.148814 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 77377.1us
W20260709 13:20:43.154907 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 54999.8us
I20260709 13:20:43.158183 12776 raft_consensus.cc:2955] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:43.184584 13376 raft_consensus.cc:2955] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:43.218189 12630 consensus_queue.cc:237] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:43.225075 12528 catalog_manager.cc:5697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 12 to 13, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 18 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:43.243505 13385 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:20:43.246459 12778 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:20:43.267413 19614 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 13:20:43.264431 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:43.318656 12525 catalog_manager.cc:5697] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index 10 to 11, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 13 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:43.319866 19634 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
W20260709 13:20:43.388404 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 69084.6us
I20260709 13:20:43.411386 19519 raft_consensus.cc:2955] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Committing config change with OpId 18.14: config changed from index 13 to 14, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:43.417544 12778 raft_consensus.cc:2955] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Committing config change with OpId 18.14: config changed from index 13 to 14, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:43.430224 12630 consensus_queue.cc:237] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:43.449786 12774 consensus_queue.cc:237] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:43.458096 13385 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:20:43.472182 13376 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:20:43.476208 13385 raft_consensus.cc:2955] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Committing config change with OpId 18.14: config changed from index 13 to 14, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:43.480507 12708 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:20:43.491122 12633 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:20:43.490869 19686 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:43.504662 19696 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:43.515775 19686 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:43.519960 19697 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:43.529067 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e2a417d650a44658bfd5e2f5305185ac with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:20:43.540935 12525 catalog_manager.cc:5697] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index 13 to 14, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New cstate: current_term: 18 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:43.548552 19634 raft_consensus.cc:2955] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Committing config change with OpId 13.12: config changed from index 11 to 12, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:43.561545 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3ca762dedae144be828acb659b8d9e4b with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:20:43.561369 19697 raft_consensus.cc:2955] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Committing config change with OpId 18.14: config changed from index 13 to 14, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:43.565827 12708 raft_consensus.cc:2955] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Committing config change with OpId 13.12: config changed from index 11 to 12, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:43.568958 12695 tablet_service.cc:1558] Processing DeleteTablet for tablet e2a417d650a44658bfd5e2f5305185ac with delete_type TABLET_DATA_TOMBSTONED (TS 639ab2db265f4747bb7afb93e833f530 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:49378
I20260709 13:20:43.577993 19699 tablet_replica.cc:333] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:20:43.587740 19699 raft_consensus.cc:2243] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:43.592338 12627 raft_consensus.cc:2955] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Committing config change with OpId 18.14: config changed from index 13 to 14, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:43.599727 19699 raft_consensus.cc:2272] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:43.603147 19699 ts_tablet_manager.cc:1916] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:43.615747 13376 raft_consensus.cc:2955] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Committing config change with OpId 18.14: config changed from index 13 to 14, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:43.648659 13383 raft_consensus.cc:2955] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Committing config change with OpId 13.12: config changed from index 11 to 12, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:43.593047 12525 catalog_manager.cc:5697] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index 11 to 12, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New cstate: current_term: 13 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:43.666970 19699 ts_tablet_manager.cc:1929] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 18.13
I20260709 13:20:43.681869 19699 log.cc:1199] T e2a417d650a44658bfd5e2f5305185ac P 639ab2db265f4747bb7afb93e833f530: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-1-root/wals/e2a417d650a44658bfd5e2f5305185ac
I20260709 13:20:43.684378 12517 catalog_manager.cc:5028] TS 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): tablet e2a417d650a44658bfd5e2f5305185ac (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:20:43.698141 12516 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5fd9f4e830ed4a178665ad4e4ecc4452 with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:20:43.703151 12528 catalog_manager.cc:5697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index 13 to 14, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New cstate: current_term: 18 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:43.723606 12773 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ca762dedae144be828acb659b8d9e4b with delete_type TABLET_DATA_TOMBSTONED (TS 3d539f8d95d343d8b1696803f18b30d2 not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:50664
I20260709 13:20:43.761921 12695 tablet_service.cc:1558] Processing DeleteTablet for tablet 5fd9f4e830ed4a178665ad4e4ecc4452 with delete_type TABLET_DATA_TOMBSTONED (TS 639ab2db265f4747bb7afb93e833f530 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:49378
I20260709 13:20:43.763259 19699 tablet_replica.cc:333] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:20:43.764467 19699 raft_consensus.cc:2243] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:43.765317 19699 raft_consensus.cc:2272] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:43.780855 19699 ts_tablet_manager.cc:1916] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:43.803572 19700 tablet_replica.cc:333] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:20:43.805266 19700 raft_consensus.cc:2243] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:43.806510 19700 raft_consensus.cc:2272] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:43.813356 19700 ts_tablet_manager.cc:1916] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:43.856725 19700 ts_tablet_manager.cc:1929] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 13.11
I20260709 13:20:43.857234 19700 log.cc:1199] T 3ca762dedae144be828acb659b8d9e4b P 3d539f8d95d343d8b1696803f18b30d2: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-2-root/wals/3ca762dedae144be828acb659b8d9e4b
I20260709 13:20:43.859045 12516 catalog_manager.cc:5028] TS 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): tablet 3ca762dedae144be828acb659b8d9e4b (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:20:43.863267 19699 ts_tablet_manager.cc:1929] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 18.13
I20260709 13:20:43.869300 19699 log.cc:1199] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 639ab2db265f4747bb7afb93e833f530: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-1-root/wals/5fd9f4e830ed4a178665ad4e4ecc4452
I20260709 13:20:43.883234 12517 catalog_manager.cc:5028] TS 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): tablet 5fd9f4e830ed4a178665ad4e4ecc4452 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:20:44.556669 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:44.648178 12708 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:20:44.648805 12708 raft_consensus.cc:606] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:20:44.650384 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:20:44.652436 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:45.083388 19703 raft_consensus.cc:993] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530: : Instructing follower 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:45.092942 19703 raft_consensus.cc:1081] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Signalling peer 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:45.095338 12634 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98"
dest_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:56832
I20260709 13:20:45.096014 12634 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:45.096385 12634 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:45.120174 12634 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:45.125396 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:45.126005 12708 raft_consensus.cc:3055] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:45.126302 12708 raft_consensus.cc:740] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:45.126526 12634 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:45.127015 12708 consensus_queue.cc:260] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:45.128253 12708 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:45.130532 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:45.132712 12778 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:45.137872 12709 consensus_queue.cc:237] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:45.147759 12778 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:20:45.153877 12597 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:45.154830 19685 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:45.155300 19685 raft_consensus.cc:697] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:45.156177 19685 consensus_queue.cc:237] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:45.158229 12633 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:20:45.160723 12672 consensus_peers.cc:597] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 612a4639a7a749be8856bbc60766e312. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:45.162154 19692 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:45.164637 12774 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:20:45.166929 19692 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:45.174880 12708 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 10.
I20260709 13:20:45.217669 19692 raft_consensus.cc:2955] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Committing config change with OpId 14.11: config changed from index -1 to 11, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:45.224759 12633 raft_consensus.cc:2955] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index -1 to 11, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:45.240186 12531 catalog_manager.cc:5697] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 9 to 10, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 10 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:45.251214 12778 raft_consensus.cc:2955] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index -1 to 11, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:45.267496 12709 consensus_queue.cc:237] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:45.273254 12525 catalog_manager.cc:5697] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index -1 to 11, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 14 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:45.444422 12774 raft_consensus.cc:1275] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:20:45.446686 12633 raft_consensus.cc:1275] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:20:45.448601 19703 consensus_queue.cc:1048] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:45.451289 19704 consensus_queue.cc:1048] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:20:45.461316 12672 consensus_peers.cc:597] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 98a43c5d78c945bb8c5541bc892f4865. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:45.491598 19703 raft_consensus.cc:2955] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Committing config change with OpId 14.10: config changed from index -1 to 10, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:45.502908 12633 raft_consensus.cc:2955] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index -1 to 10, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:45.507169 12774 raft_consensus.cc:2955] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index -1 to 10, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:45.535605 12528 catalog_manager.cc:5697] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index -1 to 10, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 14 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:45.563750 12709 consensus_queue.cc:237] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:45.590780 12630 raft_consensus.cc:1275] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:20:45.591897 12774 raft_consensus.cc:1275] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:20:45.592459 19692 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:20:45.595141 12672 consensus_peers.cc:597] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f2f0a3339d324fd4902b6ff970ff0272. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:45.600558 19692 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:45.610169 12778 raft_consensus.cc:1275] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:20:45.612287 19702 consensus_queue.cc:1048] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:45.627310 12709 raft_consensus.cc:1275] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:20:45.629102 19702 consensus_queue.cc:1048] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:45.644101 12633 raft_consensus.cc:2955] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Committing config change with OpId 17.11: config changed from index -1 to 11, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:45.653625 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:45.640911 19692 raft_consensus.cc:2955] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Committing config change with OpId 17.11: config changed from index -1 to 11, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:45.735500 19724 ts_tablet_manager.cc:933] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:45.735241 12630 consensus_queue.cc:237] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:45.738700 12778 raft_consensus.cc:2955] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Committing config change with OpId 17.11: config changed from index -1 to 11, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:45.747392 19724 tablet_copy_client.cc:323] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.66:45173
I20260709 13:20:45.750290 12528 catalog_manager.cc:5697] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index -1 to 11, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 17 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:45.756927 12718 tablet_copy_service.cc:140] P 639ab2db265f4747bb7afb93e833f530: Received BeginTabletCopySession request for tablet 612a4639a7a749be8856bbc60766e312 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:38496)
I20260709 13:20:45.766144 12718 tablet_copy_service.cc:161] P 639ab2db265f4747bb7afb93e833f530: Beginning new tablet copy session on tablet 612a4639a7a749be8856bbc60766e312 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:38496: session id = 522fd2ba0e9f402691b5ef29af8579cd-612a4639a7a749be8856bbc60766e312
I20260709 13:20:45.796859 12718 tablet_copy_source_session.cc:215] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:45.855810 19724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 612a4639a7a749be8856bbc60766e312. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:20:45.900082 12597 consensus_peers.cc:597] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 20db0f202ea546699789c2325d241b35. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:20:45.901430 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 58294.8us
I20260709 13:20:45.904265 12778 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 13:20:45.906750 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 86169.8us
I20260709 13:20:45.926478 19731 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:45.960845 12707 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:20:45.964214 19724 tablet_copy_client.cc:806] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:45.964854 19724 tablet_copy_client.cc:670] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:45.984638 19725 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:45.991210 19732 raft_consensus.cc:2955] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Committing config change with OpId 18.12: config changed from index -1 to 12, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:45.992765 12774 raft_consensus.cc:2955] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index -1 to 12, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:46.017865 12525 catalog_manager.cc:5697] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index -1 to 12, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 18 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:20:46.074525 12704 raft_consensus.cc:2955] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index -1 to 12, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
W20260709 13:20:46.136022 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 97049.4us
I20260709 13:20:46.141292 19724 tablet_copy_client.cc:538] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:46.167297 19739 ts_tablet_manager.cc:933] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:46.186220 19724 tablet_bootstrap.cc:492] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:20:46.212066 19741 ts_tablet_manager.cc:933] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:46.211047 19739 tablet_copy_client.cc:323] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.66:45173
I20260709 13:20:46.231459 12718 tablet_copy_service.cc:140] P 639ab2db265f4747bb7afb93e833f530: Received BeginTabletCopySession request for tablet 98a43c5d78c945bb8c5541bc892f4865 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:38496)
I20260709 13:20:46.232157 12718 tablet_copy_service.cc:161] P 639ab2db265f4747bb7afb93e833f530: Beginning new tablet copy session on tablet 98a43c5d78c945bb8c5541bc892f4865 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:38496: session id = 522fd2ba0e9f402691b5ef29af8579cd-98a43c5d78c945bb8c5541bc892f4865
I20260709 13:20:46.241066 12718 tablet_copy_source_session.cc:215] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:46.250286 19739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98a43c5d78c945bb8c5541bc892f4865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:46.250655 19741 tablet_copy_client.cc:323] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.66:45173
I20260709 13:20:46.252899 12718 tablet_copy_service.cc:140] P 639ab2db265f4747bb7afb93e833f530: Received BeginTabletCopySession request for tablet f2f0a3339d324fd4902b6ff970ff0272 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:38496)
I20260709 13:20:46.253499 12718 tablet_copy_service.cc:161] P 639ab2db265f4747bb7afb93e833f530: Beginning new tablet copy session on tablet f2f0a3339d324fd4902b6ff970ff0272 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:38496: session id = 522fd2ba0e9f402691b5ef29af8579cd-f2f0a3339d324fd4902b6ff970ff0272
I20260709 13:20:46.273072 12718 tablet_copy_source_session.cc:215] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:46.302963 19741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2f0a3339d324fd4902b6ff970ff0272. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:46.330184 19741 tablet_copy_client.cc:806] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:46.330896 19741 tablet_copy_client.cc:670] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:46.337092 19741 tablet_copy_client.cc:538] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:46.798928 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:20:47.149501 19724 log.cc:927] Time spent T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.502s	user 0.001s	sys 0.000s
W20260709 13:20:47.218253 19739 consensus_meta.cc:287] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.899s	user 0.021s	sys 0.007s
I20260709 13:20:47.229271 19744 ts_tablet_manager.cc:933] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:20:47.248749 19739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:51762 (request call id 1321) took 1103 ms. Trace:
I20260709 13:20:47.249902 19739 rpcz_store.cc:276] 0709 13:20:46.145451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:46.145598 (+   147us) service_pool.cc:224] Handling call
0709 13:20:46.172745 (+ 27147us) ts_tablet_manager.cc:934] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
0709 13:20:46.248733 (+ 75988us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:20:46.318612 (+ 69879us) tablet_metadata.cc:665] Metadata flushed
0709 13:20:46.318747 (+   135us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:20:47.227479 (+908732us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:20:47.248181 (+ 20702us) tablet_replica.cc:206] Creating consensus instance
0709 13:20:47.248594 (+   413us) ts_tablet_manager.cc:989] Replica registered.
0709 13:20:47.248674 (+    80us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":313,"tablet-copy.queue_time_us":21129,"thread_start_us":56275,"threads_started":2}
I20260709 13:20:47.251673 19739 tablet_copy_client.cc:806] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
W20260709 13:20:47.251993 12747 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.003s	sys 0.018s
I20260709 13:20:47.252269 19739 tablet_copy_client.cc:670] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:47.252760 19741 tablet_bootstrap.cc:492] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
W20260709 13:20:47.252346 12747 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.004s	sys 0.019s
I20260709 13:20:47.256505 19724 tablet_bootstrap.cc:492] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:47.264420 19744 tablet_copy_client.cc:323] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.65:43603
I20260709 13:20:47.257634 19724 tablet_bootstrap.cc:492] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:20:47.287045 19724 ts_tablet_manager.cc:1403] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 1.101s	user 0.041s	sys 0.011s
I20260709 13:20:47.328068 19741 tablet_bootstrap.cc:492] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:47.329159 19741 tablet_bootstrap.cc:492] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:20:47.330519 19741 ts_tablet_manager.cc:1403] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.078s	user 0.032s	sys 0.014s
W20260709 13:20:47.438086 13377 thread.cc:641] tablet-copy [worker] (thread pool) Time spent creating pthread: real 0.999s	user 0.008s	sys 0.023s
W20260709 13:20:47.438511 13377 thread.cc:608] tablet-copy [worker] (thread pool) Time spent starting thread: real 1.071s	user 0.014s	sys 0.026s
I20260709 13:20:47.460536 19751 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:47.461256 19751 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:47.463739 19751 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:47.464308 19751 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:47.465581 19751 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:47.468714 19751 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:47.469830 12774 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 1069) took 1020 ms. Trace:
I20260709 13:20:47.470278 12774 rpcz_store.cc:276] 0709 13:20:46.449067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:46.449285 (+   218us) service_pool.cc:224] Handling call
0709 13:20:47.469244 (+1019959us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:47.469440 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:47.469468 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:47.469511 (+    43us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:47.469547 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.469625 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:47.469745 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:47.529944 19749 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:47.530611 19749 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:47.533911 19751 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:47.534514 19751 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:47.570441 19751 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:47.568946 19749 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:47.562443 19734 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:47.571734 19734 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:47.575600 19749 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:47.581745 19749 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:47.568037 19724 raft_consensus.cc:359] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:47.584672 19724 raft_consensus.cc:740] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:20:47.592614 19697 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:47.593482 19697 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:47.596730 19697 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:47.598570 19734 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:47.599411 19724 consensus_queue.cc:260] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:47.614177 19734 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:47.615108 19734 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:47.620448 19758 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:47.621217 19758 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:47.623626 19758 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:47.624580 19696 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:47.630311 19734 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:47.629256 19756 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:47.631985 19756 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:47.634419 19756 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:47.625030 19696 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:47.637940 19734 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:47.638577 19734 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:47.639338 19696 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:47.640658 19734 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:47.642623 19734 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:47.643251 19734 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:47.647066 19734 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:47.657905 19745 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:47.690718 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:47.664626 19759 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:47.695412 19759 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:47.696759 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:47.697391 12781 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 22.
I20260709 13:20:47.693514 19745 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:47.698701 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:47.649461 19749 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:47.700855 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:47.699777 19745 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:47.685638 19761 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:47.700680 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:47.668037 19697 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:47.667827 19758 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:47.726259 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:47.674342 19734 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:47.736138 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:47.737061 12708 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:20:47.674533 19751 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:47.703485 19759 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:47.703166 19703 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:47.724056 19761 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:47.750502 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:47.752359 19546 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:47.753516 19546 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:47.761159 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:47.768795 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:47.774595 12775 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:20:47.760030 19734 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:47.783914 19762 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:47.784529 19762 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:47.785131 19734 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:47.786360 19741 raft_consensus.cc:359] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:47.786949 19762 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:47.734663 19697 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:47.796092 19724 ts_tablet_manager.cc:1434] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.508s	user 0.021s	sys 0.009s
W20260709 13:20:47.796681 19724 ts_tablet_manager.cc:1467] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd: Tablet startup took 508ms
W20260709 13:20:47.797153 19724 ts_tablet_manager.cc:1469] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd: Trace:
0709 13:20:45.657802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:45.657986 (+   184us) service_pool.cc:224] Handling call
0709 13:20:45.735998 (+ 78012us) ts_tablet_manager.cc:934] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
0709 13:20:45.843447 (+107449us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:20:45.908595 (+ 65148us) tablet_metadata.cc:665] Metadata flushed
0709 13:20:45.917270 (+  8675us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:20:45.957215 (+ 39945us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:20:45.957526 (+   311us) tablet_replica.cc:206] Creating consensus instance
0709 13:20:45.959577 (+  2051us) ts_tablet_manager.cc:989] Replica registered.
0709 13:20:45.959634 (+    57us) inbound_call.cc:177] Queueing success response
0709 13:20:46.185729 (+226095us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:20:47.288203 (+1102474us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:20:47.288774 (+   571us) tablet_replica.cc:261] Starting instrumentation
0709 13:20:47.289842 (+  1068us) tablet_replica.cc:280] Starting consensus
0709 13:20:47.794039 (+504197us) spinlock_profiling.cc:243] Waited 5.28 ms on lock 0x7b18004504c0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6cf5a4bc 00007f4c6a2a848b 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c2455 00007f4c6d3aa060 00007f4c6d3a7d71 00007f4c70da4b2e
Metrics: {"dns_us":294,"mutex_wait_us":325358,"raft.queue_time_us":145218,"raft.run_wall_time_us":8921,"spinlock_wait_cycles":19008128,"tablet-copy.queue_time_us":77011,"tablet-download-pool.queue_time_us":105,"tablet-download-pool.run_cpu_time_us":1117,"tablet-download-pool.run_wall_time_us":175765,"thread_start_us":169001,"threads_started":3}
I20260709 13:20:47.787321 19741 raft_consensus.cc:740] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:20:47.801949 19546 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:47.813021 19767 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:47.813705 19730 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:47.815521 19776 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:20:47.818579 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 58152.7us
I20260709 13:20:47.755493 19759 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:47.756014 19703 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:47.787951 19753 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:47.837802 19741 consensus_queue.cc:260] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:47.823901 19745 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:47.823976 19734 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:47.745455 19758 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:47.839320 19760 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:47.759475 19751 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:47.842202 19757 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:47.838023 19773 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:47.868002 19749 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:20:47.871690 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 90062.3us
I20260709 13:20:47.854465 19731 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:47.869720 19767 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:47.887769 12672 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:47.883850 19696 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:47.890904 19762 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:47.874959 19759 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:47.884492 19750 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:47.954532 19750 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:47.890669 12644 tablet_copy_service.cc:140] P 19cd864c053744bebd96093a7982537e: Received BeginTabletCopySession request for tablet 20db0f202ea546699789c2325d241b35 from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:39590)
I20260709 13:20:47.956547 12644 tablet_copy_service.cc:161] P 19cd864c053744bebd96093a7982537e: Beginning new tablet copy session on tablet 20db0f202ea546699789c2325d241b35 from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:39590: session id = 522fd2ba0e9f402691b5ef29af8579cd-20db0f202ea546699789c2325d241b35
I20260709 13:20:47.957930 19750 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:47.886075 19761 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:47.967469 19761 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:47.968061 19761 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:47.889732 19748 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:47.882489 19734 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:47.974413 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 82970.1us
I20260709 13:20:47.884549 19745 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:47.982764 19546 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:47.878083 19730 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:47.883280 19753 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:47.927126 19752 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:47.883816 19760 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:47.983471 19546 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:47.951712 19762 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:47.892128 19749 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.011310 12644 tablet_copy_source_session.cc:215] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:47.970204 19748 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:47.875488 19776 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:48.014216 19546 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.012534 19783 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:48.017549 19787 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:48.017403 19783 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:48.018309 19787 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:48.019975 19783 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.020474 19787 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.020532 19783 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:47.885088 19757 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:47.891566 19731 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:47.987085 19752 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:47.887619 19773 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.026647 19788 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:48.027693 19546 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:47.952452 19696 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.030500 19546 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.021134 19783 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.021803 19770 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:48.024439 19697 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:48.031179 19752 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:48.068137 19546 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:48.028060 19791 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:48.054519 19776 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.071702 19546 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:48.056887 19738 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:48.075941 19771 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:48.079017 19696 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:48.079241 19771 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.074234 19782 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:48.080447 19782 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:48.058568 19751 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:48.073393 19546 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.055125 19784 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:48.081246 19754 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:48.074208 19755 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:48.078438 19774 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:48.078256 19772 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:48.078481 19779 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:48.087095 19703 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.088374 19753 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.088120 19779 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.087962 19772 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.089035 19765 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:48.078627 19781 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:48.090054 19781 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.090408 19734 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.090775 19772 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:48.090894 19759 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:48.085908 19768 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:48.092109 19745 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:48.092933 19749 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:48.087313 19774 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.090430 19779 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:48.093748 19758 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:48.095060 19748 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.095211 19757 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:48.087211 19773 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:48.086431 19755 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.096799 19760 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:48.097219 19546 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.072613 19766 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:48.098003 19755 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.097922 19766 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.086095 19754 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.100167 19766 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.100481 19754 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:48.091867 19768 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.102062 19789 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:48.102449 19790 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:48.102190 19784 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.103235 19768 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.102830 19789 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.103775 19768 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:48.070137 19770 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:48.104367 19546 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:48.104259 19768 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.105296 19546 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.105914 19789 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.106283 19770 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:48.092166 19781 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:48.106828 19772 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:48.107275 19762 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:48.095160 19774 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:48.107388 19772 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.108285 19767 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.107206 19738 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.103029 19790 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.058197 19788 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.344151 19703 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:48.124457 19752 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:48.345670 19703 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:48.520076 19763 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:20:48.296712 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 214253us
I20260709 13:20:48.671362 19763 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:48.672432 12747 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:48.124303 19768 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.391614 19766 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:48.610582 19767 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:48.391496 19696 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:48.680122 19767 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.680281 19696 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.129593 19764 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:48.156764 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:48.675134 19766 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.322283 19801 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:48.112778 19765 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.490886 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.163726 19754 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:48.704373 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:48.705344 19754 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.706524 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:48.389683 19731 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:48.665830 19794 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:48.709394 19794 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.711189 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:48.712181 19794 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:48.712272 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:48.712579 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:48.712649 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.714128 19754 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.678674 19783 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:48.679088 12672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.002s	sys 0.017s
I20260709 13:20:48.683274 19767 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:48.736337 12672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.003s	sys 0.017s
W20260709 13:20:48.736829 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 837480us
I20260709 13:20:48.736862 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:48.739522 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:48.395525 19761 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:48.740667 13383 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:20:48.741509 12672 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:48.742678 19778 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:48.743180 19778 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:48.743597 19778 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:48.747879 19766 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.326470 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.156677 19753 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:48.750315 19772 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.750664 13377 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
W20260709 13:20:48.752038 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 79915.5us
I20260709 13:20:48.752300 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.754364 13385 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
I20260709 13:20:48.740669 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:48.757908 13381 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 21.
I20260709 13:20:48.758217 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.758687 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.759039 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.759573 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.759907 12633 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:20:48.760349 12634 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:20:48.761058 19801 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.761263 19753 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.762265 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:48.763481 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:48.763731 19753 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.765297 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:48.391568 19755 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:48.767215 19755 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.684306 19703 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.770339 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:48.684619 19771 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:48.079910 19741 ts_tablet_manager.cc:1434] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.749s	user 0.014s	sys 0.022s
W20260709 13:20:48.772140 19741 ts_tablet_manager.cc:1467] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd: Tablet startup took 1441ms
I20260709 13:20:48.178031 19739 tablet_copy_client.cc:538] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:48.129184 19660 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:48.774489 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.774612 19660 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.776952 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.777355 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.161114 19789 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:48.111510 19546 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.779130 19789 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.780438 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.781956 19789 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:48.691010 12675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.005s	sys 0.015s
W20260709 13:20:48.808540 12675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.006s	sys 0.015s
I20260709 13:20:48.809954 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.810557 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:48.811303 12774 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:20:48.692021 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:20:48.695017 12597 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.016s	sys 0.011s
I20260709 13:20:48.811975 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:48.348596 19752 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.812228 13384 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:20:48.718456 19785 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:48.075659 19791 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.813230 19785 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:48.813738 19785 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:48.815397 19791 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:48.546140 19769 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:48.816893 19769 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.818243 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:48.819152 12708 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
I20260709 13:20:48.820021 19769 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:48.724360 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:48.821851 13382 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:20:48.822513 19763 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.823246 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:48.805541 12746 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:48.824875 12746 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:48.713236 13379 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:20:48.826442 19795 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:48.826890 19795 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:48.827330 19795 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:48.827404 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:48.828337 12703 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:20:48.727463 19784 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:48.113377 19782 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.114769 19730 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:48.766191 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:48.771168 13378 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:20:48.832376 13377 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
W20260709 13:20:48.516696 13407 debug-util.cc:398] Leaking SignalData structure 0x7b08003463c0 after lost signal to thread 11520
W20260709 13:20:48.834529 13407 debug-util.cc:398] Leaking SignalData structure 0x7b0800304080 after lost signal to thread 12566
I20260709 13:20:48.835184 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.782842 19790 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.783617 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.836077 12625 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:20:48.784018 13376 raft_consensus.cc:1217] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEARNER]: Deduplicated request from leader. Original: 14.10->[14.11-14.11]   Dedup: 14.11->[]
I20260709 13:20:48.835378 19744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20db0f202ea546699789c2325d241b35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:48.683020 19764 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.840188 19764 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.842142 19761 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:48.842748 19761 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:48.545532 19779 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:48.789816 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.847450 19779 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:48.849804 19761 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:48.792299 19765 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:20:48.792682 13381 raft_consensus.cc:1217] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 LEARNER]: Deduplicated request from leader. Original: 17.10->[17.11-17.11]   Dedup: 17.11->[]
I20260709 13:20:48.796988 19696 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:48.798940 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:48.800223 19788 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:48.801883 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:48.855374 12776 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
I20260709 13:20:48.794512 19738 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.856633 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:48.857326 12703 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:20:48.842201 19801 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:48.769925 19755 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.804378 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.805619 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.806005 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.806950 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.862722 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:20:48.812088 12597 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.836s	user 0.017s	sys 0.012s
I20260709 13:20:48.863538 12703 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
W20260709 13:20:48.689742 12598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.015s	sys 0.016s
I20260709 13:20:48.835244 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:48.865803 12598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.998s	user 0.015s	sys 0.017s
I20260709 13:20:48.777334 19660 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:48.868182 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:48.869823 19782 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:48.853994 19779 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.807598 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:48.870450 19782 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.867091 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.867611 19769 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:48.871847 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:48.872327 19769 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:48.872714 19782 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:48.874711 19769 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.874863 19784 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:48.872588 12703 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:20:48.867990 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.875627 19784 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.867269 12746 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:48.882099 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:48.883075 12703 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:20:48.883421 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.883391 19795 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.885795 19696 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:48.888051 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.894150 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.896085 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.897246 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.897722 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.897871 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.901156 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.900700 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.904480 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.904428 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.881924 12634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98"
dest_uuid: "19cd864c053744bebd96093a7982537e"
mode: GRACEFUL
new_leader_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:33786
W20260709 13:20:48.915210 13407 debug-util.cc:398] Leaking SignalData structure 0x7b08002f9360 after lost signal to thread 12658
I20260709 13:20:48.908205 19784 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:48.910883 12634 raft_consensus.cc:606] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Received request to transfer leadership to TS 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:48.919802 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:48.921293 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.915011 19795 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:48.926005 12776 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17. Ops: []
W20260709 13:20:48.926923 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 68009.5us
I20260709 13:20:48.934841 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:48.926148 19696 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:20:48.772593 19741 ts_tablet_manager.cc:1469] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd: Trace:
0709 13:20:46.170954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:46.171065 (+   111us) service_pool.cc:224] Handling call
0709 13:20:46.217944 (+ 46879us) ts_tablet_manager.cc:934] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
0709 13:20:46.301396 (+ 83452us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:20:46.320544 (+ 19148us) tablet_metadata.cc:665] Metadata flushed
0709 13:20:46.320671 (+   127us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:20:46.328578 (+  7907us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:20:46.328922 (+   344us) tablet_replica.cc:206] Creating consensus instance
0709 13:20:46.329314 (+   392us) ts_tablet_manager.cc:989] Replica registered.
0709 13:20:46.329369 (+    55us) inbound_call.cc:177] Queueing success response
0709 13:20:47.252330 (+922961us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:20:47.330943 (+ 78613us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:20:47.331518 (+   575us) tablet_replica.cc:261] Starting instrumentation
0709 13:20:47.332089 (+   571us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":263,"mutex_wait_us":406664,"raft.queue_time_us":92444,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"tablet-copy.queue_time_us":40329,"tablet-download-pool.queue_time_us":599,"tablet-download-pool.run_cpu_time_us":1139,"tablet-download-pool.run_wall_time_us":3150,"thread_start_us":163666,"threads_started":3}
I20260709 13:20:48.940491 19696 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.942741 19696 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:48.931823 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:48.942461 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:48.945013 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:48.948060 19778 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:48.961819 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.963017 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.965734 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:48.970652 19801 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:48.971509 19801 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.972059 12673 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:48.973035 19752 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:48.973454 19752 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:48.973634 12673 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:48.973910 19752 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:48.974557 19801 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:48.976676 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.977277 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:20:48.978066 13407 debug-util.cc:398] Leaking SignalData structure 0x7b080021b300 after lost signal to thread 12733
I20260709 13:20:48.979394 19761 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:48.983523 19752 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:48.985989 19752 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.989780 19778 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:48.990592 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:48.990758 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:49.006155 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:20:49.006596 13407 debug-util.cc:398] Leaking SignalData structure 0x7b080022b8e0 after lost signal to thread 12807
I20260709 13:20:48.990994 19739 tablet_bootstrap.cc:492] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:20:49.007403 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:20:49.008189 13407 debug-util.cc:398] Leaking SignalData structure 0x7b0800389000 after lost signal to thread 13410
I20260709 13:20:49.008611 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:48.994012 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:48.994534 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:48.994956 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.995180 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:48.995399 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:48.996384 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:48.996690 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:48.997774 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:48.997867 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.998716 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:48.998826 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:48.998862 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:48.999953 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.012692 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:49.013083 13378 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 24.
I20260709 13:20:49.013510 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:20:48.999960 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:49.000015 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:49.001864 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:49.002655 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.002894 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:49.016527 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:49.017158 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.003834 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.004328 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.019207 19761 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:49.022703 19788 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:49.023200 19788 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:49.023623 19788 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:49.028395 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:49.032774 19788 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:49.017076 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:49.033826 19778 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:49.034353 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.034562 19801 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:49.034974 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.035082 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.036895 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.039110 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:49.039561 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.040994 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.041016 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.042316 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.043195 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.021437 19785 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
W20260709 13:20:49.042320 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 68785.4us
I20260709 13:20:49.049929 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:49.047259 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:49.055056 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.055231 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.055835 12781 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 17: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:49.060197 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.058761 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.058349 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.060292 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:49.064779 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:49.068466 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.064886 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:49.070257 12774 raft_consensus.cc:1240] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:49.072502 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.080801 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.081682 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.083568 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.089738 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.090634 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:49.082717 19801 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.096259 19801 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:49.097486 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.099345 12704 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 23.
I20260709 13:20:49.103320 19761 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:49.103794 19761 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:49.104179 19761 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:49.110968 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.126780 12747 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:49.127895 19696 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:49.131240 13348 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:49.132928 12718 tablet_copy_service.cc:342] P 639ab2db265f4747bb7afb93e833f530: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-612a4639a7a749be8856bbc60766e312 received from {username='slave'} at 127.0.0.1:38496
I20260709 13:20:49.133296 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.133934 12718 tablet_copy_service.cc:434] P 639ab2db265f4747bb7afb93e833f530: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-612a4639a7a749be8856bbc60766e312 on tablet 612a4639a7a749be8856bbc60766e312 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:49.135030 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.135799 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.136510 12703 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 20.
I20260709 13:20:49.136905 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.137471 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.137698 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.138175 12706 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:20:49.138845 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.139130 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.136391 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.141420 19764 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:20:49.141920 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 57337.1us
I20260709 13:20:49.143362 13348 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:49.145228 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:49.146674 19764 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:49.147316 13348 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:49.208915 19755 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:49.210021 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:20:49.210346 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 63112.8us
I20260709 13:20:49.210786 12700 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:20:49.211191 13348 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:49.212077 19755 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:49.147821 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:49.077252 19778 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.149943 19744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:51754 (request call id 1501) took 2782 ms. Trace:
I20260709 13:20:49.151298 19785 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:49.152083 19761 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:49.153707 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 52699.6us
I20260709 13:20:49.167554 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:49.170845 12597 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:49.177352 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:49.178303 19788 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:49.180395 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:49.180752 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:49.181262 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:49.190277 19696 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:49.193679 19752 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:49.197145 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:49.202936 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.203155 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.203948 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.204232 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.203995 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.207041 12746 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:49.129551 12747 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:49.217985 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.225884 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:49.230526 12718 tablet_copy_service.cc:342] P 639ab2db265f4747bb7afb93e833f530: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-f2f0a3339d324fd4902b6ff970ff0272 received from {username='slave'} at 127.0.0.1:38496
I20260709 13:20:49.226665 12708 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 19.
I20260709 13:20:49.232077 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:49.234476 12718 tablet_copy_service.cc:434] P 639ab2db265f4747bb7afb93e833f530: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-f2f0a3339d324fd4902b6ff970ff0272 on tablet f2f0a3339d324fd4902b6ff970ff0272 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:49.234813 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:20:49.235121 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 64339.6us
I20260709 13:20:49.236317 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:49.239461 19696 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:49.240464 19763 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:20:49.240924 19763 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:49.241333 19763 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:20:49.241621 19775 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:49.241811 19752 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.242419 19775 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:49.244335 19752 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:49.250229 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:49.254217 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:49.256330 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:49.260161 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:20:49.270078 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 140574us
I20260709 13:20:49.232908 19795 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:49.270123 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:49.271366 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:49.283049 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:49.285904 12782 raft_consensus.cc:2393] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 19: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:49.273831 19775 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:49.274906 19795 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:49.295526 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:20:49.298135 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 57691.8us
I20260709 13:20:49.297262 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:49.291813 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:49.307322 12706 raft_consensus.cc:1240] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:49.308053 13349 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:49.309816 19761 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:49.310703 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.312670 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.297781 19778 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:49.298156 19696 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:49.313722 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:49.218186 19744 rpcz_store.cc:276] 0709 13:20:46.367470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:46.367634 (+   164us) service_pool.cc:224] Handling call
0709 13:20:47.257801 (+890167us) ts_tablet_manager.cc:934] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
0709 13:20:48.832425 (+1574624us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:20:48.986032 (+153607us) tablet_metadata.cc:665] Metadata flushed
0709 13:20:48.986161 (+   129us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:20:49.148737 (+162576us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:20:49.149101 (+   364us) tablet_replica.cc:206] Creating consensus instance
0709 13:20:49.149702 (+   601us) ts_tablet_manager.cc:989] Replica registered.
0709 13:20:49.149810 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":286,"tablet-copy.queue_time_us":845335,"thread_start_us":1076372,"threads_started":2}
I20260709 13:20:49.315138 19744 tablet_copy_client.cc:806] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:49.315264 19696 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:49.315760 19744 tablet_copy_client.cc:670] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:49.315795 19696 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:49.316522 19696 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:49.296617 19795 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:49.291352 19806 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:49.319617 19775 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:49.327102 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:49.326478 12672 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:49.330144 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.338512 19764 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:49.340828 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.341827 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:20:49.363729 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 45023.3us
I20260709 13:20:49.343381 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.322006 19806 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:49.331305 19730 consensus_queue.cc:1059] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 13:20:49.367008 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.367062 19806 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:49.369699 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:49.370570 19805 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:49.371019 19805 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:49.371364 19805 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:49.383605 19805 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:49.391340 19805 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:49.384449 19806 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.385624 19781 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:20:49.385833 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 59499.6us
I20260709 13:20:49.398514 12672 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:49.398640 12746 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:49.384860 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.399156 19806 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:49.399637 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 98696.5us
I20260709 13:20:49.400285 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.400957 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:49.333143 19775 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:49.403064 19761 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:49.434180 19761 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:49.429464 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.400946 19806 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:49.452687 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:20:49.454499 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 74720.1us
I20260709 13:20:49.428193 19755 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:49.454128 19761 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:49.465931 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 65059.3us
I20260709 13:20:49.457012 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.467463 13349 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:49.465919 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.468832 13349 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:49.469425 19765 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:49.458770 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.474711 12709 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 22. Current term is 23. Ops: []
I20260709 13:20:49.479641 19782 raft_consensus.cc:3055] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:49.480109 19782 raft_consensus.cc:740] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:49.434115 19778 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:49.487836 19763 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.434566 19769 raft_consensus.cc:993] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e: : Instructing follower 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:20:49.489996 19769 raft_consensus.cc:1081] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Signalling peer 3d539f8d95d343d8b1696803f18b30d2 to start an election
I20260709 13:20:49.492956 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:49.493191 19763 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 25 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:49.496235 13349 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:20:49.501711 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 164070us
I20260709 13:20:49.501883 19765 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:49.502413 19765 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:49.502549 12746 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:49.502867 19765 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:49.503460 19805 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:49.503655 12746 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:49.504074 19805 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:49.504464 19805 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:49.504834 12746 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:49.505746 19809 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:49.506146 19809 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:49.506346 19785 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:49.507085 19785 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:49.507649 19785 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:49.499156 19779 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:49.506541 19809 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:49.518016 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 119589us
I20260709 13:20:49.520133 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:49.522531 12672 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:49.523515 19804 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:49.456876 19806 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:49.533764 19764 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:49.535804 19809 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.545679 19806 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:49.428272 19775 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:49.481011 19782 consensus_queue.cc:260] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.554198 12747 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:49.554759 12597 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:49.551815 19781 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:49.558033 19781 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:49.558060 19799 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:49.558506 19781 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:49.557459 12597 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:49.561543 19763 consensus_queue.cc:1059] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:49.563911 19784 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:20:49.565069 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 67494.6us
I20260709 13:20:49.538806 19804 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:49.565650 19799 raft_consensus.cc:3055] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:49.566007 19804 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:49.566133 19799 raft_consensus.cc:740] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:49.566406 13349 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:49.567617 19790 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:49.568111 13349 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:49.567307 19799 consensus_queue.cc:260] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:49.563980 12597 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:49.570040 19799 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:49.570567 19763 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:49.572427 19696 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:49.550771 19805 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.581321 19790 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:49.585028 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:49.583945 19795 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.591693 19795 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:49.582082 13349 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:49.596683 19806 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.587204 19805 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:49.597982 19790 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:49.598395 19782 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:49.598872 13349 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:49.593609 19696 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:20:49.599300 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 76929.6us
W20260709 13:20:49.599926 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 94255.5us
I20260709 13:20:49.601298 19790 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:49.609999 19790 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:49.609886 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:49.612259 12782 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:49.613381 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:49.613641 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:49.615361 12782 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:49.602556 19795 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:20:49.616416 19795 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:49.616775 12782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
 from {username='slave'} at 127.0.0.1:36488
I20260709 13:20:49.617321 19795 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:20:49.600869 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 46769us
I20260709 13:20:49.618862 19809 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:49.619561 13348 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:49.622318 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:49.623941 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:49.623631 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:49.626432 12701 raft_consensus.cc:2393] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 19: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:49.626636 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:49.624718 19764 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:49.627497 19764 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:49.627863 19779 consensus_queue.cc:1059] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:20:49.626432 19784 consensus_queue.cc:1059] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 13:20:49.627863 19764 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:49.629415 19784 raft_consensus.cc:3055] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:49.629842 19784 raft_consensus.cc:740] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:49.631232 19807 raft_consensus.cc:3055] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:20:49.631686 19807 raft_consensus.cc:740] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:49.626003 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:49.633174 19807 consensus_queue.cc:260] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:49.634997 19761 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:49.636013 12782 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:49.636402 12782 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:49.630802 19784 consensus_queue.cc:260] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.638867 19807 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:49.641295 19784 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:49.646777 12782 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:49.650174 19764 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.655675 12672 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:49.656576 19801 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:49.653723 19795 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.657837 19789 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:49.658450 19791 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:49.659601 19795 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:49.660899 13349 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:49.662571 19764 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:49.663551 13348 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:49.664992 19755 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:49.666704 19755 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:49.667496 19755 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:49.668265 19790 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.671296 19791 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:49.705744 19790 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:49.675177 19761 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:49.678889 13348 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:49.715487 13348 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:49.716900 13348 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:49.717760 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:49.717921 19789 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:49.718254 19791 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:49.679247 19806 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:49.682646 19764 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:49.693773 19765 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.719676 13348 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:49.719916 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:49.720536 19791 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:49.720746 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:49.721637 19765 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:49.697949 12782 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:49.702925 12672 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:49.723075 19778 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:49.724460 12672 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:49.725172 19778 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:49.727001 12672 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:49.727751 19801 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:49.729627 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:49.730237 12672 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:49.710592 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.712301 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:49.712589 19775 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:49.682384 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 58050.2us
I20260709 13:20:49.673226 19781 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.718778 19789 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:49.730639 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:49.730988 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:49.733422 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:49.733475 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:49.722970 13348 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:49.723198 19755 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.707237 19785 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.742106 19744 tablet_copy_client.cc:538] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:49.747741 12629 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:20:49.776118 19781 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:49.765653 19801 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:49.780366 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:49.780665 19775 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:49.750823 19804 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.783440 12782 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:49.783600 19785 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:49.784251 19804 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:49.786044 19763 consensus_queue.cc:1059] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:49.787144 19763 raft_consensus.cc:3055] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:49.787492 19763 raft_consensus.cc:740] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:49.788376 19763 consensus_queue.cc:260] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.758597 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:49.791085 19763 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:49.764611 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:49.765322 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:49.767625 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:49.768136 19755 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:49.771050 12775 raft_consensus.cc:1240] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:49.771761 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:49.760169 12672 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:20:49.794020 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 71142.8us
I20260709 13:20:49.804069 13348 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:49.794571 19790 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:49.805240 13348 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:20:49.805968 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 45888.6us
I20260709 13:20:49.806666 12672 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:49.795589 12782 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:49.807871 12672 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:49.804818 19799 consensus_queue.cc:1059] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:20:49.810331 19804 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:49.812201 19781 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:49.813182 19801 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:49.815778 12672 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:49.817082 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:49.805212 19755 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:49.818399 19781 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:49.818918 19781 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:49.819595 19781 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:49.819906 19790 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:49.822714 13348 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:49.828410 19789 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:49.830089 19744 tablet_bootstrap.cc:492] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
I20260709 13:20:49.833909 19799 raft_consensus.cc:3055] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:49.834791 19799 raft_consensus.cc:740] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:49.838007 19779 consensus_queue.cc:1059] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:49.839116 19730 raft_consensus.cc:3055] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:49.839507 19730 raft_consensus.cc:740] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:49.840493 19730 consensus_queue.cc:260] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.841753 19730 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:49.842389 12782 raft_consensus.cc:1240] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:49.844389 19779 consensus_queue.cc:1059] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:49.845500 19779 raft_consensus.cc:3055] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:49.845865 19779 raft_consensus.cc:740] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:49.846920 19779 consensus_queue.cc:260] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:49.848258 19779 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:49.859256 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:49.836066 19799 consensus_queue.cc:260] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.866361 12625 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 21.
I20260709 13:20:49.868634 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:49.869700 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:49.870175 13382 raft_consensus.cc:1240] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:49.870638 19778 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:49.869340 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:20:49.890679 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 73669.5us
I20260709 13:20:49.895185 12747 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:49.898257 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:49.900158 19805 consensus_queue.cc:1059] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:49.901484 13382 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:49.901532 19805 raft_consensus.cc:3055] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:49.902064 19805 raft_consensus.cc:740] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:49.902963 19805 consensus_queue.cc:260] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.906001 19805 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:49.915126 19739 tablet_bootstrap.cc:492] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:49.916230 19739 tablet_bootstrap.cc:492] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:20:49.917006 19739 ts_tablet_manager.cc:1403] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.926s	user 0.067s	sys 0.039s
I20260709 13:20:49.918052 19779 consensus_queue.cc:1059] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:20:49.919200 19730 raft_consensus.cc:3055] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:49.920087 19730 raft_consensus.cc:740] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:49.920966 19730 consensus_queue.cc:260] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.922295 19730 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:49.924000 19781 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:49.895723 19799 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:49.926057 12782 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:49.927706 19782 consensus_queue.cc:1059] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:49.928757 19782 raft_consensus.cc:3055] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:49.929085 19782 raft_consensus.cc:740] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:49.930101 19782 consensus_queue.cc:260] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:49.931100 19782 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:49.934401 19775 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:49.934659 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:49.940631 12672 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:49.935879 19775 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:49.938053 19801 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:49.947177 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:20:49.946142 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:49.948966 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:49.946031 19801 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:49.944938 19804 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:49.951447 19801 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:49.953639 12782 raft_consensus.cc:1240] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:49.955538 12782 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:49.956830 13382 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:49.957106 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:49.959429 19784 consensus_queue.cc:1059] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:49.960999 19784 raft_consensus.cc:3055] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:49.961330 19784 raft_consensus.cc:740] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:49.959657 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:49.962352 19784 consensus_queue.cc:260] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:49.964294 19784 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:49.966703 12782 raft_consensus.cc:1240] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 10. Current term is 12. Ops: []
I20260709 13:20:49.958137 19807 consensus_queue.cc:1059] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 13:20:49.969103 19807 raft_consensus.cc:3055] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:49.969466 19807 raft_consensus.cc:740] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:49.969796 19804 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:49.970638 19763 consensus_queue.cc:1059] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:49.970455 19807 consensus_queue.cc:260] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.972466 19769 raft_consensus.cc:3055] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:49.972682 19807 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:49.972576 19763 consensus_queue.cc:1059] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:49.972962 19769 raft_consensus.cc:740] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:49.973871 19763 raft_consensus.cc:3055] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:49.974334 19763 raft_consensus.cc:740] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:49.974050 19769 consensus_queue.cc:260] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:49.975457 19769 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:20:49.975267 19763 consensus_queue.cc:260] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:49.977525 19763 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:49.986068 13382 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:49.986469 19805 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:49.991572 12672 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:49.992667 19781 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:49.993165 19804 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:49.996052 19739 raft_consensus.cc:359] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:49.999389 19782 consensus_queue.cc:1059] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:20:50.004550 12672 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:20:50.005160 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 54290.5us
W20260709 13:20:50.015406 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:50.020341 12625 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:20:50.066306 19804 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:49.984890 19801 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:50.060107 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 55656.1us
I20260709 13:20:50.068485 12672 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:50.068840 19801 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:50.069521 19801 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:20:50.033277 19739 raft_consensus.cc:740] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:20:50.072098 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:50.072746 12672 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:50.073629 19804 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:50.076994 12672 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:50.077926 19804 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:20:50.071157 12673 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:50.080032 19804 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:50.079890 19739 consensus_queue.cc:260] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:50.079157 12672 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:50.081240 19804 raft_consensus.cc:697] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:50.083992 19801 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:50.093530 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:50.099061 19739 ts_tablet_manager.cc:1434] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.182s	user 0.012s	sys 0.002s
I20260709 13:20:50.107460 19807 raft_consensus.cc:3055] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:20:50.108034 19807 raft_consensus.cc:740] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:50.098865 19804 consensus_queue.cc:237] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:50.099895 19809 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:50.100919 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:50.108994 19807 consensus_queue.cc:260] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:50.113274 19809 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:20:50.113761 19807 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:50.114842 12747 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:20:50.122860 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:50.123797 19809 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:50.126035 12747 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:50.127900 13349 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.141152 19755 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:50.144320 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:50.146359 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:50.137751 12598 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.149539 12598 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.151047 19782 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:50.151926 19782 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:50.154822 12598 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.155748 19782 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:50.157027 19809 raft_consensus.cc:2804] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:50.157678 19809 raft_consensus.cc:697] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:50.158936 19809 consensus_queue.cc:237] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:50.177397 12528 catalog_manager.cc:5697] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 14 to 15, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 15 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:50.185349 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 59379.5us
I20260709 13:20:50.186837 12747 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:50.188146 19775 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:50.188457 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:50.189433 19775 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:20:50.193679 12598 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.194589 19807 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:50.195750 12598 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.208384 12598 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.203073 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:50.209040 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:50.208665 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:50.213084 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:50.214435 12782 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:50.214388 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:50.216121 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:50.216698 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:50.217101 19775 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:50.210575 19782 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:50.219774 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:50.221743 12598 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.223062 12598 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.223909 19784 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:50.224583 19784 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:50.217658 19775 raft_consensus.cc:697] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:50.225318 19807 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:50.224625 12598 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.229224 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:50.234171 12598 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.231546 19784 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:50.252895 19775 consensus_queue.cc:237] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:50.260627 12706 raft_consensus.cc:1240] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:50.261859 19784 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:50.263173 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:50.263173 12598 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.271549 19784 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:50.273658 12598 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.274916 19784 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:50.279248 19784 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:50.280761 19807 raft_consensus.cc:3055] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:20:50.281212 19807 raft_consensus.cc:740] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:50.282227 19807 consensus_queue.cc:260] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:50.293824 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:50.294587 12701 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:20:50.299731 19807 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:50.299870 12709 raft_consensus.cc:1240] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:50.302273 12598 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.303114 19784 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:50.307891 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:50.308579 12709 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:50.309149 12701 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:20:50.318751 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:50.319585 12625 raft_consensus.cc:2368] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 11. Current term is 12.
I20260709 13:20:50.320755 12706 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:20:50.323423 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:50.328058 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:50.329053 12702 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:50.317210 12709 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:20:50.331604 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:50.332599 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:50.339654 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:50.340153 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:50.334393 12708 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:50.334532 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:50.341360 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:50.332368 12626 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:20:50.360704 12634 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
W20260709 13:20:50.362202 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 128530us
I20260709 13:20:50.363308 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.365321 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:50.367934 12718 tablet_copy_service.cc:342] P 639ab2db265f4747bb7afb93e833f530: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-98a43c5d78c945bb8c5541bc892f4865 received from {username='slave'} at 127.0.0.1:38496
I20260709 13:20:50.368417 12718 tablet_copy_service.cc:434] P 639ab2db265f4747bb7afb93e833f530: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-98a43c5d78c945bb8c5541bc892f4865 on tablet 98a43c5d78c945bb8c5541bc892f4865 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:50.370071 12709 raft_consensus.cc:1240] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:50.371423 12626 raft_consensus.cc:1240] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 24. Current term is 25. Ops: []
I20260709 13:20:50.375380 13378 raft_consensus.cc:1217] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEARNER]: Deduplicated request from leader. Original: 14.9->[14.10-14.10]   Dedup: 14.10->[]
W20260709 13:20:50.386459 19821 log.cc:927] Time spent T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:20:50.388171 13349 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.390456 19789 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:50.390991 12628 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:20:50.399161 12625 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:20:50.401160 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.396171 19775 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:50.400786 19807 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:50.409672 19807 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:50.410754 19744 tablet_bootstrap.cc:492] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=3 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:50.411918 19744 tablet_bootstrap.cc:492] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:20:50.412757 19744 ts_tablet_manager.cc:1403] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.583s	user 0.047s	sys 0.010s
I20260709 13:20:50.412801 19778 consensus_queue.cc:1059] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:20:50.414324 19801 raft_consensus.cc:3055] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:50.414800 19801 raft_consensus.cc:740] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:50.416090 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:50.415796 19801 consensus_queue.cc:260] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:50.416833 12782 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:20:50.417789 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:50.398542 12525 catalog_manager.cc:5697] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 18 to 19, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 19 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:50.418704 13378 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 14: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:20:50.419555 19801 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:50.419248 19807 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:50.425173 19804 consensus_queue.cc:1059] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:20:50.426287 19781 raft_consensus.cc:3055] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:20:50.426702 19781 raft_consensus.cc:740] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:50.427597 19781 consensus_queue.cc:260] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:50.428956 19781 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:20:50.394723 13349 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.432857 19789 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:20:50.438355 19744 raft_consensus.cc:359] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:50.442440 19744 raft_consensus.cc:740] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:20:50.444073 13349 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.443547 19744 consensus_queue.cc:260] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:50.444895 19789 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:20:50.448014 19744 ts_tablet_manager.cc:1434] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.001s
W20260709 13:20:50.460454 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 59606.7us
I20260709 13:20:50.461521 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.462211 19775 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:50.466533 19785 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:50.467815 12747 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.468778 19785 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:20:50.469993 19789 consensus_queue.cc:1059] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:50.470194 13349 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.471129 19790 raft_consensus.cc:3055] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:50.471135 19791 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:50.471789 19790 raft_consensus.cc:740] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:50.472658 12747 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.472760 19790 consensus_queue.cc:260] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:50.473533 19785 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:20:50.474151 19790 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:50.474999 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.475842 19785 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:20:50.477088 12747 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.477885 19785 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:20:50.480012 12644 tablet_copy_service.cc:342] P 19cd864c053744bebd96093a7982537e: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-20db0f202ea546699789c2325d241b35 received from {username='slave'} at 127.0.0.1:39590
I20260709 13:20:50.480523 12644 tablet_copy_service.cc:434] P 19cd864c053744bebd96093a7982537e: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-20db0f202ea546699789c2325d241b35 on tablet 20db0f202ea546699789c2325d241b35 with peer 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:50.481902 19785 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:50.482802 19785 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:50.484756 19775 consensus_queue.cc:1059] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:50.486135 19785 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:50.486806 19809 raft_consensus.cc:3055] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:50.489176 19809 raft_consensus.cc:740] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:50.490226 19809 consensus_queue.cc:260] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:50.491616 19809 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:50.494374 19806 consensus_queue.cc:1059] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:50.495843 19785 raft_consensus.cc:3055] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:50.496263 19785 raft_consensus.cc:740] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:50.497380 19785 consensus_queue.cc:260] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:50.500419 19785 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:50.508404 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:50.511382 12672 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:50.513844 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:50.515443 19801 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:50.515817 12672 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:50.513381 19775 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:50.515432 19805 consensus_queue.cc:1059] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:20:50.521770 19775 raft_consensus.cc:697] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:50.513171 19806 consensus_queue.cc:1059] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.008s
I20260709 13:20:50.523456 19775 consensus_queue.cc:237] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:50.536983 12775 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:50.538564 19826 raft_consensus.cc:3055] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:20:50.539011 19826 raft_consensus.cc:740] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:50.539951 19826 consensus_queue.cc:260] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:50.541472 19826 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:50.544513 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:50.545893 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:50.548199 12747 leader_election.cc:400] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 11. Current term is 12.
I20260709 13:20:50.548506 12747 leader_election.cc:403] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:20:50.550976 12672 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:50.551988 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:50.552604 12774 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 23. Current term is 24. Ops: []
I20260709 13:20:50.565376 12782 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:20:50.567785 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:50.569635 19818 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:20:50.532570 19785 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:50.571444 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:50.535398 19801 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:50.565002 19778 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:50.582551 19804 raft_consensus.cc:1064] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:20:50.584136 19806 raft_consensus.cc:3055] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:50.591882 19778 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:20:50.590448 19785 raft_consensus.cc:697] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:50.595183 19785 consensus_queue.cc:237] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:50.588474 19806 raft_consensus.cc:740] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:50.604120 19806 consensus_queue.cc:260] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:50.607250 12747 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:50.610396 12747 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:50.608633 19781 raft_consensus.cc:1064] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:20:50.619239 12598 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.620891 19807 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:20:50.612154 19823 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:50.627416 19826 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Leader election lost for term 11. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 11. Current term is 12.
I20260709 13:20:50.629141 19823 raft_consensus.cc:697] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:50.630335 19823 consensus_queue.cc:237] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:50.633936 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:20:50.634502 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:50.635902 12598 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.638650 19807 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:50.647742 19806 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:50.615849 19805 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:50.648466 19781 consensus_queue.cc:237] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 9, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:50.657191 12782 raft_consensus.cc:1275] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:20:50.672184 19819 consensus_queue.cc:1048] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:50.658818 19805 raft_consensus.cc:697] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:50.677006 19805 consensus_queue.cc:237] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:50.649695 19804 consensus_queue.cc:237] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:50.708936 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 96563.9us
I20260709 13:20:50.720707 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:50.756397 12703 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:20:50.714205 19755 consensus_queue.cc:1059] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.028s
W20260709 13:20:50.774237 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 119909us
I20260709 13:20:50.756379 19826 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:20:50.796814 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 53242.7us
I20260709 13:20:50.827185 12625 raft_consensus.cc:1275] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:20:50.791796 19807 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:50.843920 19831 raft_consensus.cc:3055] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:20:50.866659 19831 raft_consensus.cc:740] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:50.868167 19831 consensus_queue.cc:260] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:50.829228 19826 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:20:50.849063 19781 consensus_queue.cc:1048] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:20:50.876529 19831 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:50.910782 19807 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:50.911505 19807 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:50.914815 12634 raft_consensus.cc:1275] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:20:50.917464 12634 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:20:50.918838 19781 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:50.927913 19807 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:50.933669 19781 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:50.935361 19807 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:50.936038 19807 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:50.938705 19807 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:50.947235 13378 raft_consensus.cc:3060] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 LEARNER]: Advancing to term 19
I20260709 13:20:50.955029 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:50.955942 12774 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:20:50.956467 13378 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 LEARNER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:20:50.957175 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:50.958002 12782 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:20:50.973453 12782 raft_consensus.cc:1275] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:20:50.977186 12782 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:20:50.986058 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:50.988891 12597 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:50.991274 19819 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:50.994500 19819 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:51.002619 19805 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:51.016173 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:51.020457 13378 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:20:51.018227 19807 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:51.030164 19807 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:51.030670 19807 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:50.993113 12528 catalog_manager.cc:5697] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 20 to 21, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 21 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:51.058983 12628 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:20:51.064127 12528 catalog_manager.cc:5697] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 13 to 14, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 14 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:51.068662 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:20:51.071272 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 82488.8us
I20260709 13:20:51.071559 19854 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:51.072062 19854 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:51.072477 19854 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:51.075419 12528 catalog_manager.cc:5697] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 18 to 19, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 19 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:51.080175 13377 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEARNER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:20:51.081881 19789 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:51.082464 19789 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:51.084556 19789 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:51.088869 12528 catalog_manager.cc:5697] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 10 to 12, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 12 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:51.092319 13377 raft_consensus.cc:1275] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 LEARNER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:20:51.097450 12528 catalog_manager.cc:5697] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 18 to 19, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 19 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:51.144582 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:51.146342 12630 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:20:51.152436 12782 raft_consensus.cc:2393] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 18: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:51.153542 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:51.156539 19860 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.161814 19807 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:51.151393 19844 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:51.221964 19844 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:51.224524 19844 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:51.165748 12626 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:20:51.203720 19781 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:51.203068 19801 raft_consensus.cc:2955] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Committing config change with OpId 17.12: config changed from index 11 to 12, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:51.230240 19781 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:51.231050 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 55246.9us
I20260709 13:20:51.232679 19781 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:51.206426 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 54229.3us
W20260709 13:20:51.206795 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 55388.9us
I20260709 13:20:51.235980 19826 raft_consensus.cc:1064] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
W20260709 13:20:51.236529 19826 raft_consensus.cc:1068] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2: Unable to promote non-voter 522fd2ba0e9f402691b5ef29af8579cd: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:20:51.237042 19826 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:51.240797 12709 raft_consensus.cc:1275] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:20:51.242800 19829 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:51.244498 19829 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:51.245581 13382 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:51.245659 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:51.247395 12629 raft_consensus.cc:1275] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:20:51.249778 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 96659us
I20260709 13:20:51.250948 12630 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:20:51.210777 19807 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:51.252007 19807 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.251899 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:51.252550 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 90978.1us
I20260709 13:20:51.252840 12632 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 20.
I20260709 13:20:51.215112 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:51.254153 13378 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:51.253880 19819 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:51.257485 19819 raft_consensus.cc:1064] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:20:51.259215 12703 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:51.261046 19845 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:51.264374 12673 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:51.265290 19804 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:51.265775 19804 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:51.266330 19804 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:51.252569 19807 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:51.273721 19775 consensus_queue.cc:1048] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:51.287949 13376 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:51.294781 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:51.295609 12774 raft_consensus.cc:2393] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 20: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:51.268625 19818 raft_consensus.cc:2955] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Committing config change with OpId 14.12: config changed from index 11 to 12, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:51.302211 12701 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:20:51.302865 19854 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:51.304064 12625 raft_consensus.cc:2368] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 18. Current term is 19.
I20260709 13:20:51.307099 13349 leader_election.cc:400] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 18. Current term is 19.
I20260709 13:20:51.307394 13349 leader_election.cc:403] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:51.308388 12634 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:20:51.308976 12775 raft_consensus.cc:2955] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Committing config change with OpId 14.12: config changed from index 11 to 12, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:51.308790 19807 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:51.314759 19844 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:20:51.315062 12774 raft_consensus.cc:2955] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 11 to 12, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:51.317417 19805 consensus_queue.cc:1048] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:51.323936 19826 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:51.327124 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:51.327997 12783 raft_consensus.cc:2393] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 11: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:51.328967 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:51.335072 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:51.342455 19845 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:51.343160 19845 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:51.344154 19819 consensus_queue.cc:237] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:51.346239 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:51.346462 19845 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:51.347038 12625 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:20:51.348279 12673 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:51.258920 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:51.355280 13376 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:20:51.355819 12782 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:51.363786 12597 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:51.371434 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:51.372926 13376 raft_consensus.cc:2393] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 14: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:20:51.375306 12703 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:20:51.380316 19830 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:51.380852 19830 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:51.381256 19830 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:51.387192 19844 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 18. Current term is 19.
I20260709 13:20:51.389573 19781 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:51.390218 19781 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:51.390690 19805 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:51.392418 19781 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:51.393080 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:51.393877 12629 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 20.
I20260709 13:20:51.395043 12673 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:51.395865 19778 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:51.396279 19778 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:51.396620 19778 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:51.360649 13377 raft_consensus.cc:2955] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Committing config change with OpId 14.12: config changed from index 11 to 12, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:51.397697 19805 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:51.399538 19826 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:51.406263 19804 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:51.377813 19829 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:20:51.411036 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:51.417737 13380 raft_consensus.cc:2393] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 20: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:20:51.420181 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:51.418171 12626 raft_consensus.cc:2955] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Committing config change with OpId 14.12: config changed from index 11 to 12, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:51.440521 19804 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:51.442291 19778 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:51.376097 19860 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:51.444738 12628 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 11.
I20260709 13:20:51.445240 19860 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:51.434008 19789 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:51.446406 19789 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:51.448918 19789 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:51.375500 19858 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:51.450551 13349 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:51.375617 19755 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:20:51.485204 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 61175.2us
I20260709 13:20:51.488315 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:51.489933 12783 raft_consensus.cc:2393] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 20: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:51.494814 12782 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:20:51.502346 13378 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:20:51.507428 13382 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:20:51.468619 19831 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:51.445313 19830 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:51.532891 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:51.535346 12775 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 22.
I20260709 13:20:51.531793 13383 raft_consensus.cc:2955] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 11 to 12, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:51.507447 19778 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:51.514753 19790 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.487892 19755 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:51.529165 19806 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:51.526930 19831 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:51.561501 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:51.576713 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:51.577852 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:51.579061 19860 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:51.579635 19860 raft_consensus.cc:697] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:51.580539 19860 consensus_queue.cc:237] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:51.587493 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:51.591097 12634 raft_consensus.cc:2955] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 11 to 12, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:51.610699 13348 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:51.571221 19831 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:51.544029 19790 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:51.571082 19806 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:51.629236 19842 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:51.646639 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:51.648286 12782 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:51.660844 12782 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:20:51.666379 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:20:51.665845 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 53405.3us
I20260709 13:20:51.674476 12775 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:20:51.641377 19854 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:51.654503 19858 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:51.681253 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 45383.8us
I20260709 13:20:51.684276 19830 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:51.658674 19842 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:51.687760 19755 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:51.696777 19807 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.698189 12632 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 20.
I20260709 13:20:51.698925 12633 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 20.
I20260709 13:20:51.713109 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:51.727820 19871 log.cc:927] Time spent T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:20:51.705742 19858 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:51.731859 19854 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.733052 19858 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:51.732659 19854 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:51.734747 19869 log.cc:927] Time spent T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 13:20:51.735850 19854 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:51.750381 12634 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 20.
I20260709 13:20:51.709955 19842 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:51.764760 12597 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:51.765899 12597 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:51.766862 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:51.767750 12775 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 20.
I20260709 13:20:51.769685 12597 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:51.715490 19807 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:51.774164 19858 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:51.784647 19806 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:51.767658 19843 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:51.773474 19854 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:51.795680 19884 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:51.796294 19884 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:51.797289 19862 log.cc:927] Time spent T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.315s	user 0.006s	sys 0.000s
I20260709 13:20:51.767060 19809 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:20:51.804538 19790 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:51.806740 19854 raft_consensus.cc:697] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:51.807333 19809 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.807914 19809 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:51.808157 19854 consensus_queue.cc:237] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:51.790295 19858 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:51.809866 19858 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:51.810861 19809 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:51.793743 19805 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.819099 19805 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:51.824663 19807 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:51.800117 19843 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:51.827888 19843 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:51.835834 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:51.836946 13382 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 21.
I20260709 13:20:51.847380 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:51.848273 12775 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 22.
I20260709 13:20:51.849979 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:51.854164 12597 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:51.792917 19860 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:51.873941 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:51.877279 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:51.878769 12633 raft_consensus.cc:2393] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 25: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:20:51.882791 13383 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:20:51.874568 19860 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:51.885233 19860 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:51.799741 19884 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:51.880013 12597 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:51.888868 19842 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:51.877162 19807 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:20:51.872645 19892 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:51.898605 19805 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:51.906497 19831 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:51.915907 19847 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.916584 19847 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:51.917834 19876 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.918426 19876 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:51.919059 19847 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:51.920786 19876 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:51.925835 13378 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:20:51.929461 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:51.930371 12706 raft_consensus.cc:2393] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 22: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:51.930871 19831 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:51.931305 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:51.932161 12706 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 16: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:51.934656 19892 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:51.938269 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:51.938963 12708 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 25.
I20260709 13:20:51.910631 12673 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:51.913724 19807 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:51.954895 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 60562.7us
I20260709 13:20:51.958601 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:51.959605 13385 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:20:51.974020 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:51.974812 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:51.975219 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:20:51.970810 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 60267.9us
I20260709 13:20:51.976214 12703 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 23.
I20260709 13:20:51.978812 19898 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:51.979209 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:51.979594 19898 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:51.980193 19898 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:51.982527 12706 raft_consensus.cc:2393] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 16: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:51.977015 12673 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:51.987176 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:51.988178 12775 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 20.
W20260709 13:20:51.984565 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 104633us
I20260709 13:20:51.968116 19807 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:51.997843 12775 raft_consensus.cc:1275] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:20:52.001961 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:52.003149 12775 raft_consensus.cc:2393] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 20: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:52.004110 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:52.010244 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:20:52.024350 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 47448.7us
I20260709 13:20:52.030704 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:52.031855 12706 raft_consensus.cc:2393] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 23: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:51.970520 19781 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:20:52.033809 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 74626.1us
I20260709 13:20:52.036221 12774 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:20:52.036521 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:52.037911 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:52.034626 19884 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:52.041287 19884 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.043251 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:52.043660 19884 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:52.034798 19872 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:52.045836 19886 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:51.986176 19830 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.046159 19872 raft_consensus.cc:697] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 20 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:52.046583 19830 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:52.047168 19872 consensus_queue.cc:237] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.047746 12708 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:20:52.038585 12625 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 25.
I20260709 13:20:52.048966 19830 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:52.051118 12625 raft_consensus.cc:1275] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:20:52.053071 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:52.052132 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:52.054564 12706 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:20:52.056488 12775 raft_consensus.cc:2393] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 11: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:51.984025 19880 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:52.068697 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:52.006943 19875 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.062804 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:51.995092 19819 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:51.968238 19860 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.069721 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:52.059083 12632 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:20:52.068678 13348 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:52.072459 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:52.079406 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:52.079914 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:52.080297 12775 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:20:52.080734 13377 raft_consensus.cc:2393] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 20: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:20:52.078868 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:52.081920 13348 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:52.082388 13380 raft_consensus.cc:2393] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 14: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:20:52.045851 19854 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:52.087999 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:52.089340 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:52.086445 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:52.101613 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:52.103271 12775 raft_consensus.cc:2393] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 20: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:52.100668 13383 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
I20260709 13:20:52.082995 19831 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:20:52.105059 19854 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:52.107843 19894 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.108991 19860 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:52.111797 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:52.145736 13376 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:20:52.054805 19781 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.119221 13380 raft_consensus.cc:1275] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEARNER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:20:52.118323 19833 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.046375 19886 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.078250 19858 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.103361 19884 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:52.086403 19892 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:52.154342 12706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:20:52.103704 19845 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:52.160594 12706 raft_consensus.cc:606] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:20:52.079331 19875 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:52.144527 19894 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.082638 19819 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.160012 19833 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.118727 19831 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:52.163497 19781 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:52.163770 19883 raft_consensus.cc:476] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Not starting pre-election -- already a leader
I20260709 13:20:52.164183 12701 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 20.
I20260709 13:20:52.161008 12628 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:20:52.166219 19886 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:52.168829 19864 log.cc:927] Time spent T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.614s	user 0.002s	sys 0.002s
I20260709 13:20:52.227427 19825 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.227984 19825 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:52.231024 19819 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:52.227448 19833 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:52.233428 19825 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:52.233820 19894 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:52.169703 13376 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 21.
I20260709 13:20:52.176249 19854 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:52.234514 19831 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:20:52.160048 19884 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:52.237859 19884 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:52.239828 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:52.240013 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:52.240520 12783 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:52.240649 12774 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:20:52.182701 19842 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:52.182853 13349 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:52.219967 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:52.177948 12531 catalog_manager.cc:5697] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 11 to 12, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 14 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:52.242595 19842 raft_consensus.cc:2764] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:20:52.184765 19858 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:52.185861 19898 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.244851 19898 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:52.195812 19841 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.198822 12775 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:20:52.245584 19883 raft_consensus.cc:1064] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:20:52.245505 19841 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.248313 19841 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:52.245846 12775 raft_consensus.cc:606] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Received request to transfer leadership to TS 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:52.248450 19883 consensus_queue.cc:237] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 12, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.176664 19845 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:52.249543 19845 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:20:52.249646 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 66876.7us
I20260709 13:20:52.250468 19875 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:52.251614 12783 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 21.
I20260709 13:20:52.239928 19886 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.253929 19886 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:52.255870 19886 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:52.256021 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:52.256699 12630 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 20.
I20260709 13:20:52.258780 19894 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.185777 12631 raft_consensus.cc:2368] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 11. Current term is 12.
I20260709 13:20:52.249116 12775 raft_consensus.cc:614] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:52.259402 19894 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.216566 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:52.260557 13385 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
I20260709 13:20:52.225087 19807 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:52.262567 13349 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:52.263746 19841 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:52.264029 12629 raft_consensus.cc:2368] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 23. Current term is 24.
I20260709 13:20:52.264128 19841 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:52.264729 19841 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:52.265475 12775 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:20:52.266052 12775 raft_consensus.cc:606] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:20:52.266366 12775 raft_consensus.cc:614] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:52.267443 19807 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:52.268030 19894 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:52.271204 12775 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:20:52.271837 12775 raft_consensus.cc:606] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:20:52.274386 19841 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.275945 13349 leader_election.cc:400] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 11. Current term is 12.
I20260709 13:20:52.279301 13349 leader_election.cc:403] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:20:52.278765 12775 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fa34731f37084c608660f0088a2dba09"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:20:52.278777 12531 catalog_manager.cc:5697] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 11 to 12, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 17 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:52.280164 19791 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:52.280606 12775 raft_consensus.cc:606] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:20:52.284238 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:52.284878 12629 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:20:52.288767 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 216373us
I20260709 13:20:52.299464 12746 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:52.305979 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:52.306818 13383 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:52.305189 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:20:52.311165 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 238583us
W20260709 13:20:52.314165 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 216751us
I20260709 13:20:52.321106 12597 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:52.310628 19880 raft_consensus.cc:697] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 20 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:52.324471 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:52.330996 12597 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:52.331979 13376 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 24.
I20260709 13:20:52.334800 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:52.335155 12673 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:52.335507 12775 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:20:52.335508 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:52.336378 12783 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:52.337634 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:52.338227 12774 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:20:52.336601 12747 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:52.332999 12672 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:52.342813 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:52.348151 12629 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 21.
I20260709 13:20:52.330623 19805 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:52.351068 19805 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:52.351456 19805 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:52.351784 19902 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:20:52.352327 19902 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:52.352710 19902 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:20:52.353734 19885 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:52.354315 19885 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:52.354717 19885 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:52.355475 19848 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:52.355947 19848 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:52.356295 19848 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:52.358639 12775 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:20:52.359223 12775 raft_consensus.cc:606] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 23 LEADER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:20:52.364394 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:52.365105 12779 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:20:52.373788 12783 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:20:52.346750 19854 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:52.366271 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:52.366994 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:52.377171 12701 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:20:52.365186 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:52.381625 12774 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 24.
I20260709 13:20:52.381956 12706 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:52.367376 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:52.365713 19827 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:52.333859 19858 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:52.382886 12782 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 14: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:20:52.375679 12597 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:52.388325 12701 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 22.
I20260709 13:20:52.375411 19841 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:52.399307 12673 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:52.401718 19827 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:52.402305 19827 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:52.404137 12706 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 24.
I20260709 13:20:52.401778 19819 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:52.405357 19819 raft_consensus.cc:697] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:52.406105 19880 consensus_queue.cc:237] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:52.406358 19819 consensus_queue.cc:237] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.409013 19858 raft_consensus.cc:697] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 21 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:52.409348 12597 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:52.411528 12746 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:52.415755 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:52.416530 12625 raft_consensus.cc:2368] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 21. Current term is 22.
I20260709 13:20:52.419106 13383 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 22.
I20260709 13:20:52.410054 19858 consensus_queue.cc:237] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.322244 19884 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:52.428810 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 51380.1us
I20260709 13:20:52.430070 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:52.431712 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:20:52.432665 12629 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:20:52.433676 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:52.410655 19860 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:52.439765 12747 leader_election.cc:400] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 23. Current term is 24.
I20260709 13:20:52.376847 19854 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:52.441210 12747 leader_election.cc:403] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:52.446614 12673 leader_election.cc:400] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 21. Current term is 22.
I20260709 13:20:52.447002 12673 leader_election.cc:403] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:52.446993 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:52.447885 12701 raft_consensus.cc:2393] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 22: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:52.449458 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:52.439833 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:52.447629 12598 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:20:52.491514 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 43983.1us
I20260709 13:20:52.491065 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:52.429001 19892 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:52.428110 19874 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:52.428738 19872 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.492671 13380 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:20:52.494012 12598 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:52.494688 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:52.450874 19854 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:52.537267 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:52.441040 19860 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:20:52.562301 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 122681us
I20260709 13:20:52.423779 19845 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.428520 19805 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.493920 19892 raft_consensus.cc:697] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:52.494462 19894 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:20:52.571657 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 222334us
I20260709 13:20:52.582408 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:52.583180 12704 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:20:52.586124 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:52.514938 19791 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 11. Current term is 12.
I20260709 13:20:52.593678 12704 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:20:52.498759 19907 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:20:52.540549 19876 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 21. Current term is 22.
I20260709 13:20:52.599152 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:52.626834 12703 raft_consensus.cc:2368] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 20. Current term is 21.
W20260709 13:20:52.541785 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 132511us
I20260709 13:20:52.646314 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:52.568621 19860 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:52.569361 19826 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 23. Current term is 24.
I20260709 13:20:52.657728 13383 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:20:52.663681 13376 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:20:52.667142 13382 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 LEARNER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:20:52.593698 19909 consensus_queue.cc:1048] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.351s
I20260709 13:20:52.672267 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:52.682852 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:52.588671 19843 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.671514 13349 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:52.509886 19872 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.691063 13380 raft_consensus.cc:2368] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 21. Current term is 22.
I20260709 13:20:52.605846 19833 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.600981 19907 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:52.650868 19807 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:20:52.547230 19831 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.538877 19874 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.706296 19889 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:52.706934 19881 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:52.708158 19919 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:52.709143 19805 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:52.712445 19878 raft_consensus.cc:993] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2: : Instructing follower 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:52.722743 19845 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:52.762315 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 58930.2us
I20260709 13:20:52.724889 19831 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:52.725363 13377 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 25.
W20260709 13:20:52.729826 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:52.731622 19872 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:52.732144 19892 consensus_queue.cc:237] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.696280 19843 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.593771 19894 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.598187 19848 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:52.687680 19818 consensus_queue.cc:1048] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:20:52.650326 19885 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.745932 12706 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 20.
I20260709 13:20:52.696537 19907 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:20:52.752028 12709 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 23.
W20260709 13:20:52.756754 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 262815us
I20260709 13:20:52.762271 12775 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:20:52.763017 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 84114.1us
I20260709 13:20:52.763675 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:52.792303 13382 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:52.768461 12782 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:20:52.784087 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:52.796599 12631 raft_consensus.cc:2393] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 25: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:20:52.798695 12632 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:20:52.806209 13378 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 14: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:20:52.774447 19878 raft_consensus.cc:1081] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Signalling peer 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:52.747141 19833 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.809115 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:52.769908 19919 raft_consensus.cc:697] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:52.811547 12627 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
I20260709 13:20:52.814186 19915 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 13:20:52.814618 19894 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:52.814972 19918 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:20:52.815666 19914 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:20:52.816301 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:52.796984 19884 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:52.770156 19806 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:20:52.776101 19827 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:52.763767 19881 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.843533 19848 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:52.821807 12708 raft_consensus.cc:2368] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17.
I20260709 13:20:52.861886 19823 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:20:52.864033 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 50638.4us
I20260709 13:20:52.911032 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:52.877895 13382 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 21.
I20260709 13:20:52.882279 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:52.927357 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:52.883781 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:52.887703 19914 raft_consensus.cc:697] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 24 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:52.861474 13385 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:20:52.847206 19781 consensus_queue.cc:1048] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:20:52.935846 19928 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:52.937621 19854 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:52.938472 19843 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:52.940080 19854 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:52.940738 19854 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:52.941730 19925 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 13:20:52.883036 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:52.964408 12775 raft_consensus.cc:2393] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 21: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:52.880983 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:52.930024 12630 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:20:52.944463 12629 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:20:52.923444 19823 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.956303 19833 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:52.958092 19881 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:52.959861 19827 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:52.961676 19885 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:52.962044 19922 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:20:52.968946 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 88041.9us
W20260709 13:20:52.923408 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 122723us
I20260709 13:20:52.970890 12782 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 22.
I20260709 13:20:52.973384 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:52.976428 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:52.980127 13382 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:52.984172 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:52.985396 13348 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:52.965161 19874 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:52.988366 19809 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:52.991506 19809 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.993673 19843 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:20:52.994518 19843 raft_consensus.cc:697] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:52.995728 19843 consensus_queue.cc:237] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:52.993711 19927 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:52.998534 19927 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:52.998943 19927 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:52.987669 19922 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:53.002487 19809 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:52.941501 19854 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:53.004381 19922 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:53.004236 19914 consensus_queue.cc:237] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:53.020017 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:53.048493 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:53.050341 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:53.051183 12776 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:20:53.076841 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:20:53.078509 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 65799.2us
I20260709 13:20:53.077761 12778 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 17.
I20260709 13:20:53.079829 12746 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:53.088481 12779 raft_consensus.cc:2368] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 16. Current term is 17.
I20260709 13:20:53.090641 13382 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
W20260709 13:20:53.014698 19902 consensus_meta.cc:287] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.661s	user 0.008s	sys 0.006s
W20260709 13:20:53.159437 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 56742.3us
W20260709 13:20:53.162058 19916 log.cc:927] Time spent T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.269s	user 0.005s	sys 0.000s
I20260709 13:20:53.175199 12746 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:53.175529 13349 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:53.077944 19920 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:53.178681 19885 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:53.178956 19884 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:52.986701 19775 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.119s
I20260709 13:20:53.180807 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:53.181756 12634 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:20:53.181955 19860 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:53.221253 19860 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:53.183560 13349 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:53.174360 19902 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:53.225885 19844 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:53.226331 19844 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:53.226709 19844 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:53.186851 19907 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:53.231063 12774 raft_consensus.cc:2393] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 25: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:53.232003 19907 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:53.203742 19885 raft_consensus.cc:697] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:53.208949 19883 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:53.233896 19883 raft_consensus.cc:697] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:53.234797 19883 consensus_queue.cc:237] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:53.212355 19805 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.237973 19805 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:53.240232 19805 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:53.176844 19925 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:52.981978 19919 consensus_queue.cc:237] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:53.181141 19884 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:53.244963 19884 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:53.184814 19920 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:53.256726 19920 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:53.241093 19925 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:53.258322 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:53.260454 19925 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:53.264276 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:53.264501 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:53.265178 13384 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:20:53.265440 13385 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:20:53.233268 19885 consensus_queue.cc:237] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:53.264501 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:53.266459 12774 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:20:53.271888 12746 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:53.275259 19902 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 26 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:53.277498 19775 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:53.273212 13385 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 25.
I20260709 13:20:53.277956 19775 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:53.278426 19775 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:53.283859 12746 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:53.291390 19805 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:20:53.314625 13377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51762 (request call id 1389) took 1213 ms. Trace:
I20260709 13:20:53.315129 13377 rpcz_store.cc:276] 0709 13:20:52.101022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:52.101251 (+   229us) service_pool.cc:224] Handling call
0709 13:20:53.314392 (+1213141us) spinlock_profiling.cc:243] Waited 622 ms on lock 0x7b54005f0354. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:20:53.314517 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1431141120}
I20260709 13:20:53.329735 19805 raft_consensus.cc:697] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 25 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:53.330852 19805 consensus_queue.cc:237] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:53.339061 19844 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:53.339851 19871 log.cc:927] Time spent T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.849s	user 0.011s	sys 0.000s
I20260709 13:20:53.343529 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:53.344244 12629 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:53.350618 19809 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:53.351145 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:53.352069 19775 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:53.294070 19927 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:53.354804 12782 raft_consensus.cc:2368] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17.
I20260709 13:20:53.356371 19927 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:53.353300 19809 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:53.356992 19809 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:53.363184 19884 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:53.364445 19823 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:53.365842 19775 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:53.373266 12629 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:20:53.375272 13349 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:53.337080 19854 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:53.379567 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:53.380275 13377 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:20:53.381328 12627 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fa34731f37084c608660f0088a2dba09"
dest_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:56838
I20260709 13:20:53.382155 12627 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:53.382617 12627 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:53.385807 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:53.386768 13380 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:53.390125 19844 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:53.394876 19884 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:53.395155 19920 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:53.396006 19920 raft_consensus.cc:697] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 15 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:53.357415 19809 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:53.400497 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:53.411852 12633 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:53.398011 12627 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:53.385046 19829 raft_consensus.cc:993] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2: : Instructing follower 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:53.375867 19922 raft_consensus.cc:993] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2: : Instructing follower 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:53.418022 12630 raft_consensus.cc:1240] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:53.425160 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:53.425928 12774 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:20:53.431257 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.434137 12627 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:53.397419 19920 consensus_queue.cc:237] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:53.445475 19930 raft_consensus.cc:1081] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Signalling peer 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:53.462379 12633 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:20:53.480671 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:53.478596 13380 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
I20260709 13:20:53.481838 12627 raft_consensus.cc:2393] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 15: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:20:53.459995 19922 raft_consensus.cc:1081] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 23 LEADER]: Signalling peer 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:53.483757 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:53.497546 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:53.498428 12627 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:53.505956 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:53.513933 12777 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:20:53.514060 12627 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
I20260709 13:20:53.514086 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:53.515153 12783 raft_consensus.cc:2368] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17.
I20260709 13:20:53.516444 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:53.517135 12776 raft_consensus.cc:3055] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:53.513298 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:53.521041 12779 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:20:53.521152 12597 leader_election.cc:400] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17.
I20260709 13:20:53.521521 12597 leader_election.cc:403] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:20:53.517534 12776 raft_consensus.cc:740] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:53.524338 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:53.523267 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:53.525028 12778 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:20:53.523142 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:53.525333 19900 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17.
I20260709 13:20:53.526757 19900 consensus_queue.cc:1059] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:53.527918 19915 raft_consensus.cc:3055] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:53.525331 12633 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:20:53.536935 13380 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:20:53.531762 12597 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:53.539503 12597 leader_election.cc:400] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17.
I20260709 13:20:53.539831 12597 leader_election.cc:403] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:20:53.547731 12774 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:20:53.533705 12776 consensus_queue.cc:260] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:53.565277 12776 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:53.550854 13381 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:20:53.556439 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:53.538432 12746 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:53.566469 12709 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:20:53.516981 19807 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.011s	sys 0.009s
I20260709 13:20:53.557459 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:53.567282 13348 leader_election.cc:400] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 16. Current term is 17.
I20260709 13:20:53.567530 12703 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:53.567643 13348 leader_election.cc:403] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:20:53.528296 19915 raft_consensus.cc:740] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:53.576035 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:20:53.571585 19807 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.012s	sys 0.009s
I20260709 13:20:53.589802 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:53.554373 19801 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:53.533322 19914 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:20:53.570401 19860 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:53.566962 19925 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:53.556068 19900 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:53.566763 19826 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:53.579074 19844 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:53.610914 12709 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 21.
I20260709 13:20:53.613111 12631 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa"
dest_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:56838
I20260709 13:20:53.591012 12708 raft_consensus.cc:2393] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 17: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:53.613590 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:53.602599 19893 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:53.571401 19775 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:53.635257 12703 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
I20260709 13:20:53.592850 12747 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:53.576077 19909 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:53.575152 19884 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 16. Current term is 17.
I20260709 13:20:53.600104 12700 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:20:53.578987 19825 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:53.579689 19892 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:53.650583 12632 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7"
dest_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:56838
I20260709 13:20:53.651366 12632 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:53.651813 12632 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:53.614486 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:53.595216 19807 raft_consensus.cc:697] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 22 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:53.654093 19807 consensus_queue.cc:237] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:53.616029 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:53.617609 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:20:53.638813 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 46067.8us
I20260709 13:20:53.634668 19844 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:53.660899 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:53.647882 19805 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:53.648277 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:53.613690 12631 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:53.668236 12631 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:53.662113 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:53.590167 19917 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:53.609094 19900 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:53.610836 19860 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:53.607697 19801 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:53.694949 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:53.617496 19826 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:53.711951 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 120258us
W20260709 13:20:53.714257 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 1.25198e+06us
I20260709 13:20:53.717140 12774 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:53.645279 19893 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:53.666252 19898 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:53.648151 19775 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:53.700588 19895 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:53.688315 19825 raft_consensus.cc:697] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:53.693681 19922 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:53.693378 19900 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:53.716156 19829 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:20:53.744632 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.723757 12632 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:53.744218 19854 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:53.722541 19827 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:53.753537 19854 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:53.753860 19827 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:53.755918 19827 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:53.755925 19854 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:53.757443 19826 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:53.757911 19801 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:53.747314 19829 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:53.762051 19829 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:53.658305 19909 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:53.763285 19860 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:53.764053 19860 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:53.764345 19909 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:53.666123 19892 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:53.697014 19917 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:53.767076 19892 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:53.768256 19917 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:53.684491 19941 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.086s
I20260709 13:20:53.764787 19860 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:53.748121 19898 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:53.782137 19775 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:53.685928 19915 consensus_queue.cc:260] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:53.748899 19895 raft_consensus.cc:697] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:53.787645 12776 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 20.
I20260709 13:20:53.788164 19805 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:53.779685 19893 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:53.782069 19898 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:53.802595 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:53.805625 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 57890.3us
I20260709 13:20:53.802918 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:53.806528 12626 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 21.
W20260709 13:20:53.797551 19945 log.cc:927] Time spent T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:20:53.799458 12631 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:53.808511 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:53.808790 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:53.809365 12774 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 21.
I20260709 13:20:53.811506 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:53.810928 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:53.811719 19915 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:53.813485 12597 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:53.814888 12597 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:53.815164 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:53.815737 12705 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:20:53.816622 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:53.817878 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:53.817950 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:53.818853 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:53.819236 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:53.819442 12774 raft_consensus.cc:3055] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:53.819815 12774 raft_consensus.cc:740] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:53.803567 19858 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:20:53.822638 19953 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:53.823307 19953 raft_consensus.cc:697] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:53.824177 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:53.824936 12703 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:20:53.828824 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.813714 12633 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 23. Current term is 24. Ops: []
W20260709 13:20:53.813437 19942 log.cc:927] Time spent T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:20:53.804967 19877 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:53.781944 19825 consensus_queue.cc:237] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:53.894271 19895 consensus_queue.cc:237] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:53.843987 19955 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:53.906095 19953 consensus_queue.cc:237] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:53.807264 19892 consensus_queue.cc:1175] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "522fd2ba0e9f402691b5ef29af8579cd" responder_term: 20 status { last_received { term: 17 index: 14 } last_committed_idx: 14 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 19. Current term is 20. Ops: [19.15-19.15]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:20:53.839747 19907 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:53.805099 19805 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:53.911677 19809 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:53.917822 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 77929.8us
W20260709 13:20:53.954423 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 1.26853e+06us
I20260709 13:20:53.955438 12672 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:53.917994 12705 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 25.
I20260709 13:20:53.919023 12632 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:53.932672 19917 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:53.917071 19955 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:53.933777 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:53.951743 19807 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:53.933763 13348 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:53.926872 19833 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:53.928210 19907 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:20:53.989138 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 55505.7us
I20260709 13:20:53.925752 19877 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:53.969270 19848 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.979022 12598 leader_election.cc:400] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17.
W20260709 13:20:54.058683 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 93131.7us
I20260709 13:20:54.058862 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:20:53.985620 19940 log.cc:927] Time spent T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.379s	user 0.002s	sys 0.000s
W20260709 13:20:53.973829 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 256773us
I20260709 13:20:53.994069 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:53.994721 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:53.983321 12631 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:53.994385 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:54.040222 12780 raft_consensus.cc:1275] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 13:20:54.056460 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:53.964556 12703 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
I20260709 13:20:54.070641 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:54.076644 12776 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:54.077014 13377 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:20:54.074105 13380 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:20:54.076063 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:54.074445 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:53.995945 19907 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:54.079144 12709 raft_consensus.cc:2368] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17.
I20260709 13:20:53.976642 19892 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:53.967262 19805 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:53.991108 19893 raft_consensus.cc:2804] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:54.091624 19892 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:54.092096 19893 raft_consensus.cc:697] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:53.974802 19781 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:54.094168 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:54.094805 12707 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:54.076614 12781 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:20:54.095263 13380 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:20:53.979821 19844 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:53.969393 19909 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:20:54.082453 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 123656us
I20260709 13:20:54.110011 19909 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:54.112754 19909 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:54.114995 19892 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:54.050674 19925 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17.
I20260709 13:20:54.093284 19893 consensus_queue.cc:237] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:53.965687 19955 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:53.980013 19807 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:54.104380 12597 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:54.126793 19952 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:54.127384 19952 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:54.127780 19807 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:54.128002 19952 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:54.129978 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:54.130635 12782 raft_consensus.cc:3055] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:20:54.130759 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:54.130977 12782 raft_consensus.cc:740] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:54.131791 12782 consensus_queue.cc:260] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:54.133054 12782 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:54.133616 19805 raft_consensus.cc:2955] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Committing config change with OpId 19.14: config changed from index 12 to 14, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:54.136322 19877 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:54.137542 19807 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.138123 19807 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:54.140843 19807 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:53.971493 19872 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:53.973773 19860 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:53.971305 19950 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:54.162385 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:54.163141 12627 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:20:54.085500 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:53.964704 19917 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:54.086747 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:54.087812 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:54.088454 12672 leader_election.cc:400] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 21. Current term is 22.
I20260709 13:20:54.089138 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:54.170240 12704 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:53.978915 19809 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:54.070302 19848 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:54.089937 19957 consensus_queue.cc:1048] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:54.090989 12709 raft_consensus.cc:1275] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 13:20:53.976780 19898 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.089227 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:54.097661 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:54.097900 12775 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:54.101081 12776 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 21.
I20260709 13:20:53.974690 19801 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:54.103180 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:53.994033 19922 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:54.093854 19781 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:54.105087 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:54.106588 19960 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:54.091331 19959 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:53.967949 19833 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:54.080669 19818 raft_consensus.cc:2764] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:20:54.112967 12747 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:54.153316 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:54.006587 19829 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:54.159180 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:54.084193 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:54.163658 12707 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 20.
W20260709 13:20:54.170689 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 82395.4us
I20260709 13:20:54.172744 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:20:54.360903 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 257499us
I20260709 13:20:54.375802 12746 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:54.177977 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:54.377389 13380 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:54.377733 13382 raft_consensus.cc:2393] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 21: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:54.249993 13376 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 20.
I20260709 13:20:54.179343 12629 raft_consensus.cc:2410] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:20:54.189000 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:20:54.299270 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:54.190142 13381 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:20:54.277070 12782 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 24.
I20260709 13:20:54.382058 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:54.381795 12780 raft_consensus.cc:2393] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 20: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:20:54.381740 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:54.383663 13377 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:20:54.385030 12626 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:20:54.207355 12627 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 26.
I20260709 13:20:54.282589 12634 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:54.213239 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:54.388130 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:54.388653 13385 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
W20260709 13:20:54.364898 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:54.162412 12625 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
I20260709 13:20:54.194741 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:54.379422 13384 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 25.
I20260709 13:20:54.392072 12673 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:54.386078 12672 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:54.394345 12631 raft_consensus.cc:2393] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 17: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:54.392902 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:54.385082 13378 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:20:54.401389 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:54.351994 19963 log.cc:927] Time spent T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.254s	user 0.000s	sys 0.000s
W20260709 13:20:54.342612 19900 consensus_meta.cc:287] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.607s	user 0.012s	sys 0.000s
I20260709 13:20:54.403955 12701 raft_consensus.cc:2368] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17.
I20260709 13:20:54.377470 13349 leader_election.cc:400] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 20. Current term is 21.
I20260709 13:20:54.396785 19826 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:54.415225 19848 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:54.396759 19807 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:54.392338 19960 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:54.253674 19875 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:54.397610 19922 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:54.169322 19950 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:54.173772 19920 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.420092 13349 leader_election.cc:403] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:54.422857 13380 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:20:54.375228 19910 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:20:54.431293 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 45324.4us
I20260709 13:20:54.462634 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:54.437794 12631 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:20:54.438521 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:54.439620 12779 raft_consensus.cc:1240] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:20:54.433830 12597 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:54.440646 13379 raft_consensus.cc:2368] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 24. Current term is 25.
W20260709 13:20:54.459352 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 82007.5us
I20260709 13:20:54.468619 13349 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:54.432368 19826 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:54.481969 13378 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 21.
I20260709 13:20:54.431344 19970 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:20:54.440349 19778 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:54.443908 19806 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:54.397243 19801 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:54.169490 19872 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:54.495780 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 382909us
I20260709 13:20:54.497908 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:54.500499 13378 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
I20260709 13:20:54.495733 19842 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 20. Current term is 21.
I20260709 13:20:54.501924 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:54.502333 19781 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:54.502504 19960 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:54.482553 19909 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:54.504593 19909 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:54.505870 19960 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:54.506466 19960 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:54.506894 19909 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:54.452870 19844 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:54.507578 12704 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 24.
I20260709 13:20:54.508951 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:54.469168 19914 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:54.417685 19885 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.062s
I20260709 13:20:54.393169 19898 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:54.446568 19915 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.062s
W20260709 13:20:54.518167 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 126176us
I20260709 13:20:54.497184 19826 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:54.443876 19807 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:54.469244 19875 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:54.531701 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:54.520817 19844 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:54.547151 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:54.548652 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:54.536602 19960 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:54.508356 19925 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:20:54.537848 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 69344.1us
I20260709 13:20:54.550427 12633 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:54.468948 19920 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:54.550652 12627 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:54.551234 13349 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:54.545773 12704 raft_consensus.cc:2368] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 16. Current term is 17.
I20260709 13:20:54.480445 19930 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.046s
I20260709 13:20:54.522811 19970 raft_consensus.cc:697] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 26 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:54.580714 19977 raft_consensus.cc:2764] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 13:20:54.556389 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:54.549310 19844 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:54.582278 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 148507us
I20260709 13:20:54.588197 12634 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
I20260709 13:20:54.509976 19910 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:54.589044 12597 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:54.596393 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:54.558504 19925 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:54.526479 19914 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:54.417686 19959 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:54.525700 19955 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:54.602013 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:54.433171 19900 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:20:54.591220 19917 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:54.604897 19973 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:54.605623 19872 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:54.609917 19969 consensus_queue.cc:1059] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 13:20:54.613971 19971 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:20:54.594193 19977 raft_consensus.cc:2764] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:20:54.614562 19922 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:54.614745 19801 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:54.615130 19833 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:54.615955 19898 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:54.600740 19925 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:54.601944 12597 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:54.617059 19920 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:54.655040 13378 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
W20260709 13:20:54.653589 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 102422us
I20260709 13:20:54.691562 12626 raft_consensus.cc:1240] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 21. Current term is 22. Ops: []
W20260709 13:20:54.665481 19907 consensus_meta.cc:287] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.518s	user 0.011s	sys 0.000s
W20260709 13:20:54.699448 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 97633.3us
W20260709 13:20:54.639331 19953 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.011s	sys 0.010s
I20260709 13:20:54.659997 19969 raft_consensus.cc:3055] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:54.715890 19969 raft_consensus.cc:740] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:54.660456 19970 consensus_queue.cc:237] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:54.662281 19950 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:54.664345 19959 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:54.722304 13381 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:54.649855 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:54.666061 19914 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:54.667941 19829 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:54.729854 19914 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:54.730439 19914 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:54.730993 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:54.732223 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:54.732641 19914 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:54.733846 19833 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:54.734416 12702 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:20:54.734891 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:54.668783 12531 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40528 (request call id 229) took 2914 ms. Trace:
I20260709 13:20:54.671167 19910 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:54.735787 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:20:54.673264 19952 consensus_meta.cc:287] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.545s	user 0.009s	sys 0.003s
I20260709 13:20:54.736454 19952 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:54.737498 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:54.738207 12625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56838 (request call id 2863) took 1126 ms. Trace:
I20260709 13:20:54.738898 12632 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
W20260709 13:20:54.676836 19944 log.cc:927] Time spent T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.960s	user 0.000s	sys 0.007s
I20260709 13:20:54.677488 12628 raft_consensus.cc:1240] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:20:54.681908 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:54.682991 12629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56838 (request call id 2864) took 1070 ms. Trace:
I20260709 13:20:54.741299 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:54.683344 19900 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:54.600724 19980 raft_consensus.cc:2764] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 13:20:54.616226 19875 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:54.753701 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:54.754433 12781 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:20:54.760109 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:54.767180 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:54.755317 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:54.791581 12778 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:20:54.791986 13380 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:20:54.764528 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:54.793637 12777 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:20:54.775859 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:54.754714 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:54.810420 13380 raft_consensus.cc:1240] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
W20260709 13:20:54.713039 19953 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.014s	sys 0.011s
W20260709 13:20:54.822672 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:54.672470 12774 consensus_queue.cc:260] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:54.676860 19778 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:54.698458 19907 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:54.705505 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:54.706697 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:54.707144 19807 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:54.712241 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:54.833592 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:54.701786 19898 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:20:54.720573 19969 consensus_queue.cc:260] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:54.835974 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:20:54.715211 19976 log.cc:927] Time spent T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.113s	user 0.000s	sys 0.002s
I20260709 13:20:54.744524 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:54.738821 12625 rpcz_store.cc:276] 0709 13:20:53.612121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:53.613350 (+  1229us) service_pool.cc:224] Handling call
0709 13:20:54.738039 (+1124689us) spinlock_profiling.cc:243] Waited 511 ms on lock 0x7b540007a3d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:20:54.738151 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1176291200}
I20260709 13:20:54.827474 12705 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:20:54.832299 19907 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:20:54.833779 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 121615us
I20260709 13:20:54.740818 12629 rpcz_store.cc:276] 0709 13:20:53.612381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:53.614280 (+  1899us) service_pool.cc:224] Handling call
0709 13:20:54.682697 (+1068417us) spinlock_profiling.cc:243] Waited 526 ms on lock 0x7b54002cdb54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:20:54.682870 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1209242368}
I20260709 13:20:54.838577 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:54.840037 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:54.841545 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:54.843855 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:54.734355 19833 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:54.862509 12702 raft_consensus.cc:3055] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:20:54.864575 12702 raft_consensus.cc:740] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:54.875854 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:54.879658 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:54.873670 12702 consensus_queue.cc:260] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:54.847669 19778 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:54.887909 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 379007us
I20260709 13:20:54.874828 19805 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.004026 19805 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:55.006160 19805 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:54.864907 19953 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:55.007099 19953 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:55.009163 19953 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:54.870883 19898 raft_consensus.cc:697] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 24 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:55.010322 19898 consensus_queue.cc:237] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:54.887521 12597 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:54.808097 19826 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:55.013963 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 126589us
I20260709 13:20:55.014576 19833 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:55.014652 12597 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:55.015332 19826 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:54.912914 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:55.015705 19858 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:55.015826 12597 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:54.914250 19981 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:55.016220 19858 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:55.016574 19981 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:55.016654 19858 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:55.017041 19981 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:55.017876 19980 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:55.018323 19980 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:55.018702 19980 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:55.020258 19892 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:55.020695 19892 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:55.021047 19892 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:55.021842 12629 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
W20260709 13:20:55.023182 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:55.023245 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:55.024863 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:55.025270 19858 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:55.026376 12774 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:55.027514 19858 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:55.028513 19892 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:54.920709 19955 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:54.863437 19977 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:20:55.040995 19875 consensus_queue.cc:1059] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:20:54.895632 19952 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:55.046133 19848 consensus_queue.cc:1059] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:55.043522 12673 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:54.929244 12598 leader_election.cc:400] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17.
I20260709 13:20:54.886958 19914 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:55.047192 19848 raft_consensus.cc:3055] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:55.047384 19804 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:54.929121 19900 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:55.047637 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 121628us
I20260709 13:20:54.908145 19807 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:55.047592 19914 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:54.938545 12523 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54814 (request call id 311) took 3045 ms. Trace:
I20260709 13:20:55.048090 19900 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:55.048499 19807 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:55.048712 12523 rpcz_store.cc:276] 0709 13:20:51.893138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:51.915232 (+ 22094us) service_pool.cc:224] Handling call
0709 13:20:54.938379 (+3023147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":237637,"reported_tablets":4}
I20260709 13:20:55.050077 19914 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:54.738092 12531 rpcz_store.cc:276] 0709 13:20:51.754491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:51.757439 (+  2948us) service_pool.cc:224] Handling call
0709 13:20:54.668672 (+2911233us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:52.750528 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:52.838400 (+ 87872us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:52.838442 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:20:52.838460 (+    18us) tablet.cc:662] Decoding operations
0709 13:20:52.881488 (+ 43028us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:52.886525 (+  5037us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:52.888552 (+  2027us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:20:52.934447 (+ 45895us) tablet.cc:701] Row locks acquired
0709 13:20:52.934473 (+    26us) write_op.cc:260] PREPARE: finished
0709 13:20:52.960863 (+ 26390us) write_op.cc:270] Start()
0709 13:20:52.961030 (+   167us) write_op.cc:276] Timestamp: P: 1783603252960801 usec, L: 0
0709 13:20:52.961056 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:20:53.077497 (+116441us) log.cc:844] Serialized 1037 byte log entry
0709 13:20:53.362753 (+285256us) spinlock_profiling.cc:243] Waited 7.05 ms on lock 0x7b540011eed4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3ade28 00007f4c70de8581 00007f4c70de7d58 00007f4c70dec9f1 00007f4c70dec9a9 00007f4c70dec939 00007f4c70dec901 00007f4c70debb2d 000000000038f354 000000000038f199
0709 13:20:53.798341 (+435588us) op_driver.cc:464] REPLICATION: finished
0709 13:20:54.086221 (+287880us) write_op.cc:301] APPLY: starting
0709 13:20:54.086307 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:54.088349 (+  2042us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:54.088373 (+    24us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:54.117294 (+ 28921us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:54.117516 (+   222us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:20:54.117615 (+    99us) write_op.cc:312] APPLY: finished
0709 13:20:54.117834 (+   219us) log.cc:844] Serialized 33 byte log entry
0709 13:20:54.118007 (+   173us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:54.118099 (+    92us) write_op.cc:454] Released schema lock
0709 13:20:54.118141 (+    42us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:54.118587 (+   446us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:54.118623 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":127847,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":287538,"apply.run_cpu_time_us":5175,"apply.run_wall_time_us":32331,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12495,"num_ops":2,"prepare.queue_time_us":378840,"prepare.run_cpu_time_us":68417,"prepare.run_wall_time_us":616193,"raft.queue_time_us":295541,"raft.run_cpu_time_us":125492,"raft.run_wall_time_us":308752,"replication_time_us":821727,"spinlock_wait_cycles":16224000,"thread_start_us":874525,"threads_started":4,"wal-append.queue_time_us":352928}]]}
I20260709 13:20:55.048141 12673 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:55.050419 19900 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:55.050618 19807 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:54.908574 19778 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:55.052501 19778 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:55.052551 19953 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:54.973044 12631 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:20:54.983012 19925 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:54.987237 19860 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:54.962919 19910 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.053175 19953 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:55.047627 19848 raft_consensus.cc:740] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:55.053838 19910 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:55.054734 19969 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:55.055054 12633 raft_consensus.cc:1240] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 16. Current term is 17. Ops: []
W20260709 13:20:54.915342 19988 log.cc:927] Time spent T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:20:55.030494 12746 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:55.066154 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:55.045262 12597 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:55.030177 12528 catalog_manager.cc:5697] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:55.064834 12774 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:20:55.032596 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:55.040254 19818 consensus_queue.cc:1059] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:55.040949 19902 consensus_queue.cc:1059] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:20:55.081641 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:55.082523 12775 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:20:55.046439 19875 raft_consensus.cc:3055] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:20:54.909493 19941 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:54.948202 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:55.085135 12778 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 22.
I20260709 13:20:55.053752 19860 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:55.002751 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:55.085794 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:55.086488 12774 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36488 (request call id 3893) took 1250 ms. Trace:
I20260709 13:20:54.998389 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:55.060446 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:55.069414 12528 catalog_manager.cc:5697] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:55.068084 12746 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:55.084897 12597 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:55.091013 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:55.105685 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:55.107823 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:55.111692 12705 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:20:55.048420 19804 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:55.040899 19981 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:55.132892 12779 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:55.134927 12708 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:20:55.144623 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 59792.7us
I20260709 13:20:55.054786 19848 consensus_queue.cc:260] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:55.147786 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:55.086937 12774 rpcz_store.cc:276] 0709 13:20:53.818423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:53.818638 (+   215us) service_pool.cc:224] Handling call
0709 13:20:54.142979 (+324341us) spinlock_profiling.cc:243] Waited 19.2 ms on lock 0x7b4c001f6cc0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6b67ea20 00007f4c6b68fa29 00007f4c6b655644 00007f4c6b67cf2d 00007f4c6b67cb93 00007f4c6d3ad18b 00007f4c6d3adcb5 00007f4c6d3a8f30 00007f4c6d3aa7de 00007f4c6d3b627c 00007f4c72377b17
0709 13:20:54.671958 (+528979us) spinlock_profiling.cc:243] Waited 49.9 ms on lock 0x7b540028b520. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d33006f 00007f4c6d3a8f60 00007f4c6d3aa7de 00007f4c6d3b627c 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2
0709 13:20:55.026196 (+354238us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b58002bb5d0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d30e0aa 00007f4c6d390447 00007f4c6d3a8fe7 00007f4c6d3aa7de 00007f4c6d3b627c 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c
0709 13:20:55.069147 (+ 42951us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":559448576}
I20260709 13:20:55.080869 19880 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:20:55.140779 12598 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.705s (SENT)
I20260709 13:20:55.080538 19910 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:55.122511 12705 raft_consensus.cc:2955] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 12 to 14, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:55.086810 19972 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:55.086614 19875 raft_consensus.cc:740] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:55.172446 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:55.176103 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:55.091346 19960 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:55.151443 19915 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:55.146341 19826 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:55.144694 19804 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:55.144210 19833 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:55.144459 19807 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:20:55.184157 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 305653us
I20260709 13:20:55.192505 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:55.196796 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:55.214258 19881 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:20:55.206362 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:55.206843 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:55.281981 12706 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 22.
I20260709 13:20:55.210053 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:55.283185 12672 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:55.284070 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:55.285126 12776 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 20: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
W20260709 13:20:55.262862 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 48128.6us
I20260709 13:20:55.286958 12776 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:20:55.287091 19818 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:55.287575 12776 raft_consensus.cc:606] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:20:55.157585 19952 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:55.166596 19972 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:55.157302 19914 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.094s
I20260709 13:20:55.131408 19941 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:55.287329 19912 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 13:20:55.158972 19880 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:55.316341 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:55.194584 19826 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:55.191002 19915 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:55.277575 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:55.188108 19872 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:55.277868 12774 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:20:55.293936 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:20:55.302505 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:55.304391 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:55.194228 19960 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:55.280573 19809 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:55.312179 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:55.262610 19833 raft_consensus.cc:697] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:55.274420 19900 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.318720 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:55.316390 13385 raft_consensus.cc:2955] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 12 to 14, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:55.354588 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:55.315352 19854 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.348161 19880 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:55.360117 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:55.367873 12774 raft_consensus.cc:2393] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 22: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:55.311868 19972 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:55.372148 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:55.374099 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:55.353173 19915 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:55.375038 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:55.375975 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:55.356492 19826 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:55.312192 19807 raft_consensus.cc:697] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:55.310971 19952 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:55.308686 19881 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:55.320113 19847 consensus_queue.cc:1059] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 13:20:55.391366 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:55.401530 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:55.403709 13348 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:55.308745 19875 consensus_queue.cc:260] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:55.406082 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:55.410928 12778 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:55.374922 19941 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:55.409399 12597 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:55.363502 19809 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:55.368291 19872 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:55.423385 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:55.370918 19854 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:55.514074 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 229028us
I20260709 13:20:55.515522 12672 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:20:55.517163 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:55.518360 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:55.395383 19892 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:55.463632 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:55.542189 19886 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:55.370711 19900 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:55.398388 19973 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
W20260709 13:20:55.412045 19893 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.274s	user 0.034s	sys 0.060s
I20260709 13:20:55.458102 19981 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:20:55.460777 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 56584.1us
I20260709 13:20:55.461050 19953 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:55.426961 19975 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.046s
W20260709 13:20:55.504968 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 364219us
I20260709 13:20:55.397794 19833 consensus_queue.cc:237] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:55.426584 19977 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:55.468166 19875 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:20:55.504479 19985 log.cc:927] Time spent T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.242s	user 0.001s	sys 0.000s
I20260709 13:20:55.450390 19941 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:20:55.543134 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 119845us
I20260709 13:20:55.534935 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:55.542505 19914 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:55.579962 12597 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:55.584826 19854 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:55.584997 19969 consensus_meta.cc:287] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.530s	user 0.011s	sys 0.000s
W20260709 13:20:55.580760 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 45979.3us
I20260709 13:20:55.585968 12709 raft_consensus.cc:2410] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 15 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:20:55.564857 19966 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:55.554667 19886 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:55.590723 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:55.591578 19778 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:55.592136 19778 raft_consensus.cc:697] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 21 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:55.593880 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:55.594844 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:55.595227 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:55.595768 19914 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:55.596148 19872 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:55.596762 19886 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:55.552798 19893 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.423s	user 0.035s	sys 0.069s
I20260709 13:20:55.538121 19994 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.594022 19941 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:55.596367 19914 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:55.597952 19994 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:55.598955 19900 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:55.599839 19914 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:55.600215 19809 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:55.600737 19881 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:55.536974 19807 consensus_queue.cc:237] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:55.568560 19980 consensus_meta.cc:287] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.549s	user 0.003s	sys 0.007s
I20260709 13:20:55.606297 19980 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:55.607465 12630 raft_consensus.cc:1240] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:55.609151 19980 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:55.609604 19925 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:55.609957 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:55.581493 19971 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:20:55.611358 19998 log.cc:927] Time spent T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:20:55.616269 19952 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:55.584522 19826 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:55.563764 19960 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:55.614215 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:55.573330 12598 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:55.618798 19826 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:55.619140 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 45883.9us
I20260709 13:20:55.619591 19952 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:55.620436 12598 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:55.621498 12779 raft_consensus.cc:2393] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 16: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:55.624822 12778 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:20:55.627166 19854 consensus_queue.cc:1048] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:55.630486 12598 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:55.632124 19925 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:20:55.636688 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 231891us
I20260709 13:20:55.637684 13348 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:55.642690 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:55.646656 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:55.622574 19914 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:55.651192 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:55.636059 12597 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:20:55.653385 19972 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:55.653896 19980 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:55.654780 19980 raft_consensus.cc:697] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:55.655859 19980 consensus_queue.cc:237] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:55.659978 19900 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:55.660676 19900 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:55.662868 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:55.662992 19900 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:55.665923 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:55.668727 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:55.593019 19778 consensus_queue.cc:237] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:55.676057 19804 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:55.678378 19994 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:55.679364 19860 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:55.683549 19880 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:55.686247 19989 log.cc:927] Time spent T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.325s	user 0.000s	sys 0.003s
I20260709 13:20:55.687381 19907 raft_consensus.cc:993] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2: : Instructing follower 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:55.687909 19907 raft_consensus.cc:1081] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Signalling peer 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:55.701542 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:55.650965 19975 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:55.707901 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:55.709877 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:55.710726 12631 raft_consensus.cc:2393] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 15: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:20:55.711676 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:55.712615 12625 raft_consensus.cc:2393] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 22: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:20:55.716356 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:55.717236 12626 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 22.
I20260709 13:20:55.719470 12629 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 13:20:55.719995 12629 pending_rounds.cc:85] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 14
I20260709 13:20:55.739966 12626 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:20:55.746800 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 160216us
I20260709 13:20:55.720350 12629 pending_rounds.cc:107] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:20:55.753172 12629 raft_consensus.cc:2889] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:55.690635 19914 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:55.706423 19880 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:55.784318 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 59248.5us
I20260709 13:20:55.790376 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:55.799588 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:55.833738 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:55.785939 12632 raft_consensus.cc:2955] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 12 to 14, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:55.786886 12747 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:55.836848 12626 raft_consensus.cc:1240] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:55.837980 12631 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
W20260709 13:20:55.846138 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 59448.8us
I20260709 13:20:55.841388 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:55.847653 13382 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 20.
I20260709 13:20:55.843077 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:55.838245 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:55.846664 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:55.859645 12631 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:55.863538 12705 raft_consensus.cc:1240] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:55.880438 13380 raft_consensus.cc:2393] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 17: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:20:55.706676 19915 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:55.899055 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:55.899288 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:55.900084 13384 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:55.907928 12631 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
W20260709 13:20:55.909585 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 119277us
I20260709 13:20:55.910557 12633 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:55.911602 19950 raft_consensus.cc:3055] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:55.911860 12634 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce"
dest_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:56838
I20260709 13:20:55.998416 12634 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:55.998873 12634 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:55.912439 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:55.912613 19902 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:55.914783 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:56.007884 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:56.008674 13376 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:55.881364 19986 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:55.753497 20000 consensus_queue.cc:1059] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.175s
I20260709 13:20:55.918193 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:55.911638 19949 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.008s
I20260709 13:20:55.919673 20012 consensus_queue.cc:1059] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:55.920825 19915 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:55.929971 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:56.027190 13383 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:56.027661 20000 raft_consensus.cc:3055] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:56.028072 20000 raft_consensus.cc:740] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:56.029250 20012 raft_consensus.cc:3055] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:56.029654 19941 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:56.029269 12709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38496 (request call id 1428) took 1298 ms. Trace:
I20260709 13:20:56.030241 12709 rpcz_store.cc:276] 0709 13:20:54.730636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:54.730762 (+   126us) service_pool.cc:224] Handling call
0709 13:20:56.029037 (+1298275us) spinlock_profiling.cc:243] Waited 854 ms on lock 0x7b54001ccfd4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:20:56.029174 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1964038528}
I20260709 13:20:56.030887 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:56.031520 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:55.900718 19971 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.032048 12706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:20:56.032338 19971 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:56.033120 20008 log.cc:927] Time spent T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 13:20:56.033630 12702 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:56.012813 19986 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:56.034366 19986 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:56.034536 19971 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:55.958065 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:20:55.949389 20015 log.cc:927] Time spent T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 13:20:56.035550 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 77566us
I20260709 13:20:55.888844 19914 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.036396 12672 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:20:56.022424 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 104361us
I20260709 13:20:55.949990 19804 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:56.029670 20012 raft_consensus.cc:740] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:20:56.006299 20007 log.cc:927] Time spent T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:20:56.038216 12597 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:56.046464 12627 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:56.030129 12701 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 13:20:56.040351 12747 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:56.047109 12701 pending_rounds.cc:85] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 12
I20260709 13:20:56.043548 12702 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 15. (term mismatch)
I20260709 13:20:56.047358 12701 pending_rounds.cc:107] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 13:20:56.047566 12702 pending_rounds.cc:85] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 14
I20260709 13:20:56.047631 12701 raft_consensus.cc:2889] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:56.047876 12702 pending_rounds.cc:107] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 20.15
I20260709 13:20:56.048148 12702 raft_consensus.cc:2889] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:56.028945 20000 consensus_queue.cc:260] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.043852 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:56.044057 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:20:56.044924 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:56.056006 13380 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
I20260709 13:20:56.044842 12708 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:20:56.006445 19902 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:55.997793 19950 raft_consensus.cc:740] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:56.064857 12597 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:56.045650 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:55.913343 19925 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:56.058477 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 1434) took 1222 ms. Trace:
I20260709 13:20:56.056345 13383 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:20:56.068461 13379 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:20:56.063908 12634 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:56.064857 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:56.070971 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:56.074148 12708 raft_consensus.cc:2393] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 20: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:56.067903 12702 rpcz_store.cc:276] 0709 13:20:54.826334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:54.826598 (+   264us) service_pool.cc:224] Handling call
0709 13:20:54.862383 (+ 35785us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:55.019505 (+157122us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b1800157b00. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3ed646 00007f4c6d32758e 00007f4c6d3a8f9e 00007f4c6d3aa7de 00007f4c6d3b4b49 00007f4c6d3b5416 00007f4c6d3b1d8f 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4
0709 13:20:56.027773 (+1008268us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b58008240d0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d30e0aa 00007f4c6d390447 00007f4c6d3a8fe7 00007f4c6d3aa7de 00007f4c6d3b4b49 00007f4c6d3b5416 00007f4c6d3b1d8f 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4
0709 13:20:56.048558 (+ 20785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:56.048741 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1648605824}
I20260709 13:20:56.075316 12673 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:56.076196 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:56.066730 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:56.040716 20009 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:56.072813 12634 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:56.079531 13376 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:20:56.080859 13384 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:20:56.046746 20003 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:56.074708 19892 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:56.050966 19854 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:56.066140 19902 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:56.093748 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:56.098364 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:56.100260 13384 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:56.103085 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:56.058293 19880 consensus_queue.cc:1059] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 13:20:56.108260 12702 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:20:56.077042 19941 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:56.083534 20010 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:56.081410 20009 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:56.083210 19804 consensus_queue.cc:1059] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:20:56.111328 19892 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:56.126433 19941 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.127152 19892 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.128672 19941 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:56.129171 19892 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:56.090276 20000 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:56.114125 12597 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:56.115705 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:56.132257 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:56.133405 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:56.134085 19880 raft_consensus.cc:3055] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:56.134483 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:56.136166 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:56.137168 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:56.137745 12702 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:20:56.138504 19892 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:56.077504 19925 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.092202 19807 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:56.123974 20009 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:56.103216 19854 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:56.126255 20020 log.cc:927] Time spent T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:20:56.182807 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:56.206317 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:56.105930 20012 consensus_queue.cc:260] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:56.208158 20012 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:56.138526 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:56.145117 12705 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:20:56.117707 13384 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 21.
I20260709 13:20:56.212157 12702 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:20:56.213143 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:20:56.214200 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:56.161639 19880 raft_consensus.cc:740] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:56.216117 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:20:56.215013 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 71089us
I20260709 13:20:56.154172 20010 raft_consensus.cc:697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 18 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:56.163576 19807 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:56.141297 19986 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.227276 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:56.227988 12707 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 21.
I20260709 13:20:56.228847 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:56.235733 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:56.219441 19914 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:56.241044 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:56.242931 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:56.244057 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:56.245472 12626 raft_consensus.cc:2393] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 17: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:20:56.248142 12628 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:56.250290 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:56.222563 19807 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:56.255678 19925 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:56.257817 19854 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:56.258455 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:56.262123 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:56.267346 12707 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:20:56.270210 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:20:56.278402 19875 consensus_meta.cc:287] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.699s	user 0.010s	sys 0.000s
I20260709 13:20:56.279179 19818 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:56.264374 19880 consensus_queue.cc:260] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.280126 19902 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.282713 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:56.283439 12709 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 20.
I20260709 13:20:56.284353 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:56.285643 19880 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:56.286566 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:56.287230 12709 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 12.
I20260709 13:20:56.288125 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:56.288686 12709 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:56.293748 20010 consensus_queue.cc:237] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.300173 12672 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:56.302318 19804 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:56.304548 12709 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 22.
I20260709 13:20:56.305465 12633 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:56.307158 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:56.308279 12707 raft_consensus.cc:2368] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17.
W20260709 13:20:56.310123 19990 log.cc:927] Time spent T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.952s	user 0.003s	sys 0.012s
I20260709 13:20:56.314045 13349 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:56.311512 12672 leader_election.cc:400] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 pre-election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 24. Current term is 25.
I20260709 13:20:56.323292 12672 leader_election.cc:403] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:56.323897 13349 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:56.325162 12707 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:20:56.326627 19994 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:56.327325 13377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 1771) took 1572 ms. Trace:
I20260709 13:20:56.331279 12778 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:56.335275 12776 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:56.333719 13377 rpcz_store.cc:276] 0709 13:20:54.755162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:54.755452 (+   290us) service_pool.cc:224] Handling call
0709 13:20:54.755746 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:54.755951 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:54.755975 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:54.770228 (+ 14253us) log.cc:844] Serialized 26 byte log entry
0709 13:20:55.151790 (+381562us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:55.151830 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:55.151912 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:56.303821 (+1151909us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:20:56.327025 (+ 23204us) raft_consensus.cc:1685] finished
0709 13:20:56.327047 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:56.327245 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24362368,"thread_start_us":381122,"threads_started":1,"wal-append.queue_time_us":504797}
I20260709 13:20:56.361608 12781 raft_consensus.cc:1240] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 21. Current term is 22. Ops: []
I20260709 13:20:56.362229 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:56.370978 12598 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:56.331751 19981 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:56.332481 20000 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 24. Current term is 25.
I20260709 13:20:56.374728 12598 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:56.377625 12598 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:56.324807 20006 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.048s
I20260709 13:20:56.384586 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:56.388115 12778 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:56.383697 12598 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:56.386220 19854 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:56.386924 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:56.386947 19902 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:56.330731 19971 consensus_queue.cc:1059] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:20:56.392073 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:20:56.392954 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:56.396083 12598 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:56.399535 12781 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:20:56.382300 19858 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:56.400702 12598 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:56.383332 20030 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:56.371336 19807 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.404213 19854 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:56.401863 12707 raft_consensus.cc:3060] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:56.409828 12598 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:56.418433 12598 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:56.403084 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:56.413389 12781 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:20:56.384534 19898 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 13:20:56.397764 19892 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:20:56.435030 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:56.441371 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:56.443975 12626 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:56.421123 12598 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
W20260709 13:20:56.424065 20037 log.cc:927] Time spent T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:20:56.398020 20009 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.429908 19953 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:56.420260 19959 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:56.432739 19807 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:56.411959 19925 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:56.446201 12629 raft_consensus.cc:2393] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 21: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:20:56.446266 12598 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:56.454448 12672 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:20:56.455267 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 120961us
I20260709 13:20:56.471481 12598 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:56.467957 19925 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:56.419939 19872 consensus_queue.cc:1059] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:20:56.472343 19925 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:56.461196 19858 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:56.473282 20010 consensus_queue.cc:1059] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:20:56.470954 13349 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:56.477540 12598 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:56.477597 19953 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:20:56.479920 12598 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:56.480712 19953 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:56.481377 19953 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:56.481796 19953 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:56.482121 19872 raft_consensus.cc:3055] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:56.482393 19994 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:56.482736 20010 raft_consensus.cc:3055] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:20:56.483182 20010 raft_consensus.cc:740] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:56.483987 20009 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:56.484280 20010 consensus_queue.cc:260] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.477380 19970 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:56.482160 19953 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:56.488415 19884 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:56.488662 19986 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:56.490121 20010 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:56.462288 19959 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:56.496374 13378 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 15 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:56.493286 12700 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 22.
I20260709 13:20:56.491816 19858 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:56.495386 13377 raft_consensus.cc:1240] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:56.491523 12598 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:56.499377 12598 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:56.508056 12701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38496 (request call id 1435) took 1630 ms. Trace:
I20260709 13:20:56.508668 12701 rpcz_store.cc:276] 0709 13:20:54.877813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:54.878641 (+   828us) service_pool.cc:224] Handling call
0709 13:20:56.029803 (+1151162us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:56.048177 (+ 18374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:56.507777 (+459600us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54001ccfd0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:20:56.507971 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2610010368}
I20260709 13:20:56.500504 19854 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:56.511142 19854 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:56.511615 19854 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:56.461366 19880 consensus_queue.cc:1059] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:20:56.513722 19804 consensus_queue.cc:1059] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 17.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:20:56.501902 12672 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:56.516311 12598 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:56.514290 19880 raft_consensus.cc:3055] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:56.518123 19971 raft_consensus.cc:3055] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:56.518113 19898 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:56.518091 12598 leader_election.cc:400] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17.
I20260709 13:20:56.518558 19902 consensus_queue.cc:1059] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:56.519255 19804 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:20:56.528439 13377 raft_consensus.cc:1240] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:56.520056 19950 consensus_queue.cc:260] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:56.529826 19950 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:56.530836 19858 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:20:56.498147 19959 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:56.534673 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:56.536217 20006 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:56.536939 13377 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:56.536787 20006 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.482636 19872 raft_consensus.cc:740] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:56.538719 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:56.527118 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:56.525261 19898 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:56.539127 20006 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:56.539450 19898 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:56.539737 20006 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:56.523492 19971 raft_consensus.cc:740] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:56.527891 19902 consensus_queue.cc:1243] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:20:56.527984 19910 raft_consensus.cc:3055] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:20:56.520030 13385 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:56.549228 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:56.537026 19844 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:56.538386 19872 consensus_queue.cc:260] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.540714 12630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 4623) took 1035 ms. Trace:
W20260709 13:20:56.541894 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 622831us
I20260709 13:20:56.559746 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:56.521832 19807 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:20:56.562407 19807 raft_consensus.cc:697] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 22 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:56.544073 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:56.522495 19880 raft_consensus.cc:740] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:56.546797 19910 raft_consensus.cc:740] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:56.569397 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:56.546136 20006 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.555617 19958 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:20:56.554692 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 514448us
I20260709 13:20:56.558101 12630 rpcz_store.cc:276] 0709 13:20:55.505804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:55.505945 (+   141us) service_pool.cc:224] Handling call
0709 13:20:55.547883 (+ 41938us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:55.607828 (+ 59945us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:56.540491 (+932663us) spinlock_profiling.cc:243] Waited 59.4 ms on lock 0x7b5400408b54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:20:56.540655 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":136603904}
I20260709 13:20:56.496557 19842 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
W20260709 13:20:56.540797 19970 consensus_queue.cc:1175] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3d539f8d95d343d8b1696803f18b30d2" responder_term: 14 status { last_received { term: 14 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 14 index: 10 } }
I20260709 13:20:56.565997 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:56.552879 19971 consensus_queue.cc:260] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.569258 13376 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:20:56.553685 19848 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:56.573168 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:56.627038 12779 raft_consensus.cc:3055] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:20:56.590965 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:56.566654 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:56.630782 20000 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:56.631374 20000 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.571640 19874 consensus_queue.cc:1059] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 13:20:56.634426 19958 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:20:56.635248 19874 consensus_queue.cc:1059] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:56.571379 19801 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.065s
I20260709 13:20:56.635978 19986 consensus_queue.cc:1059] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:56.636493 19874 raft_consensus.cc:3055] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:56.578199 19974 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:56.636981 19874 raft_consensus.cc:740] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:56.656029 13380 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:20:56.637804 19974 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.657330 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:56.658633 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:56.660634 19974 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:56.640069 19880 consensus_queue.cc:260] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:56.640746 19971 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:56.617074 19804 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.662601 19804 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:56.662853 19880 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:56.563369 19807 consensus_queue.cc:237] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:56.644759 20006 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:56.627799 19960 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.564128 19844 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.651289 13378 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:56.652134 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:56.653096 12706 raft_consensus.cc:606] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:20:56.719216 12746 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:56.654105 13384 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:56.653738 19872 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:56.722736 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:56.667346 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:56.719556 12706 raft_consensus.cc:614] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:56.724577 12708 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:20:56.654862 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:56.627493 12779 raft_consensus.cc:740] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:20:56.730108 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:56.730976 12746 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:56.636986 19986 raft_consensus.cc:3055] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:20:56.766959 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:56.726432 12779 consensus_queue.cc:260] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:56.775951 12779 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:56.771363 19805 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:56.771711 19966 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 13:20:56.770972 19829 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:56.794471 13385 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
I20260709 13:20:56.715819 20012 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:56.656942 19874 consensus_queue.cc:260] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.773931 19986 raft_consensus.cc:740] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:56.801280 19805 raft_consensus.cc:697] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:56.722832 19960 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:56.807999 19829 raft_consensus.cc:697] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:20:56.784914 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:56.864051 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:56.803875 20012 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:56.827818 19874 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:20:56.814049 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 191310us
W20260709 13:20:56.877079 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:56.881263 19959 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:56.925046 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 53670.4us
I20260709 13:20:56.936403 12702 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:20:56.947502 12779 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 22.
I20260709 13:20:56.918829 19959 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:56.937919 13376 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:20:56.953853 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:56.954674 12631 raft_consensus.cc:2393] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 15: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:20:56.954783 12705 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:20:56.890126 19829 consensus_queue.cc:237] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.962339 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:56.963460 12701 raft_consensus.cc:2393] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 23: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:56.967932 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:56.962143 19975 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:56.964107 19842 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:56.969372 19975 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.970000 19842 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.970266 19974 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:56.970880 19974 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.975441 19842 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:56.976593 19974 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:56.958563 20006 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.875753 19986 consensus_queue.cc:260] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:56.930346 19925 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.954294 19917 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:57.004878 19854 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:57.001273 19842 consensus_queue.cc:1059] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
I20260709 13:20:57.008865 19842 raft_consensus.cc:3055] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:57.011188 19854 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:56.985306 19972 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:56.967314 19994 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.012627 20012 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:56.997615 20006 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.955236 19805 consensus_queue.cc:237] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:57.034243 20006 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:57.009334 19842 raft_consensus.cc:740] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:57.039716 19842 consensus_queue.cc:260] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:57.041213 19842 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:57.030659 19994 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:56.979506 19960 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:57.046211 19898 consensus_meta.cc:287] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.506s	user 0.007s	sys 0.009s
I20260709 13:20:57.050594 19925 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:57.051899 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:57.051275 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:57.054111 12597 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:57.073205 20012 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.073944 20012 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:57.074421 19960 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:57.074980 19960 raft_consensus.cc:697] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 21 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:57.076081 20012 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:57.075917 19960 consensus_queue.cc:237] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:57.054093 20000 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:56.974947 19844 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:57.054768 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:57.057108 12704 raft_consensus.cc:2410] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:20:57.080438 19778 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.080972 19778 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:57.029929 19917 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:57.059408 13383 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:20:57.061726 19898 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:57.065687 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:57.067104 20030 consensus_meta.cc:287] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.647s	user 0.010s	sys 0.010s
I20260709 13:20:57.068171 19994 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:20:57.070439 19975 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:57.057912 20006 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:20:57.089012 19953 consensus_meta.cc:287] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.602s	user 0.009s	sys 0.006s
I20260709 13:20:56.990767 13349 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:57.104564 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:57.106739 12597 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:57.111788 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:20:57.093443 12704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 3143) took 1919 ms (client timeout 1607 ms). Trace:
W20260709 13:20:57.130193 12704 rpcz_store.cc:269] 0709 13:20:55.173895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:55.175797 (+  1902us) service_pool.cc:224] Handling call
0709 13:20:57.093179 (+1917382us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b54003ec6d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:20:57.093350 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4323242368}
I20260709 13:20:57.080199 19804 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:57.131152 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 140521us
I20260709 13:20:57.096369 19907 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:20:57.125377 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 542122us
I20260709 13:20:57.132284 13382 raft_consensus.cc:2393] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 15: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:20:57.142516 12706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:55238 (request call id 41) took 1204 ms. Trace:
I20260709 13:20:57.147732 12706 rpcz_store.cc:276] 0709 13:20:55.920624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:56.031926 (+111302us) service_pool.cc:224] Handling call
0709 13:20:57.124345 (+1092419us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b54003eb7d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3ac853 00007f4c7237a60a 00007f4c6b9007d4 00007f4c6b900762 00007f4c6b900691 00007f4c6b90060c 00007f4c6b8ff8c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:20:57.124467 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1426935680}
I20260709 13:20:57.149243 19825 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:57.150027 19825 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:57.142946 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:57.153229 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:20:57.156486 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:57.159165 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:57.161006 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:57.162214 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:57.119529 20030 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:57.165738 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:20:57.166461 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:57.166844 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:57.167117 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:57.167300 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:57.155951 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:57.146512 19953 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:57.170970 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:20:57.176563 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:57.179210 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:57.180444 12780 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:20:57.180799 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:57.182013 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:57.182994 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:57.144148 19941 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:57.183506 12630 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 22 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:20:57.184305 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:57.185846 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:57.154399 19994 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:57.186612 12778 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:20:57.188081 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:57.191682 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:57.193251 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:57.202189 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:57.164994 19907 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:57.207902 12776 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:20:57.208593 12776 raft_consensus.cc:606] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Received request to transfer leadership to TS 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:57.210017 20030 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:57.210628 20030 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:57.214007 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:57.214316 20030 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:57.215742 12709 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:20:57.216845 19953 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:57.221827 19907 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:57.223022 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:57.177731 20045 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:57.224190 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:57.132483 20006 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:57.229954 19898 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:57.228539 12776 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58239dd301f64758846dd758e449e4d8"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:20:57.230904 12776 raft_consensus.cc:606] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Received request to transfer leadership to TS 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:57.203418 19825 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:57.235994 12672 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:57.239853 19804 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:57.227481 20052 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:57.252249 12776 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:20:57.253084 12776 raft_consensus.cc:606] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:20:57.256394 12776 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:20:57.257126 12776 raft_consensus.cc:606] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Received request to transfer leadership to TS 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:57.258607 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:20:57.231320 19994 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:57.243564 19805 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:20:57.275230 20006 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:57.276113 12700 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 3146) took 2079 ms (client timeout 1971 ms). Trace:
W20260709 13:20:57.276686 12700 rpcz_store.cc:269] 0709 13:20:55.196393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:55.196544 (+   151us) service_pool.cc:224] Handling call
0709 13:20:57.275866 (+2079322us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540021e3d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:20:57.276020 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2340632960}
I20260709 13:20:57.278556 12708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 3467) took 1193 ms. Trace:
I20260709 13:20:57.278820 19778 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:57.279063 12708 rpcz_store.cc:276] 0709 13:20:56.084939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:56.102845 (+ 17906us) service_pool.cc:224] Handling call
0709 13:20:57.278352 (+1175507us) spinlock_profiling.cc:243] Waited 621 ms on lock 0x7b54003eb7d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:20:57.278482 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1427267328}
W20260709 13:20:57.280638 19872 consensus_meta.cc:287] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.551s	user 0.014s	sys 0.004s
I20260709 13:20:57.281026 19917 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:57.250059 20010 consensus_meta.cc:287] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.753s	user 0.017s	sys 0.000s
I20260709 13:20:57.239813 20045 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:57.282125 19994 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:57.283701 20045 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:57.284093 19994 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:57.290202 12672 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:57.292748 19854 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:57.293644 20010 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:57.294361 19970 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:20:57.305001 20050 log.cc:927] Time spent T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.061s	user 0.006s	sys 0.000s
I20260709 13:20:57.309327 19986 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:57.317548 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:57.319566 12528 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54792 (request call id 344) took 4969 ms. Trace:
I20260709 13:20:57.319931 12528 rpcz_store.cc:276] 0709 13:20:52.350138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:52.369921 (+ 19783us) service_pool.cc:224] Handling call
0709 13:20:57.319466 (+4949545us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:55.554641 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:55.583609 (+ 28968us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:55.583642 (+    33us) write_op.cc:435] Acquired schema lock
0709 13:20:55.583660 (+    18us) tablet.cc:662] Decoding operations
0709 13:20:55.584066 (+   406us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:55.584119 (+    53us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:55.584150 (+    31us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:20:55.584452 (+   302us) tablet.cc:701] Row locks acquired
0709 13:20:55.584466 (+    14us) write_op.cc:260] PREPARE: finished
0709 13:20:55.584608 (+   142us) write_op.cc:270] Start()
0709 13:20:55.584724 (+   116us) write_op.cc:276] Timestamp: P: 1783603255584559 usec, L: 0
0709 13:20:55.584742 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:20:55.585391 (+   649us) log.cc:844] Serialized 913 byte log entry
0709 13:20:56.254871 (+669480us) op_driver.cc:464] REPLICATION: finished
0709 13:20:56.432122 (+177251us) write_op.cc:301] APPLY: starting
0709 13:20:56.432219 (+    97us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:56.434000 (+  1781us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:56.434113 (+   113us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:56.439461 (+  5348us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:56.439741 (+   280us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:20:56.439814 (+    73us) write_op.cc:312] APPLY: finished
0709 13:20:56.440085 (+   271us) log.cc:844] Serialized 33 byte log entry
0709 13:20:56.440320 (+   235us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:56.440441 (+   121us) write_op.cc:454] Released schema lock
0709 13:20:56.440495 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:56.460562 (+ 20067us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:56.461107 (+   545us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":805,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":157548,"apply.run_cpu_time_us":5413,"apply.run_wall_time_us":8890,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12495,"num_ops":2,"prepare.queue_time_us":414669,"prepare.run_cpu_time_us":27682,"prepare.run_wall_time_us":268110,"raft.queue_time_us":191092,"raft.run_cpu_time_us":43080,"raft.run_wall_time_us":129801,"replication_time_us":669991,"thread_start_us":670585,"threads_started":4,"wal-append.queue_time_us":342547}]]}
I20260709 13:20:57.324141 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:57.324990 12632 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
I20260709 13:20:57.292745 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:57.326368 19872 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:57.326005 12525 catalog_manager.cc:5697] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 19 to 20, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 20 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:57.327943 12525 catalog_manager.cc:5697] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 19 to 20, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 20 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:57.329284 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:57.330901 12702 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 13:20:57.331225 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:57.332041 12625 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 12.
W20260709 13:20:57.332847 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 186695us
I20260709 13:20:57.335901 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:57.336712 12625 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:20:57.338084 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:57.341958 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:57.360496 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:57.361857 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:57.365842 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:57.373204 12625 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:20:57.388909 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:57.389799 12704 raft_consensus.cc:1275] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:20:57.392704 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:57.407510 13349 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:20:57.424012 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:57.430594 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 52794.5us
I20260709 13:20:57.432121 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:57.433467 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:57.434108 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:57.436779 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:57.437204 12706 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 21.
I20260709 13:20:57.437394 12707 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
W20260709 13:20:57.430608 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 235878us
I20260709 13:20:57.437861 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:57.445259 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:57.438766 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:57.434830 12625 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:20:57.448137 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:57.445766 12702 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:20:57.448952 12701 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:20:57.448271 12673 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:57.452476 12673 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:57.457219 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:57.458091 12704 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:20:57.462050 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:57.462836 12704 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:57.406847 19842 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:57.458515 19902 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:57.464339 19902 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:57.452266 19973 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:57.475359 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:57.475891 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:57.470844 19848 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.073s
I20260709 13:20:57.459885 19874 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:57.472939 19902 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:57.406728 19910 consensus_queue.cc:260] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:57.465662 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:57.572923 12706 raft_consensus.cc:2376] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15. Re-sending same reply.
I20260709 13:20:57.591454 12704 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
W20260709 13:20:57.619270 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:57.559631 20006 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:57.634660 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:57.471488 19842 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:57.640318 12704 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:57.582463 19874 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:20:57.647539 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 181984us
I20260709 13:20:57.651883 12704 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 21.
I20260709 13:20:57.622828 19925 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:57.665251 19925 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:57.666483 12706 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:20:57.675297 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:57.675949 12705 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:57.668543 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:57.674988 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:57.681080 12704 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:57.666492 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:57.638862 19833 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.641752 19818 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:20:57.669437 20060 log.cc:927] Time spent T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:20:57.673621 20012 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.721391 12704 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:20:57.666621 19805 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:57.675077 19874 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:57.667368 19925 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:57.679004 19975 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:20:57.741782 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:57.698318 19970 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:20:57.756345 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 77500.8us
I20260709 13:20:57.760188 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:57.772993 12705 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:20:57.793712 12747 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:57.806771 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:57.804947 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:57.749001 19805 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:57.817338 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:57.860424 12625 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 10.
I20260709 13:20:57.861501 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:57.707347 19833 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:57.802480 19925 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:57.809806 19825 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 13:20:57.743264 20012 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:57.824352 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:20:57.826440 20057 log.cc:927] Time spent T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.275s	user 0.008s	sys 0.001s
I20260709 13:20:57.808840 19966 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:57.872265 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:57.742518 19854 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:57.812937 12778 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:20:57.813427 19941 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:57.830999 19958 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:57.833135 12632 raft_consensus.cc:1275] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:20:57.879956 12783 raft_consensus.cc:1275] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:20:57.837643 19910 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:57.839913 19970 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.842118 19973 raft_consensus.cc:993] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2: : Instructing follower 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:57.741791 19818 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:57.849874 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:57.886901 12780 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:20:57.802176 19898 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:57.793021 20063 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.111s
I20260709 13:20:57.796833 19975 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:57.881913 13376 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:20:57.884434 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:57.862633 19805 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:57.887908 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:57.898084 19898 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:57.888828 19970 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:57.884523 19941 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:57.893262 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:57.896869 12746 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:57.905412 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:57.907912 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 456764us
I20260709 13:20:57.908610 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:57.903698 19898 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:57.919636 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:57.920938 19941 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:57.921116 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:57.884563 19854 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:57.922104 12706 raft_consensus.cc:2393] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 17: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:57.923801 12632 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:20:57.899243 19973 raft_consensus.cc:1081] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Signalling peer 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:20:57.926838 13381 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:20:57.885164 19925 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:57.921195 19994 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:57.925573 19974 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:20:57.963469 20065 log.cc:927] Time spent T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:20:57.969340 19848 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:57.963697 19994 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:57.972829 19902 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:57.972846 19848 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:57.975571 19994 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:57.976640 20068 log.cc:927] Time spent T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:20:57.933967 20058 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:20:57.997108 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 293490us
W20260709 13:20:57.997459 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 287294us
W20260709 13:20:58.014885 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:57.967895 19842 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:57.970567 19970 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:58.031976 12672 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:20:58.039343 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:58.040743 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:58.042034 12707 raft_consensus.cc:2393] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 18: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:58.053305 12598 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:58.057933 12598 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:58.062584 20081 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:58.062829 19818 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:58.063494 19818 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.063958 19818 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:58.072772 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:58.073446 12778 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:20:58.074206 19953 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:58.074347 19818 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:58.076009 19854 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:58.077539 12598 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:58.078655 19902 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:58.079653 19854 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:58.081003 12598 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:58.086349 19902 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.086445 19975 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:58.086943 19941 consensus_queue.cc:1048] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:58.094861 19902 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:58.070667 19898 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:58.094028 20012 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:58.097038 19902 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:58.097744 19833 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:58.098065 19898 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:58.098978 19898 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.099407 19898 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:58.100903 19986 consensus_meta.cc:287] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.791s	user 0.010s	sys 0.000s
I20260709 13:20:58.101503 19898 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:58.107193 19925 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:58.108793 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:58.111706 19895 raft_consensus.cc:993] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2: : Instructing follower 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:20:58.112264 19895 raft_consensus.cc:1081] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Signalling peer 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:20:58.112826 12598 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:58.113636 19925 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:58.114002 19925 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:58.114372 19925 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:58.121940 19925 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:58.124341 19925 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:58.131301 12598 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:20:58.142895 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 270380us
I20260709 13:20:58.075080 19848 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:58.148497 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 262517us
W20260709 13:20:58.141264 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 178611us
I20260709 13:20:58.154245 12598 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:58.165014 13348 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:20:58.171211 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 252681us
I20260709 13:20:58.170564 12598 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:58.175706 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:58.176399 12625 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:20:58.178973 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:58.179630 12625 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:58.174018 13348 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:58.187860 12598 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:20:58.199975 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 291398us
I20260709 13:20:58.204859 12625 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:20:58.187727 12597 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:58.207422 12673 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:58.208840 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:58.209250 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:58.170368 19898 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:58.211074 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:58.169783 19960 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:58.215726 12747 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:58.186810 19975 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:58.170038 19925 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:58.216878 19902 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:58.229212 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:58.229262 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:20:58.227829 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 94999us
I20260709 13:20:58.194310 19953 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:58.232280 20087 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 13:20:58.234000 19833 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:20:58.238638 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:58.195995 20077 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:20:58.216854 19844 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.029s	sys 0.032s
I20260709 13:20:58.243808 20077 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:58.257928 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:58.258172 20077 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:58.224025 19960 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:58.259479 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:58.260341 19960 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:58.240397 20040 consensus_queue.cc:1059] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.023s
I20260709 13:20:58.217319 19826 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:20:58.249807 19854 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:20:58.242218 19902 raft_consensus.cc:697] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 18 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:58.251863 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:58.252198 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:58.264299 19902 consensus_queue.cc:237] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:58.255898 20010 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.265861 20010 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:58.266309 20006 consensus_meta.cc:287] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.620s	user 0.009s	sys 0.002s
I20260709 13:20:58.266741 20006 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:58.267473 12700 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:20:58.268167 20010 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:58.242694 20073 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:20:58.268993 12706 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 18 index: 17. (index mismatch)
I20260709 13:20:58.251221 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:58.270026 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:58.252491 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:58.270690 12701 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:20:58.270282 12779 raft_consensus.cc:2393] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 15: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:20:58.245375 19874 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:58.273777 12783 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 22. Current term is 23. Ops: []
I20260709 13:20:58.281864 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:58.277433 13349 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:58.270947 19829 raft_consensus.cc:993] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2: : Instructing follower 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:20:58.212630 19842 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:20:58.269938 19884 raft_consensus.cc:3055] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:20:58.301934 13349 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:58.309161 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:58.310076 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:58.317464 12672 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:58.320663 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:58.321462 12701 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
W20260709 13:20:58.274565 19844 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.029s	sys 0.032s
I20260709 13:20:58.250322 19941 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:58.326006 13349 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:58.330543 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:58.332680 13382 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:20:58.274410 19805 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:58.315841 19884 raft_consensus.cc:740] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:58.346232 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:58.342422 19826 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.271529 19953 raft_consensus.cc:697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 21 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:58.359524 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:58.315372 19994 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:58.347803 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:58.354808 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:58.355715 12631 raft_consensus.cc:1240] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:58.358268 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:58.299750 20073 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.358769 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:58.372833 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:58.402417 12629 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 19.
I20260709 13:20:58.370035 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:58.370929 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:58.344522 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:58.404242 12705 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:20:58.370957 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:58.372032 13383 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:20:58.405685 12631 raft_consensus.cc:1240] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:58.373952 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:58.405872 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:58.372570 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:58.407258 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:58.407814 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:58.408010 12782 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 22.
I20260709 13:20:58.408529 13382 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:20:58.409545 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:58.410444 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:58.411479 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:58.409602 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:58.374195 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:58.371178 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:58.418217 13377 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:20:58.379468 12633 raft_consensus.cc:2393] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 15: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:20:58.412205 12629 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 22.
I20260709 13:20:58.421168 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:58.423738 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:58.424363 12708 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:58.426194 12707 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:58.426580 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:58.428371 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:58.429431 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:58.430204 13381 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99"
dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:51768
I20260709 13:20:58.432726 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:58.443032 13380 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:20:58.328948 20101 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:58.441242 12631 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:20:58.373441 19829 raft_consensus.cc:993] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2: : Instructing follower 639ab2db265f4747bb7afb93e833f530 to start an election
W20260709 13:20:58.435328 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 247698us
I20260709 13:20:58.358943 20082 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:58.358438 20006 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:20:58.436478 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 248702us
I20260709 13:20:58.372977 20087 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:58.368680 19941 raft_consensus.cc:697] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:58.402904 19966 raft_consensus.cc:1081] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Signalling peer 522fd2ba0e9f402691b5ef29af8579cd to start an election
I20260709 13:20:58.451898 12747 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:58.439553 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:58.486181 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:58.448303 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:58.487511 12628 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:20:58.456283 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:58.352555 19898 raft_consensus.cc:697] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:58.488664 13380 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:20:58.455046 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:58.461520 12746 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:58.458571 12598 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:58.481122 19925 raft_consensus.cc:697] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:20:58.467859 12701 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 18 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:58.489987 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:58.490080 13376 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 20.
I20260709 13:20:58.489223 19898 consensus_queue.cc:237] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:58.490931 12598 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:58.407644 20067 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.492326 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:58.492384 20067 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:58.494591 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:58.495416 20067 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:58.492744 19925 consensus_queue.cc:237] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:58.496781 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:20:58.507377 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 46337.9us
I20260709 13:20:58.508277 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:58.511135 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:58.511484 20101 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.512979 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:58.513736 12708 raft_consensus.cc:2376] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 22. Re-sending same reply.
I20260709 13:20:58.512332 20101 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:58.370139 19826 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:58.457907 20082 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:58.569481 12673 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:58.570349 12708 raft_consensus.cc:1240] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
W20260709 13:20:58.575518 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 114085us
I20260709 13:20:58.487330 20095 raft_consensus.cc:1081] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Signalling peer 639ab2db265f4747bb7afb93e833f530 to start an election
W20260709 13:20:58.579981 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 151436us
I20260709 13:20:58.587502 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:58.588302 12708 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
I20260709 13:20:58.581234 12672 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:58.516155 19829 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:58.590814 12704 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 18 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:58.591583 12780 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 20.
I20260709 13:20:58.576758 19895 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:58.429339 20073 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:58.610255 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 119383us
I20260709 13:20:58.615610 12598 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:58.581101 19972 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:58.584828 20081 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:58.579806 20082 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:20:58.638653 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:58.483683 20087 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:58.575645 20067 consensus_queue.cc:1048] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:20:58.480211 19884 consensus_queue.cc:260] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:58.601202 19829 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:58.650905 13380 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 21.
I20260709 13:20:58.577199 20012 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.051s
I20260709 13:20:58.601169 20105 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:20:58.628985 19974 consensus_meta.cc:287] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.663s	user 0.010s	sys 0.001s
W20260709 13:20:58.652674 20103 log.cc:927] Time spent T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:20:58.629280 19848 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:58.592515 19818 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 13:20:58.653287 19807 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:58.666085 12626 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 18.
I20260709 13:20:58.640244 20063 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:20:58.642066 20059 consensus_queue.cc:1059] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:20:58.667131 19974 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:58.642792 20081 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:58.628124 20040 consensus_queue.cc:1059] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:20:58.672439 12627 raft_consensus.cc:1240] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:58.673142 13381 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:58.695549 13381 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:58.661983 19910 consensus_meta.cc:287] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.777s	user 0.011s	sys 0.000s
I20260709 13:20:58.583499 19953 consensus_queue.cc:237] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:58.644690 19941 consensus_queue.cc:237] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:58.663947 20109 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:58.664182 19829 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:58.667891 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 87664.7us
I20260709 13:20:58.666915 19874 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:58.622385 19902 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:58.669660 20105 raft_consensus.cc:697] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:20:58.670173 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 100791us
I20260709 13:20:58.688267 19805 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:58.694500 20081 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:58.756485 12746 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
W20260709 13:20:58.770386 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 178843us
W20260709 13:20:58.778865 20117 log.cc:927] Time spent T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:20:58.780861 19826 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:58.743637 19902 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:58.723282 20109 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:58.784539 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:58.771795 19825 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:58.779693 20105 consensus_queue.cc:237] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:58.750067 20059 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:58.802544 20073 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:58.795848 20087 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:58.798292 20101 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:20:58.818481 20091 log.cc:927] Time spent T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.568s	user 0.002s	sys 0.002s
W20260709 13:20:58.818910 20077 consensus_meta.cc:287] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.560s	user 0.002s	sys 0.009s
I20260709 13:20:58.715946 19778 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.771507 19958 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
W20260709 13:20:58.785467 20122 log.cc:927] Time spent T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:20:58.787101 20108 log.cc:927] Time spent T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
I20260709 13:20:58.831040 20082 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:58.852167 13381 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:58.856163 20109 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:58.855261 19902 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:58.883185 20077 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:58.861581 20059 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:58.960255 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:58.962010 12628 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:20:58.849704 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 268517us
I20260709 13:20:58.870368 12703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 3169) took 1001 ms. Trace:
I20260709 13:20:58.975558 12672 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:58.978070 12628 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 13.
I20260709 13:20:58.978641 12702 raft_consensus.cc:2368] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 15. Current term is 16.
I20260709 13:20:58.975656 12703 rpcz_store.cc:276] 0709 13:20:57.869235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:57.869375 (+   140us) service_pool.cc:224] Handling call
0709 13:20:57.869692 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:57.870016 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:57.870046 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:57.870584 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 13:20:58.147612 (+277028us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:20:58.422362 (+274750us) log.cc:844] Serialized 14 byte log entry
0709 13:20:58.422660 (+   298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:58.422754 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:58.817841 (+395087us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:20:58.869954 (+ 52113us) raft_consensus.cc:1685] finished
0709 13:20:58.870096 (+   142us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:58.870287 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":89713,"prepare.queue_time_us":95503,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":14424,"raft.queue_time_us":6907,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":18653,"spinlock_wait_cycles":48213504,"thread_start_us":494209,"threads_started":2,"wal-append.queue_time_us":334472}
W20260709 13:20:58.981834 12709 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.000s	sys 0.014s
W20260709 13:20:58.982478 12709 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.000s	sys 0.014s
I20260709 13:20:58.982396 12673 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:20:58.849771 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 93395.3us
I20260709 13:20:58.984917 12672 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:58.771268 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:20:58.988365 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 217020us
I20260709 13:20:58.991353 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:58.988461 12672 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:58.994029 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:59.004092 12672 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:58.997401 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:59.011374 19880 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:58.882881 19778 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:59.021106 19825 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:59.021868 19825 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:59.022285 19825 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:59.012676 19880 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.022050 19826 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:20:59.026115 19966 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:59.026737 19910 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:59.027244 19910 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:59.027467 19966 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:20:59.027788 19910 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:59.028174 19966 raft_consensus.cc:697] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:59.029526 19805 consensus_queue.cc:1059] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:59.030222 19970 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:59.030928 19970 raft_consensus.cc:697] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:59.034096 19848 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:59.032558 19970 consensus_queue.cc:237] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:59.040689 20012 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:59.039045 19966 consensus_queue.cc:237] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:59.042100 20012 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:59.042718 12672 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:20:59.043043 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 45867.2us
I20260709 13:20:59.044019 19833 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:59.044266 19818 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:59.044775 19833 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:59.044962 12672 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:59.045336 19833 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:59.045231 19818 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:59.047791 12709 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "58239dd301f64758846dd758e449e4d8"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:42490
I20260709 13:20:59.048516 19818 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:59.048671 12709 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:59.049206 12709 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:59.052196 20082 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:59.053105 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:59.054248 12632 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:59.054683 20077 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:20:59.056118 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:59.057098 12633 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:20:59.064857 12672 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:59.066346 19805 raft_consensus.cc:3055] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:59.066740 20131 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:59.067209 19805 raft_consensus.cc:740] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:59.067602 20131 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:59.068118 20131 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:59.070044 19805 consensus_queue.cc:260] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:59.025238 19880 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:59.074481 12673 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:59.076078 20012 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:59.076889 20012 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:59.076758 19880 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:59.077462 20012 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:59.049346 19818 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:59.073890 20059 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:59.077945 19880 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:59.094355 19880 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:59.058157 19829 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.095886 12709 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.105255 12709 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:59.058346 20082 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:59.110473 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:59.118556 19805 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:59.123356 19829 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:59.126117 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:59.126863 19889 raft_consensus.cc:3055] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:59.127496 19889 raft_consensus.cc:740] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:20:59.130716 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.131037 19826 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:59.131783 19826 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:59.134740 19907 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:20:59.135951 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.138289 19826 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:59.143039 19907 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:59.149787 20082 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.147397 13349 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:59.155802 12630 raft_consensus.cc:1275] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:20:59.159878 19818 consensus_queue.cc:1048] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:59.165292 19848 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.166225 19848 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.101456 19972 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:59.168833 19848 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:59.171552 19818 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.172581 19818 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.176743 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:59.177867 12626 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:20:59.177799 19972 consensus_queue.cc:1059] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:59.179548 20098 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:59.157344 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:59.180440 20098 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:59.156044 20092 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:59.185547 20116 raft_consensus.cc:3055] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:59.186244 20116 raft_consensus.cc:740] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:59.187433 20116 consensus_queue.cc:260] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.189252 20116 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:59.189795 20131 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.191404 12703 raft_consensus.cc:2393] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 21: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:20:59.129307 19889 consensus_queue.cc:260] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.153400 20082 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:59.194499 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 49198.8us
I20260709 13:20:59.196669 19844 consensus_queue.cc:1059] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:20:59.198007 19844 consensus_queue.cc:1059] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:20:59.199308 20092 raft_consensus.cc:3055] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:59.199801 20092 raft_consensus.cc:740] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:59.201912 19842 raft_consensus.cc:3055] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:20:59.202412 19842 raft_consensus.cc:740] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:59.203543 19842 consensus_queue.cc:260] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.204116 12626 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452"
dest_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:56838
I20260709 13:20:59.204792 12626 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:59.205220 12626 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:59.200943 20092 consensus_queue.cc:260] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.206772 20092 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:59.210638 12625 raft_consensus.cc:1240] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:20:59.210947 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:59.213548 12631 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:20:59.179625 19972 consensus_queue.cc:1243] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 639ab2db265f4747bb7afb93e833f530 log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:20:59.216560 12632 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:20:59.205097 13349 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:59.215546 13381 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:59.201262 19833 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:59.230451 20040 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:59.148922 19910 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.232877 13349 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:59.193437 20131 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:59.239858 20073 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:59.241937 13349 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:59.242621 19907 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:59.215564 20098 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:59.243120 20077 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:59.243116 20095 raft_consensus.cc:3055] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:59.244458 20095 raft_consensus.cc:740] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:59.245937 12627 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 21 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:59.245528 20095 consensus_queue.cc:260] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:59.246953 20095 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:59.247035 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:59.248198 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:59.248355 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:59.249104 12704 raft_consensus.cc:2393] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 15: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:20:59.253047 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:59.253863 12629 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:20:59.256098 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:59.216557 19818 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:59.257098 12631 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:59.261037 20010 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.234630 19910 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:59.249439 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:59.261663 20010 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:59.232990 20109 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.273706 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:59.278100 12704 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:20:59.247303 19825 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:59.281597 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:59.271373 20131 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:59.291002 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:59.301970 19902 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:59.304839 19902 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:59.306989 20010 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:59.309257 13381 raft_consensus.cc:1240] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:59.310159 13385 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7"
dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:51768
I20260709 13:20:59.310900 13385 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:59.311345 13385 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:59.312000 19833 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:59.312961 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:59.313334 19972 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.313609 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:59.314009 19972 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.314453 13379 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
I20260709 13:20:59.316385 19972 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:59.316545 19778 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:59.316959 19972 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.311162 13380 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 18 index: 17. (index mismatch)
W20260709 13:20:59.297323 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 816621us
I20260709 13:20:59.334860 12628 raft_consensus.cc:1240] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:59.381772 12709 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
I20260709 13:20:59.335489 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:59.362859 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:59.320829 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:59.323830 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:59.331257 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:59.318385 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:59.379992 12525 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54800 (request call id 322) took 4376 ms. Trace:
I20260709 13:20:59.397895 13385 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.325807 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:59.318864 19842 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:59.330771 20109 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:59.403532 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.334575 19889 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:59.306993 19825 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:59.404639 12778 raft_consensus.cc:3055] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:59.403183 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:59.403184 12632 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:59.404611 12783 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 20.
I20260709 13:20:59.408121 12703 raft_consensus.cc:1240] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:59.393087 19902 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:59.414108 12631 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
W20260709 13:20:59.414855 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 173051us
I20260709 13:20:59.409778 12778 raft_consensus.cc:740] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:59.317548 19972 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:59.379921 20131 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.391997 20081 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.332583 12528 catalog_manager.cc:5697] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 19 to 21, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 21 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:59.421082 13385 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:59.414412 12629 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 20.
I20260709 13:20:59.424786 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:20:59.428930 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 155390us
I20260709 13:20:59.400947 19880 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:20:59.341624 12517 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 612a4639a7a749be8856bbc60766e312 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER failed with leader 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 639ab2db265f4747bb7afb93e833f530 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:59.431151 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:59.402316 20012 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.412961 19805 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:20:59.434262 12517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 92925.5us
I20260709 13:20:59.438861 20063 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:59.439563 20063 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.444938 12702 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:59.396502 12525 rpcz_store.cc:276] 0709 13:20:55.003272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:55.003422 (+   150us) service_pool.cc:224] Handling call
0709 13:20:59.362748 (+4359326us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:57.910114 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:57.910691 (+   577us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:57.910723 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:20:57.910739 (+    16us) tablet.cc:662] Decoding operations
0709 13:20:57.911113 (+   374us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:57.911184 (+    71us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:57.911225 (+    41us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:20:57.911642 (+   417us) tablet.cc:701] Row locks acquired
0709 13:20:57.911659 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:20:57.911809 (+   150us) write_op.cc:270] Start()
0709 13:20:57.911983 (+   174us) write_op.cc:276] Timestamp: P: 1783603257911758 usec, L: 0
0709 13:20:57.912014 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:20:57.912861 (+   847us) log.cc:844] Serialized 918 byte log entry
0709 13:20:58.716130 (+803269us) op_driver.cc:464] REPLICATION: finished
0709 13:20:58.912243 (+196113us) write_op.cc:301] APPLY: starting
0709 13:20:58.912335 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:58.916753 (+  4418us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:58.916863 (+   110us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:59.017491 (+100628us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:59.063439 (+ 45948us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 13:20:59.063532 (+    93us) write_op.cc:312] APPLY: finished
0709 13:20:59.064161 (+   629us) log.cc:844] Serialized 33 byte log entry
0709 13:20:59.064543 (+   382us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:59.065009 (+   466us) write_op.cc:454] Released schema lock
0709 13:20:59.065084 (+    75us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:59.065851 (+   767us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:59.065892 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":12927,"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":189088,"apply.run_cpu_time_us":9094,"apply.run_wall_time_us":153487,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12495,"num_ops":2,"prepare.queue_time_us":441653,"prepare.run_cpu_time_us":30027,"prepare.run_wall_time_us":321294,"raft.queue_time_us":333087,"raft.run_cpu_time_us":13246,"raft.run_wall_time_us":208182,"replication_time_us":783133,"thread_start_us":751250,"threads_started":4,"wal-append.queue_time_us":428520}]]}
I20260709 13:20:59.412672 19833 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:59.453060 13385 raft_consensus.cc:1240] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:59.442232 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:59.431677 19958 consensus_queue.cc:1059] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 13:20:59.461649 20052 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:59.462441 19958 raft_consensus.cc:3055] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:59.462369 20052 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.462930 19958 raft_consensus.cc:740] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:59.463985 19958 consensus_queue.cc:260] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.464555 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:59.465581 12704 consensus_queue.cc:237] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.465734 19833 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.466259 19884 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:59.469134 20012 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:59.470063 20131 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:59.471251 12708 raft_consensus.cc:1240] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 18. Current term is 19. Ops: [18.17-18.17]
I20260709 13:20:59.473826 12628 raft_consensus.cc:2393] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 17: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:20:59.473958 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:59.474854 13383 raft_consensus.cc:2393] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 14: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:20:59.461712 12781 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:59.477862 20114 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:59.478541 20114 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.481060 20114 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:59.432047 19826 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:59.481617 19805 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:59.482002 19826 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:20:59.484390 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:59.484836 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.469769 12528 catalog_manager.cc:5697] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 20 to 21, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 21 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:59.487473 12528 catalog_manager.cc:5697] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 21 to 22, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 22 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:59.465447 19958 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:59.490446 12705 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:20:59.493048 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.491484 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:59.489271 12528 catalog_manager.cc:5697] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 13 to 15, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 15 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:20:59.495853 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.497192 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:59.499528 12781 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:20:59.495949 12705 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:20:59.504218 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:59.441673 19902 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.501384 12528 catalog_manager.cc:5697] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:59.507797 12632 raft_consensus.cc:2468] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 23.
I20260709 13:20:59.510257 12528 catalog_manager.cc:5697] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 13 to 15, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 15 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:59.518096 12706 raft_consensus.cc:1240] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:59.517465 12528 catalog_manager.cc:5697] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:59.518922 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:59.521116 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:59.521832 12706 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:59.522161 20142 log.cc:927] Time spent T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:20:59.514494 13349 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:59.519532 12707 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 21.
I20260709 13:20:59.509212 19829 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:20:59.530766 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.526854 12634 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:20:59.507373 20012 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.545951 12597 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:59.534523 19829 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:59.524712 20095 consensus_queue.cc:1059] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 13:20:59.559959 13349 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:59.562597 12781 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:59.547798 20116 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:59.558655 12528 catalog_manager.cc:5697] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:59.572576 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:59.578562 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:59.580924 12672 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:59.587869 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:59.588320 19970 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:59.588755 19970 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:59.589145 19970 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:59.589291 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:59.587941 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:59.599174 19970 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.602231 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:59.603753 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:59.580303 12704 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:59.546094 20012 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:59.602453 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:59.560302 19829 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:59.581980 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:59.612401 12630 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:59.561285 20109 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:59.619751 20012 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:59.621182 20081 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:59.621742 19972 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:59.623692 12706 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 18.
W20260709 13:20:59.627470 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 100670us
I20260709 13:20:59.628297 19902 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:59.605990 20092 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:59.658409 20092 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:59.658861 20092 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:20:59.659108 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.609465 12626 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.560087 20063 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:59.602237 19970 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:59.639379 20052 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:20:59.649878 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 90054.1us
I20260709 13:20:59.650905 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:59.654666 20109 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.656435 19907 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:20:59.645812 12528 catalog_manager.cc:5697] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:20:59.665933 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.679394 12625 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:59.683141 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:59.689177 12630 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 21.
I20260709 13:20:59.645246 19880 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:59.691931 12626 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:59.692428 19805 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:59.696955 20045 consensus_queue.cc:1059] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
W20260709 13:20:59.701910 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 121090us
I20260709 13:20:59.702798 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:59.703502 12781 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:59.702633 12528 catalog_manager.cc:5697] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 12 to 14, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 19 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:20:59.691268 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.661871 20116 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.661842 20114 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:59.704540 12528 catalog_manager.cc:5697] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 23 to 25, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 25 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:20:59.709267 13349 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:59.708462 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:59.712589 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:59.712977 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:59.713330 12783 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:20:59.714208 20105 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:59.715337 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:59.715955 12776 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:59.715682 12778 consensus_queue.cc:260] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.716895 12778 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:59.717660 19907 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:59.718444 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:59.719408 13376 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 21.
I20260709 13:20:59.720221 19907 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:59.721258 19941 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:20:59.721841 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:20:59.693002 20063 consensus_queue.cc:1059] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 13:20:59.723672 20116 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.724071 12780 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:20:59.726346 20116 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:59.726477 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:20:59.727310 12709 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:59.727388 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:59.728047 12780 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 24.
I20260709 13:20:59.728286 20109 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:59.729000 20114 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.731292 20114 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:59.733626 19826 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:20:59.735112 12746 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:59.737890 12673 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:59.739360 19825 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:20:59.741113 19804 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:59.710610 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:59.741533 19804 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:59.742064 19804 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:20:59.742250 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 45793.8us
I20260709 13:20:59.742064 12782 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:20:59.747225 19833 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:59.748064 13349 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:59.749094 13349 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:59.745335 12708 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:20:59.749505 20127 raft_consensus.cc:3055] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:20:59.750023 20127 raft_consensus.cc:740] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:59.750700 19842 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:59.751096 12701 raft_consensus.cc:1240] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:59.750941 20127 consensus_queue.cc:260] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.752089 19842 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:59.745527 19953 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:59.753150 12703 raft_consensus.cc:1240] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:59.753958 20109 consensus_queue.cc:1048] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:59.754454 12703 raft_consensus.cc:1240] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:20:59.739552 12673 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:59.760958 12776 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 21.
I20260709 13:20:59.762151 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:59.770015 12704 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:20:59.770692 12701 raft_consensus.cc:1240] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:59.773785 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:59.779744 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:59.778640 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:59.787881 12776 raft_consensus.cc:1240] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:59.788206 12782 raft_consensus.cc:2410] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:20:59.775899 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:20:59.794481 12780 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 22. Current term is 23. Ops: []
I20260709 13:20:59.795630 12626 raft_consensus.cc:1275] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:20:59.810483 20132 raft_consensus.cc:2764] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:20:59.816995 20081 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.817655 20081 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.818588 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:59.819311 12776 raft_consensus.cc:3055] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:59.819619 12776 raft_consensus.cc:740] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:59.820036 20081 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:20:59.820431 12776 consensus_queue.cc:260] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.825229 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:59.826784 12672 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:59.828174 12672 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:59.828001 20092 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.829041 20144 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:59.829624 20144 raft_consensus.cc:697] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:59.830305 20092 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:20:59.830554 20144 consensus_queue.cc:237] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.792484 19826 consensus_queue.cc:1059] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:20:59.831946 20105 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:20:59.833144 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:59.833986 20131 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:59.834370 20131 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:59.834712 20131 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:59.832196 19826 raft_consensus.cc:3055] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:59.837222 19826 raft_consensus.cc:740] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:20:59.838114 19826 consensus_queue.cc:260] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.838680 20127 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:59.843011 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:20:59.843309 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 94315.2us
I20260709 13:20:59.844023 19826 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:59.845320 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:59.845844 12782 raft_consensus.cc:1275] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:20:59.846041 12775 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 22.
I20260709 13:20:59.846374 12782 pending_rounds.cc:85] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 10
I20260709 13:20:59.846717 12782 pending_rounds.cc:107] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:20:59.847023 12782 raft_consensus.cc:2889] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:59.783159 20114 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:59.849481 20114 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:20:59.845793 19884 consensus_queue.cc:1059] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 13:20:59.849946 20114 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:59.850407 19884 consensus_queue.cc:1243] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer 19cd864c053744bebd96093a7982537e log is divergent from this leader: its last log entry 15.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:20:59.850915 19884 raft_consensus.cc:3055] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:59.851235 19884 raft_consensus.cc:740] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:20:59.851261 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:59.852324 19884 consensus_queue.cc:260] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.853436 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:59.853524 19884 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:59.858788 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:59.859499 12783 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
W20260709 13:20:59.864744 20143 log.cc:927] Time spent T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.432s	user 0.004s	sys 0.000s
I20260709 13:20:59.866478 12746 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:59.869706 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:59.870486 13376 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:20:59.870149 12672 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:20:59.845196 13349 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:20:59.872701 12781 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:20:59.781117 19970 raft_consensus.cc:2804] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:20:59.874374 12672 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:59.877753 12709 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 21.
I20260709 13:20:59.883297 12776 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:59.884773 12746 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:59.888463 20147 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:20:59.889117 20147 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:59.889616 20147 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:20:59.889842 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:20:59.890100 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:20:59.890911 19970 raft_consensus.cc:697] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 23 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:20:59.891402 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:20:59.892052 13377 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
I20260709 13:20:59.891975 19970 consensus_queue.cc:237] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:59.879133 19907 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:59.893417 19907 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:59.893901 19907 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:59.879724 20098 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:59.896907 20098 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:59.897256 20098 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:59.899243 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:59.899899 12774 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 18.
I20260709 13:20:59.900859 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:59.879528 19902 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:20:59.813030 19829 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:59.880103 12672 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:20:59.911651 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 165346us
I20260709 13:20:59.921025 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:59.923657 12672 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:59.926990 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:20:59.927736 12781 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:20:59.927098 19825 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:59.929399 12746 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:20:59.929538 19825 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:59.930117 19825 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:59.931051 19826 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:59.931443 19826 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:20:59.931802 19826 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:59.921411 19941 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:59.934082 19941 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.938339 19825 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.940270 12778 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:20:59.944425 19889 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:59.944721 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.901468 20147 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:20:59.947947 12625 raft_consensus.cc:1275] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:20:59.942584 19842 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:59.941521 19826 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.948933 12777 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 21.
I20260709 13:20:59.951351 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:59.950604 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:20:59.957051 12672 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:20:59.883957 20092 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:20:59.963515 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.944949 19889 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.969118 20063 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:20:59.969937 12747 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:59.970082 20131 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:20:59.952003 20081 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:21:00.007323 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 45057.1us
I20260709 13:21:00.002861 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:00.016670 12747 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:20:59.985316 20092 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:20:59.944718 20045 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:20:59.973378 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:20:59.927807 19902 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:59.927626 19972 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:59.959506 19848 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.025087 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:00.009316 19842 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:00.027370 13385 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:00.042459 12776 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 18.
I20260709 13:21:00.007148 19966 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:21:00.007278 19880 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:20:59.985746 20132 consensus_queue.cc:1059] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:20:59.973426 19907 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.007692 20081 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:00.023607 19833 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:00.028730 20092 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:00.050258 12779 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:21:00.030119 19958 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:59.972783 19826 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:00.020431 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:21:00.068805 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 48457.6us
I20260709 13:21:00.025166 20147 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:00.072770 19872 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.073470 19872 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.075840 19872 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:00.076439 19872 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:00.076797 19872 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:00.077113 19872 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:00.080752 20063 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:00.081378 20063 raft_consensus.cc:697] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:00.082410 20063 consensus_queue.cc:237] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:00.059962 13385 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 13.
I20260709 13:21:00.063552 19880 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.063644 19941 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:00.063614 20081 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:21:00.107205 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.054306 20126 consensus_queue.cc:1048] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.100s
I20260709 13:21:00.104423 20147 consensus_queue.cc:1048] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:21:00.128760 13382 raft_consensus.cc:1240] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:00.112921 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:00.102111 19825 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:00.138228 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:00.054569 19842 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.065500 19848 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.145484 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:00.112985 19829 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:00.148696 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:00.132339 20012 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:00.146390 13377 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:21:00.153096 12777 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 21. Current term is 22. Ops: []
I20260709 13:21:00.153600 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:21:00.155956 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 227100us
I20260709 13:21:00.158615 19907 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:00.146755 19884 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.160799 19884 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.163542 19884 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:00.166260 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:00.181314 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:00.182956 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:00.105973 20098 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.180150 13385 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:00.188438 13348 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:00.191287 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: true dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:00.192078 13384 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:00.196372 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.202970 13377 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:21:00.197933 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 188998us
I20260709 13:21:00.212749 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:00.185678 19880 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:00.217780 12597 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:00.221203 20158 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:00.221627 19872 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:00.222035 20158 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:00.225659 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:00.226380 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:00.227988 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:00.229838 12776 raft_consensus.cc:1240] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:00.231382 13376 raft_consensus.cc:1240] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:00.149731 19826 consensus_queue.cc:1059] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.050s
W20260709 13:21:00.243664 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 286655us
W20260709 13:21:00.204066 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 256945us
I20260709 13:21:00.206204 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:00.197232 19953 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.272048 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:00.252473 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:00.254791 19941 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:00.256479 20160 log.cc:927] Time spent T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:21:00.259217 20162 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:00.261904 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:00.264936 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:00.267773 20098 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:00.250834 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:00.268987 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:00.280982 19805 consensus_queue.cc:1059] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:21:00.276536 19907 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.281629 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:00.282187 19907 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.282863 12776 raft_consensus.cc:1240] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:21:00.269840 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:00.283936 19958 consensus_queue.cc:1059] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 17.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:21:00.284657 19907 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:00.274060 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:00.270560 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:00.285679 12629 raft_consensus.cc:2393] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 15: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:21:00.277716 13385 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:21:00.253721 13378 raft_consensus.cc:3055] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:00.286720 13378 raft_consensus.cc:740] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:00.203984 20131 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:00.285168 19826 consensus_queue.cc:1059] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:00.286218 20132 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:00.287741 13378 consensus_queue.cc:260] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.288954 13378 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:00.275801 19941 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.291508 19941 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:00.292114 19941 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:00.292502 19941 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:00.292923 19941 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:00.293545 19842 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:00.294488 13349 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:00.297132 19966 raft_consensus.cc:3055] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:00.297644 19966 raft_consensus.cc:740] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:00.254148 19804 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:00.302258 19804 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:00.278993 20162 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.305078 19889 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:00.307714 19941 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.310102 19805 consensus_queue.cc:1243] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 639ab2db265f4747bb7afb93e833f530 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:21:00.310833 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:00.311970 12778 raft_consensus.cc:3055] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:00.329141 12778 raft_consensus.cc:740] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:00.312309 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:00.312758 13384 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
W20260709 13:21:00.316687 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 64303.3us
I20260709 13:21:00.294823 19842 consensus_queue.cc:1059] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:21:00.331645 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:00.329167 12629 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:00.339632 12709 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
I20260709 13:21:00.347047 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:00.348397 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:00.349162 13378 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:21:00.309883 19889 consensus_queue.cc:1059] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 22.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.021s
I20260709 13:21:00.354846 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:00.355590 12627 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:21:00.314157 19902 raft_consensus.cc:697] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 21 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:00.289364 19953 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:00.356544 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:00.359715 12781 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:00.351610 12706 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:00.316812 20132 raft_consensus.cc:3055] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:21:00.360374 12778 consensus_queue.cc:260] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.364646 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:00.364727 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:00.333825 19884 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:00.331032 20098 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:00.365321 12775 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:00.362185 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:00.365388 12782 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:00.366736 12708 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:21:00.399652 12780 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:21:00.358510 19833 raft_consensus.cc:3055] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:00.403016 12630 raft_consensus.cc:2393] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 17: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:21:00.364145 20132 raft_consensus.cc:740] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:00.441540 12774 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
W20260709 13:21:00.419297 20114 consensus_meta.cc:287] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.569s	user 0.012s	sys 0.001s
I20260709 13:21:00.442031 12774 pending_rounds.cc:85] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 9
I20260709 13:21:00.444913 12774 pending_rounds.cc:107] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:21:00.445225 12774 raft_consensus.cc:2889] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:21:00.446106 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 201220us
I20260709 13:21:00.430430 12781 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:21:00.446753 13348 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:00.414146 19833 raft_consensus.cc:740] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:00.449218 12706 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
W20260709 13:21:00.452317 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 156795us
W20260709 13:21:00.454555 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.452458 12746 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:00.437171 19842 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:21:00.475942 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 195337us
I20260709 13:21:00.443239 20098 raft_consensus.cc:697] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:21:00.472131 20127 consensus_meta.cc:287] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.619s	user 0.004s	sys 0.009s
I20260709 13:21:00.455070 19970 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:00.494211 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 251797us
I20260709 13:21:00.457954 19848 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:00.472597 19842 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:00.496241 20179 raft_consensus.cc:3055] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:21:00.445537 20114 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:00.525142 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 151101us
I20260709 13:21:00.528041 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:00.501848 19842 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:00.537026 20012 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:21:00.538014 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 83064.6us
I20260709 13:21:00.534727 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:00.538812 13348 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:00.539239 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:00.539865 13378 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 21.
I20260709 13:21:00.540437 13383 raft_consensus.cc:1240] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:00.541812 13348 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:00.516561 20179 raft_consensus.cc:740] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:00.523799 20181 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:00.471371 19902 consensus_queue.cc:237] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.537654 20012 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:00.547435 13348 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:00.547977 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:00.501116 19970 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.577450 12627 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 22.
I20260709 13:21:00.578088 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:00.564246 20180 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:00.510886 20132 consensus_queue.cc:260] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:00.540297 20186 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:21:00.589443 20176 log.cc:927] Time spent T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:21:00.564266 20192 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:21:00.592963 20190 raft_consensus.cc:2764] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:21:00.563613 12597 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:00.542223 20105 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:00.577602 12629 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:21:00.565508 12631 raft_consensus.cc:1275] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:21:00.537931 19958 consensus_queue.cc:1059] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.041s
I20260709 13:21:00.546389 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:00.545946 20063 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.546188 19941 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:00.588712 13378 raft_consensus.cc:2393] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 18: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:21:00.588891 20132 raft_consensus.cc:3060] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:00.588003 13348 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:00.579537 12673 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:00.551872 19833 consensus_queue.cc:260] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.17, Last appended by leader: 17, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.601110 20185 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:00.576100 20181 raft_consensus.cc:697] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:21:00.603972 19972 consensus_meta.cc:287] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.565s	user 0.007s	sys 0.003s
W20260709 13:21:00.605896 20092 consensus_meta.cc:287] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.534s	user 0.007s	sys 0.004s
I20260709 13:21:00.606268 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:00.607951 20173 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:00.586442 20180 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:00.658190 20180 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:00.549696 20098 consensus_queue.cc:237] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.610878 20190 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:00.660463 20190 raft_consensus.cc:697] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:21:00.609371 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 45916.9us
I20260709 13:21:00.609768 19953 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:00.614349 20063 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:00.661752 20190 consensus_queue.cc:237] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.663309 12597 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:00.663692 20063 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:00.616207 20116 raft_consensus.cc:3055] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:21:00.664288 20178 log.cc:927] Time spent T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:21:00.664311 20116 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:00.664597 19953 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:00.642441 20131 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:21:00.665479 20131 raft_consensus.cc:697] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 22 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:00.665429 20116 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 22.20, Last appended by leader: 20, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.666657 20131 consensus_queue.cc:237] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:00.666968 12597 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:00.666985 20116 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:00.617789 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:00.668310 13378 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:00.642773 20127 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:21:00.620893 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 74579.7us
I20260709 13:21:00.673942 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:00.658011 20173 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.674683 12627 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:21:00.676038 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:00.676622 20173 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:00.620998 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:00.677114 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:00.677347 13380 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:00.677738 12708 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:21:00.679313 12704 raft_consensus.cc:1240] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:00.676709 12706 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 23.
I20260709 13:21:00.628185 12626 raft_consensus.cc:1275] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:21:00.628320 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:00.682282 12633 raft_consensus.cc:1240] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:21:00.683036 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:00.684135 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:00.640630 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 52708.4us
I20260709 13:21:00.686884 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:00.686012 20168 raft_consensus.cc:2955] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Committing config change with OpId 17.13: config changed from index 12 to 13, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:00.687464 13376 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:00.688043 12775 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 22.
I20260709 13:21:00.645030 13384 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:21:00.648525 19917 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:00.651335 20179 consensus_queue.cc:260] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.655427 20162 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:00.657284 20114 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:00.676339 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:00.712489 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:21:00.635089 20081 consensus_meta.cc:287] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.508s	user 0.012s	sys 0.000s
I20260709 13:21:00.693215 13380 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:21:00.643477 12778 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:00.610946 19966 consensus_queue.cc:260] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:00.680009 19889 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.699863 12673 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:00.618696 19970 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:00.701880 13378 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:21:00.707190 19917 raft_consensus.cc:697] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:00.684638 13377 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:21:00.719655 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:00.699719 13376 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 21.
I20260709 13:21:00.701009 19972 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:00.720955 12627 raft_consensus.cc:2955] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 12 to 13, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:21:00.726292 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 53197.4us
I20260709 13:21:00.727712 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:00.731667 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 64868.5us
I20260709 13:21:00.719215 19953 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:00.711990 20186 raft_consensus.cc:697] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:00.740908 12782 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:21:00.740135 12626 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:21:00.740288 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:00.741492 12778 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:21:00.719245 20191 raft_consensus.cc:3055] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:00.742030 13377 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 23.
I20260709 13:21:00.698560 19818 consensus_queue.cc:1048] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:21:00.754802 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:00.710299 20092 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.733975 20172 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:00.760360 12597 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:00.762238 12777 raft_consensus.cc:1275] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:21:00.712411 20181 consensus_queue.cc:237] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:00.757424 19825 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 13:21:00.722999 20081 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:00.795943 13376 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 18. Current term is 19. Ops: []
W20260709 13:21:00.792232 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.743433 20191 raft_consensus.cc:740] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:00.733641 20147 consensus_queue.cc:1048] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:21:00.728168 19889 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.800247 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:00.733589 20063 consensus_queue.cc:1059] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:21:00.762842 20172 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:00.819253 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:00.822037 20063 raft_consensus.cc:3055] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:00.819636 20172 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:00.822695 20063 raft_consensus.cc:740] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:21:00.826203 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 72383.2us
I20260709 13:21:00.815827 20185 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:00.850088 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:00.802127 20127 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:21:00.853348 20201 log.cc:927] Time spent T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:21:00.864483 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:00.866583 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:00.853933 20191 consensus_queue.cc:260] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.854836 19941 consensus_queue.cc:1059] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:00.868286 19941 raft_consensus.cc:3055] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:21:00.868512 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.868681 19941 raft_consensus.cc:740] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:00.869467 13379 raft_consensus.cc:1275] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:21:00.869772 19941 consensus_queue.cc:260] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.871165 19941 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:00.871945 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:00.872622 13379 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:21:00.852773 12597 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:00.807998 19953 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:00.874679 19953 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.875033 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:00.876190 13381 raft_consensus.cc:2410] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:21:00.858991 13376 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:00.877041 19953 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:00.878448 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:00.878334 20186 consensus_queue.cc:237] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.879370 13381 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:21:00.847559 19917 consensus_queue.cc:237] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.862304 19842 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.882017 20109 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:00.881737 20127 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.882493 20109 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:00.882910 20109 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:00.862353 19889 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:00.860625 19907 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.819427 20162 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:00.884488 13380 raft_consensus.cc:2393] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 16: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:21:00.885986 12597 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:00.889663 13376 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 22.
I20260709 13:21:00.886819 20158 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 13:21:00.894788 13376 raft_consensus.cc:1240] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:00.884330 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:00.895639 13381 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:00.894907 12597 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:21:00.899102 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.884052 19842 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:00.900557 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:00.910943 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:00.921907 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:00.922863 12775 raft_consensus.cc:2410] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:21:00.924085 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:00.924759 12781 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:21:00.901376 20185 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:00.939492 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:00.946672 13376 raft_consensus.cc:2410] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
W20260709 13:21:00.943177 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.958420 12782 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:00.902814 19953 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:00.888665 20012 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:00.893795 20180 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.894309 19907 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.969098 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:21:00.970130 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 234576us
I20260709 13:21:00.941749 19889 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:00.902740 20158 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 13:21:00.977512 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:00.982364 12598 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:00.902233 20162 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:00.985177 13379 raft_consensus.cc:1240] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:00.987192 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:00.988096 13349 leader_election.cc:400] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 15. Current term is 16.
I20260709 13:21:00.985107 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:00.989156 12747 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:00.990733 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:00.994576 12774 raft_consensus.cc:2410] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 24 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 13:21:00.997438 12672 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:00.998831 12709 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:01.003015 12783 raft_consensus.cc:1275] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:21:01.004316 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:21:01.008353 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:01.010346 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:01.011255 12704 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:21:01.009419 12700 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 24. Current term is 25. Ops: []
I20260709 13:21:00.979162 20012 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:01.016378 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:01.017148 12708 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 22. Current term is 23. Ops: []
I20260709 13:21:01.018601 12747 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:01.019246 12780 raft_consensus.cc:1275] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 12. (term mismatch)
I20260709 13:21:01.041510 13376 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:21:01.044510 12780 pending_rounds.cc:85] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 11
I20260709 13:21:01.044906 12780 pending_rounds.cc:107] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 22.12
I20260709 13:21:01.045274 12780 raft_consensus.cc:2889] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:01.042200 12707 raft_consensus.cc:1240] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:01.052989 13382 raft_consensus.cc:2393] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 17: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:21:01.053879 20168 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:49378 (request call id 24) took 1715 ms. Trace:
I20260709 13:21:01.054857 20168 rpcz_store.cc:276] 0709 13:20:59.338853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:59.341264 (+  2411us) service_pool.cc:224] Handling call
0709 13:20:59.468448 (+127184us) log.cc:844] Serialized 494 byte log entry
0709 13:20:59.651844 (+183396us) spinlock_profiling.cc:243] Waited 9.29 ms on lock 0x7b18003cce80. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c53f3 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:21:01.053718 (+1401874us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34,"raft.queue_time_us":2933,"raft.run_cpu_time_us":1505,"raft.run_wall_time_us":50835,"spinlock_wait_cycles":21365504,"thread_start_us":94616,"threads_started":1,"wal-append.queue_time_us":211165,"wal-append.run_cpu_time_us":2009,"wal-append.run_wall_time_us":1022271}
W20260709 13:21:01.057380 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 90134.7us
I20260709 13:21:01.042253 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:21:01.060465 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 44872.2us
I20260709 13:21:01.062289 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:01.049660 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:01.058980 13349 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:01.067504 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:01.062402 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f2f0a3339d324fd4902b6ff970ff0272 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:21:01.003880 19958 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:21:01.014699 20092 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:01.070852 12705 raft_consensus.cc:2393] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 17: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:01.072252 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:00.987746 19889 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:01.080902 19902 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:01.047999 19880 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:21:01.081935 20197 raft_consensus.cc:2764] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:21:01.077939 19829 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:01.069998 20181 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:21:01.104244 19833 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:01.106740 13383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 1839) took 1237 ms. Trace:
I20260709 13:21:01.097079 20212 consensus_queue.cc:1059] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:21:01.107191 13383 rpcz_store.cc:276] 0709 13:20:59.868966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:59.871713 (+  2747us) service_pool.cc:224] Handling call
0709 13:20:59.875345 (+  3632us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:00.542767 (+667422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:01.106475 (+563708us) spinlock_profiling.cc:243] Waited 663 ms on lock 0x7b54004998d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:21:01.106641 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1523956480}
I20260709 13:21:01.115139 12747 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:01.071952 20185 raft_consensus.cc:697] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:01.118532 19829 raft_consensus.cc:697] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:01.119318 20162 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:01.133596 19889 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:01.133846 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:01.120518 20207 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:01.134680 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:01.135078 20207 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:01.130419 20158 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:01.136420 20158 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:01.137132 19829 consensus_queue.cc:237] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:01.070029 20206 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:01.138758 20158 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:01.139243 20172 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:01.124346 20132 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:01.141927 20132 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:01.142274 20132 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:01.146611 13381 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:21:01.141399 20172 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:01.138684 20212 raft_consensus.cc:3055] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:21:01.146590 20181 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:21:01.159116 20212 raft_consensus.cc:740] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:01.160837 20181 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:01.161267 20181 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:01.162385 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:01.164786 12705 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:21:01.116701 19902 raft_consensus.cc:697] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 18 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:01.092859 20105 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:01.137174 20207 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:01.130230 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 71337.6us
I20260709 13:21:01.131536 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:01.169615 12707 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
I20260709 13:21:01.131958 20180 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:01.135314 12702 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:01.121572 20109 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:01.166234 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 172439us
W20260709 13:21:01.129701 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:01.179687 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:01.180918 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:01.180972 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 138807us
I20260709 13:21:01.181628 12774 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 21.
I20260709 13:21:01.184002 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:01.184878 12774 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 17.
I20260709 13:21:01.186206 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:01.188395 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:01.188742 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:01.189205 12774 raft_consensus.cc:2393] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 16: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:01.192356 12783 raft_consensus.cc:1240] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 10. Current term is 11. Ops: []
I20260709 13:21:01.193405 13348 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:01.193734 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:01.194772 12783 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 17.
I20260709 13:21:01.194824 19917 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:01.195489 19917 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:01.195933 19917 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:01.195853 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:01.201830 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:01.203018 12774 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 19: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:21:01.204399 12627 raft_consensus.cc:1275] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:21:01.182952 12672 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:01.204452 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:01.169616 19872 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:01.219697 12701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:49378 (request call id 28) took 1300 ms. Trace:
I20260709 13:21:01.219599 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:01.179201 19953 raft_consensus.cc:697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:01.261287 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:01.262285 19953 consensus_queue.cc:237] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:01.161815 20212 consensus_queue.cc:260] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:01.266747 20212 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:01.271396 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:01.234975 12597 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:01.222347 12701 rpcz_store.cc:276] 0709 13:20:59.919385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:59.919551 (+   166us) service_pool.cc:224] Handling call
0709 13:21:01.212473 (+1292922us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b54003ec1d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b8735 00007f4c6d3b09fa 00007f4c723784c9 00007f4c6b8f4c54 00007f4c6b8f4be2 00007f4c6b8f4b11 00007f4c6b8f4a8c 00007f4c6b8f3d42 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:21:01.219591 (+  7118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":659692800}
I20260709 13:21:01.239954 20063 consensus_queue.cc:260] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:01.243651 20109 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:01.223970 12747 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:01.245113 19917 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:01.246866 20185 consensus_queue.cc:237] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:01.249675 20205 log.cc:927] Time spent T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
I20260709 13:21:01.251120 19907 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:01.252024 20132 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:01.257753 19958 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:21:01.211333 20219 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:21:01.286798 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 103973us
I20260709 13:21:01.352134 20219 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:01.352965 12672 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:21:01.258960 20209 log.cc:927] Time spent T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 13:21:01.288883 12632 raft_consensus.cc:1240] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:01.327286 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:01.353830 20131 consensus_queue.cc:1048] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:01.355334 12625 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:21:01.356627 20131 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:01.259254 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:01.357879 12628 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:21:01.358397 20132 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:01.326732 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 91856.7us
W20260709 13:21:01.327135 12517 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 612a4639a7a749be8856bbc60766e312 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER failed with leader 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 639ab2db265f4747bb7afb93e833f530 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:01.260344 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:01.327162 20105 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:01.302259 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:01.363818 20215 consensus_queue.cc:1059] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:21:01.366044 12597 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:01.279428 19848 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:01.284852 20147 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:01.269594 19902 consensus_queue.cc:237] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:01.366304 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 142429us
I20260709 13:21:01.282078 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:01.290597 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:01.374809 12633 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 21.
I20260709 13:21:01.376227 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:01.377835 20127 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:01.380661 12783 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:21:01.382123 13348 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:01.384902 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:01.385938 12707 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:01.386976 12783 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:21:01.365392 20217 raft_consensus.cc:3055] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:21:01.388556 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:01.389850 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:01.339341 20168 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:01.394104 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:01.395613 12783 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:21:01.373939 19907 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:01.385129 20109 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:01.380631 19848 raft_consensus.cc:697] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:01.391474 20190 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:01.380798 19970 consensus_queue.cc:1048] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:21:01.391255 20217 raft_consensus.cc:740] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:21:01.396363 12672 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.68:36021 timed out after 1.503s (SENT) [suppressed 18 similar messages]
I20260709 13:21:01.413970 19907 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:01.426584 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:01.429001 13381 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:01.380570 20147 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:01.388692 20127 consensus_queue.cc:1059] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.035s
I20260709 13:21:01.418257 20162 consensus_queue.cc:1059] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:21:01.393091 19829 consensus_queue.cc:1048] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:21:01.434742 13383 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
W20260709 13:21:01.434983 20199 log.cc:927] Time spent T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:21:01.436980 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:01.438232 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:01.425977 20190 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:01.408874 20168 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:01.420473 20158 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:21:01.431612 19907 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:01.447253 20221 consensus_queue.cc:1059] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:21:01.451030 20151 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:01.436712 19966 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:01.463958 20183 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:01.444360 20190 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:01.462946 20162 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:01.496841 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:01.497984 12783 raft_consensus.cc:2393] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 11: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:21:01.499198 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:01.463645 19848 consensus_queue.cc:237] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:01.503666 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 107322us
W20260709 13:21:01.504792 12672 leader_election.cc:336] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Timed out: RequestConsensusVote RPC to 127.11.63.68:36021 timed out after 1.865s (SENT)
I20260709 13:21:01.506558 12672 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:01.500510 19825 consensus_queue.cc:1059] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.008s
I20260709 13:21:01.509342 19825 raft_consensus.cc:3055] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:01.509763 19825 raft_consensus.cc:740] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:01.513896 20179 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:01.501037 19917 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:01.518564 20191 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:01.525195 20162 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:21:01.510813 19825 consensus_queue.cc:260] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:01.535250 12707 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:21:01.488296 20183 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:01.559682 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:01.560640 13376 raft_consensus.cc:2410] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 24 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 13:21:01.558990 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:01.561872 13376 raft_consensus.cc:1275] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:21:01.562916 13349 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:01.558825 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:01.564952 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:01.562973 13376 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 13:21:01.567781 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:01.571895 12629 raft_consensus.cc:2393] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 16: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:01.578372 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:01.584903 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:01.580583 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:01.566601 19825 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:01.617944 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
W20260709 13:21:01.620841 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 116091us
I20260709 13:21:01.558367 19970 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:01.619208 13382 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:01.612370 12781 raft_consensus.cc:1240] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:21:01.611891 13377 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:21:01.560912 20092 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:01.504737 20182 consensus_queue.cc:1059] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.079s
I20260709 13:21:01.580188 20116 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:01.625528 12626 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:01.625339 12672 leader_election.cc:336] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Timed out: RequestConsensusVote RPC to 127.11.63.68:36021 timed out after 1.890s (SENT)
I20260709 13:21:01.570571 20229 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:01.627511 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:01.636122 12672 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:01.642896 20230 log.cc:927] Time spent T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:21:01.611169 20151 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:01.495167 20217 consensus_queue.cc:260] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:01.684132 20241 log.cc:927] Time spent T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:21:01.662621 20169 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:01.618600 20238 consensus_queue.cc:1059] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.091s
I20260709 13:21:01.670902 20183 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:01.676640 20081 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:01.679291 20127 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:01.679597 20151 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:01.670286 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 47850.8us
I20260709 13:21:01.682080 20168 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:01.634764 20092 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:01.684260 20105 raft_consensus.cc:3055] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:01.670032 20116 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:01.647225 20206 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:01.713794 12626 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 19.
W20260709 13:21:01.714246 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 155500us
I20260709 13:21:01.681988 19829 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:01.716701 20212 consensus_queue.cc:1175] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "522fd2ba0e9f402691b5ef29af8579cd" responder_term: 21 status { last_received { term: 21 index: 15 } last_committed_idx: 15 last_received_current_leader { term: 21 index: 15 } }
I20260709 13:21:01.717178 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:01.717355 20238 consensus_queue.cc:1243] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer 522fd2ba0e9f402691b5ef29af8579cd log is divergent from this leader: its last log entry 14.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 13:21:01.718082 12672 leader_election.cc:336] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Timed out: RequestConsensusVote RPC to 127.11.63.68:36021 timed out after 1.897s (SENT)
W20260709 13:21:01.725435 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:01.726552 12630 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:01.696728 20219 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:01.657141 20229 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:01.712630 20207 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:01.722498 20116 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:01.737847 12672 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:01.740135 12672 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:01.738492 20116 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:01.740808 12774 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:21:01.731935 19958 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:01.750325 19958 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:01.750686 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:01.760993 20151 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:01.762193 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:01.721238 20105 raft_consensus.cc:740] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:01.767194 12672 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:01.779088 19958 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:01.779752 12783 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:21:01.779881 20063 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:01.781234 19907 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:01.794085 19848 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:01.791074 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:21:01.787719 20181 consensus_meta.cc:287] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.626s	user 0.010s	sys 0.000s
I20260709 13:21:01.785420 20190 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:01.801340 20181 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:01.801896 20092 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:01.770648 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:01.785790 12779 raft_consensus.cc:1275] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:21:01.803782 12781 raft_consensus.cc:3060] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:01.793429 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:01.764377 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:01.818506 12702 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:21:01.818914 19907 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:01.724160 20127 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:01.820333 20190 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:01.820596 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:01.821213 20127 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:01.821393 12626 raft_consensus.cc:2393] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 13: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:21:01.822054 12630 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 22.
I20260709 13:21:01.826182 13382 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
I20260709 13:21:01.827241 12633 raft_consensus.cc:1240] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:01.833245 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:01.754508 20206 raft_consensus.cc:697] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:01.846084 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:21:01.856865 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 278614us
I20260709 13:21:01.792057 20131 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:01.911947 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:01.913717 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 155007us
I20260709 13:21:01.913915 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:01.826912 20189 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:01.914582 12672 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:01.732683 19829 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:01.921303 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:01.792970 20244 raft_consensus.cc:3055] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:21:01.906507 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 63001us
I20260709 13:21:01.905465 12774 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 20.
I20260709 13:21:01.905395 13382 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
I20260709 13:21:01.922576 13349 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:01.913443 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:01.755951 20219 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:01.916739 12597 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:21:01.926860 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:01.926792 13379 raft_consensus.cc:2468] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
I20260709 13:21:01.927759 13381 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 21.
W20260709 13:21:01.856894 20208 log.cc:927] Time spent T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.761s	user 0.005s	sys 0.000s
I20260709 13:21:01.930534 12597 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:01.786228 20229 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:01.938508 19941 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:01.925716 12747 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:01.923807 20131 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:01.936003 20220 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:01.938076 20180 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:01.938432 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:01.992798 20220 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:01.942718 20147 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:01.920840 19970 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:21:01.905977 19958 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:01.945291 20242 consensus_queue.cc:1059] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:21:01.994449 19958 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:01.946525 20206 consensus_queue.cc:237] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:01.996781 19958 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:01.949515 12777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 3959) took 1744 ms. Trace:
I20260709 13:21:01.997418 12777 rpcz_store.cc:276] 0709 13:21:00.205023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.205217 (+   194us) service_pool.cc:224] Handling call
0709 13:21:00.238019 (+ 32802us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:00.762681 (+524662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:01.949279 (+1186598us) spinlock_profiling.cc:243] Waited 515 ms on lock 0x7b54003700d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:21:01.949439 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1183374080}
I20260709 13:21:01.924213 20189 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:01.929927 13349 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:01.923970 19880 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:01.965878 20087 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:21:01.999086 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 69231.8us
I20260709 13:21:01.999529 20087 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:02.000299 20189 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:02.000445 12626 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:21:02.001029 20179 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:01.966126 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:02.001968 12776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36478 (request call id 4251) took 1018 ms. Trace:
I20260709 13:21:01.993331 13381 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 23.
I20260709 13:21:01.930002 20105 consensus_queue.cc:260] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:01.930300 19826 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:01.970777 12779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 4240) took 1637 ms. Trace:
I20260709 13:21:02.003216 13349 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:01.974339 20239 consensus_queue.cc:1059] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:21:01.975590 20185 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:01.982995 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 57412.1us
I20260709 13:21:02.005400 13349 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:21:02.005918 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 84702.5us
I20260709 13:21:02.003630 12779 rpcz_store.cc:276] 0709 13:21:00.333237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.333591 (+   354us) service_pool.cc:224] Handling call
0709 13:21:00.354499 (+ 20908us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:01.803996 (+1449497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:01.970488 (+166492us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b540008a054. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:21:01.970709 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3238787456}
I20260709 13:21:01.828500 20190 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.096s
I20260709 13:21:01.986344 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:01.989238 13384 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:21:01.993306 20220 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:02.002430 12776 rpcz_store.cc:276] 0709 13:21:00.983397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:01.965581 (+982184us) service_pool.cc:224] Handling call
0709 13:21:02.001901 (+ 36320us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:02.006923 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:01.927333 20244 raft_consensus.cc:740] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:02.010733 12781 raft_consensus.cc:2468] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 22.
I20260709 13:21:02.012953 13383 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:01.955047 20181 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:02.014424 13349 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:01.954638 19829 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:02.016887 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:02.036559 12783 raft_consensus.cc:2368] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 17. Current term is 18.
I20260709 13:21:02.041810 20197 raft_consensus.cc:2804] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:21:02.042493 20197 raft_consensus.cc:697] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:02.042853 20131 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:02.043586 20197 consensus_queue.cc:237] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.044718 20081 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:02.046656 20116 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.046551 20244 consensus_queue.cc:260] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.048484 20244 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:02.049181 20116 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:02.049974 20254 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:21:02.050870 20105 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:02.051084 13383 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:21:02.015551 20179 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:21:02.013527 20147 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:21:02.052047 20181 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:02.015902 20185 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:21:02.018771 19889 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:02.062798 20252 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:21:02.064055 20255 log.cc:927] Time spent T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:21:02.076359 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:02.145709 12780 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 19.
W20260709 13:21:02.078122 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:02.077251 12672 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:02.079676 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:02.013589 19826 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:02.147511 20217 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:02.148855 19826 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:02.147311 12630 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:02.084933 20253 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:21:02.131781 20203 log.cc:927] Time spent T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 1.132s	user 0.006s	sys 0.005s
I20260709 13:21:02.134572 20257 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.139273 19842 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:02.140928 12746 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:02.143115 20185 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:02.129069 20116 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.130846 19889 raft_consensus.cc:697] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:02.128793 20252 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:02.154883 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 92335.9us
I20260709 13:21:02.153124 20243 raft_consensus.cc:3055] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:02.182909 20243 raft_consensus.cc:740] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:02.183925 20243 consensus_queue.cc:260] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:02.185290 20243 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:02.169893 20252 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:02.189707 20253 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.170032 20116 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.195179 20116 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:02.158710 19826 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:02.195815 19826 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:02.196175 19826 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:21:02.197698 20218 log.cc:927] Time spent T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:21:02.161854 19842 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.157008 20098 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:02.180102 19889 consensus_queue.cc:237] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.202911 12672 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:02.211416 12672 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:02.201424 19902 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.021s	sys 0.025s
I20260709 13:21:02.154660 20181 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:02.197202 20220 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:02.219489 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:02.217939 12630 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:21:02.216673 20081 consensus_queue.cc:1048] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:21:02.232707 20219 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:21:02.242033 19902 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.021s	sys 0.032s
I20260709 13:21:02.218007 12672 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:02.234834 12746 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:02.161687 20257 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.259538 20257 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:02.203836 20147 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:02.265470 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:02.266723 12707 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:02.267298 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:02.266431 20147 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:02.268757 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:02.268925 19880 consensus_queue.cc:1059] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:21:02.269337 12701 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:02.268785 20147 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:02.272416 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:02.273103 20012 raft_consensus.cc:3055] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:02.273487 20012 raft_consensus.cc:740] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:02.273878 19880 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:21:02.274263 20012 consensus_queue.cc:260] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.275672 20012 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:02.280663 19842 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:02.283625 20182 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:21:02.290412 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 72483.7us
I20260709 13:21:02.291496 12672 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:02.284832 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:02.294389 12631 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:02.297806 12707 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:21:02.301766 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:02.304111 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:02.297749 20182 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.304471 12598 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:02.305083 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:02.309624 12631 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 22.
I20260709 13:21:02.305526 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:02.308306 20182 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:02.313510 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:02.313853 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:02.315569 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:02.261438 19825 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.334336 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:02.277480 20229 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:21:02.277248 20131 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:02.343180 12626 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:21:02.316442 12707 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:21:02.318753 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:02.321102 12747 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:02.354954 12705 raft_consensus.cc:1275] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:21:02.352052 12747 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:21:02.357929 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 320712us
I20260709 13:21:02.344898 19902 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:21:02.343138 20229 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:02.369181 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 64841.5us
I20260709 13:21:02.345649 20131 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:02.371309 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:02.379232 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:02.380301 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:02.380323 20254 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:02.381048 12781 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 21.
I20260709 13:21:02.362501 19958 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:02.372848 20182 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:02.369480 19907 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:02.342449 19825 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.373775 20229 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:02.390141 12598 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:02.389377 12672 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:02.399106 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:02.400394 13381 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 21.
I20260709 13:21:02.399212 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:02.396332 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:02.399361 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:02.402149 13383 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:02.380704 20131 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:02.402809 13384 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:21:02.401765 13385 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:21:02.406648 13378 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:21:02.382086 19902 raft_consensus.cc:697] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 22 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:02.397327 19958 raft_consensus.cc:697] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:02.410470 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:02.412479 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:02.415345 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:02.416174 13379 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 11.
I20260709 13:21:02.386067 20220 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:02.396253 19907 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:02.414001 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:02.423187 12705 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:02.388195 19818 consensus_queue.cc:1048] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:21:02.399418 20116 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:02.423852 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:02.429193 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:02.397759 20190 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:02.425374 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:02.435652 12631 raft_consensus.cc:1240] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:02.436270 12706 raft_consensus.cc:2368] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 17. Current term is 18.
I20260709 13:21:02.419799 12701 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 24.
I20260709 13:21:02.385380 20238 consensus_queue.cc:1059] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:21:02.403322 20197 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.418944 20172 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:02.430094 12700 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:21:02.407120 20092 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.424126 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:02.435043 13383 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 24.
I20260709 13:21:02.409013 20244 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:02.432605 12708 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:21:02.385713 19826 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.394299 20221 consensus_queue.cc:1059] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 24.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:21:02.394279 19941 consensus_queue.cc:1059] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:21:02.388517 20254 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.399500 20217 consensus_queue.cc:1059] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 13:21:02.466159 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:02.467242 12704 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 20: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:02.398218 19880 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:21:02.441103 13348 leader_election.cc:400] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 17. Current term is 18.
I20260709 13:21:02.471741 13348 leader_election.cc:403] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:21:02.441455 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:02.428216 19907 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:02.449597 19825 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:02.423266 19958 consensus_queue.cc:237] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.446060 19972 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:02.394352 19848 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 13:21:02.417891 20087 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:02.472829 13348 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:02.443051 20169 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:02.497124 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:02.458112 19917 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.499626 13348 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:02.436656 20206 consensus_queue.cc:1059] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 13:21:02.506141 12704 raft_consensus.cc:2393] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 19: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:02.519184 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:02.519691 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:02.519991 12700 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 12 FOLLOWER]: Advancing to term 14
W20260709 13:21:02.512652 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 71259us
I20260709 13:21:02.440485 20190 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.439453 20116 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.524523 12708 raft_consensus.cc:2368] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 17. Current term is 18.
I20260709 13:21:02.423417 19902 consensus_queue.cc:237] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.497723 20189 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.494735 19842 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 17. Current term is 18.
I20260709 13:21:02.498169 20132 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:02.523500 12747 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:02.539474 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:02.458195 20197 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.498556 20181 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:02.514765 20257 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:02.516201 20180 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:02.462507 20092 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.504914 20186 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:02.598440 20186 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:02.598915 20186 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:02.601030 12629 raft_consensus.cc:3055] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:02.515182 20238 raft_consensus.cc:3055] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:21:02.601931 20238 raft_consensus.cc:740] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:02.602195 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:02.602999 12625 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:21:02.602875 20238 consensus_queue.cc:260] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:02.603915 20190 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:02.579499 20206 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:02.605412 20206 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.605876 20206 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:02.607882 20206 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:02.604326 20238 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:02.581854 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:02.583356 12707 raft_consensus.cc:2410] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 19 because replica has last-logged OpId of term: 18 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 15.
W20260709 13:21:02.582155 20263 log.cc:927] Time spent T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:21:02.586844 20254 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:02.589401 20116 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:02.591224 20197 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:02.591486 12779 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:21:02.622355 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:02.593467 20092 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:02.521971 19917 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.596143 20252 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.567031 20189 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:02.600703 20147 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.601514 12629 raft_consensus.cc:740] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:02.602380 19826 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:02.574728 20180 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:02.572201 12704 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:21:02.610088 12747 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:02.612208 13379 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:02.614403 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:02.614202 20063 consensus_queue.cc:1048] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:02.624029 12597 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:21:02.667913 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 58027.7us
I20260709 13:21:02.644771 12629 consensus_queue.cc:260] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.638296 19972 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:02.674767 12702 raft_consensus.cc:1240] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:02.672043 12777 raft_consensus.cc:1240] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 18. Current term is 19. Ops: []
W20260709 13:21:02.667944 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 44060.9us
I20260709 13:21:02.641413 19941 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:02.669147 19880 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:02.669426 20087 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:21:02.662209 20172 raft_consensus.cc:697] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:02.666430 20180 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:02.683660 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 175596us
I20260709 13:21:02.685083 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:21:02.671222 20105 consensus_meta.cc:287] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.587s	user 0.007s	sys 0.004s
I20260709 13:21:02.687698 12701 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:21:02.690054 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:02.672708 19826 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:21:02.632822 20132 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:02.691288 12746 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:02.694846 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:02.695869 12708 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 13:21:02.700719 12746 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:02.705955 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:02.706210 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:02.706878 13380 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
I20260709 13:21:02.708153 13384 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:21:02.710157 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:02.710633 20147 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:02.685966 20087 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:02.711241 20087 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:02.710310 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:02.713650 19907 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.714915 20132 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:02.704244 20191 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:02.715669 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:02.715881 20191 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:02.716341 20191 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:02.718997 13377 raft_consensus.cc:1240] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:02.685631 19941 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:02.722261 20268 log.cc:927] Time spent T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 13:21:02.723577 19941 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:02.724095 19941 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.727067 12779 raft_consensus.cc:1217] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Deduplicated request from leader. Original: 12.11->[13.12-13.12]   Dedup: 13.12->[]
I20260709 13:21:02.727569 12779 raft_consensus.cc:1240] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 13. Current term is 14. Ops: [13.12-13.12]
W20260709 13:21:02.728175 20243 consensus_meta.cc:287] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.527s	user 0.006s	sys 0.003s
I20260709 13:21:02.728418 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:02.728958 12632 raft_consensus.cc:2410] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:21:02.725505 19907 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:02.731761 20244 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:02.732050 20192 raft_consensus.cc:3055] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:02.731884 20229 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.732518 20192 raft_consensus.cc:740] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:02.733107 20172 consensus_queue.cc:237] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.733448 20192 consensus_queue.cc:260] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.733989 20206 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:02.734431 20206 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:02.734782 20206 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:02.736632 12629 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:02.683266 20212 consensus_queue.cc:1048] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:21:02.702463 19826 raft_consensus.cc:697] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:02.742481 19917 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:02.742836 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:02.694379 20233 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 13:21:02.748420 20189 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:02.721385 20252 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:02.749172 12631 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:02.753485 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:02.754952 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:02.764400 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:21:02.767108 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 62530.3us
I20260709 13:21:02.762377 12783 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 19.
I20260709 13:21:02.767853 12746 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:02.759506 12700 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:21:02.764981 12597 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:02.778683 12629 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:21:02.763700 13348 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:02.783362 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:02.784083 12634 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 21.
I20260709 13:21:02.749123 20109 raft_consensus.cc:3055] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:21:02.784072 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:02.790145 12626 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
I20260709 13:21:02.786051 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:02.791229 13379 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:02.791426 13377 raft_consensus.cc:1217] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Deduplicated request from leader. Original: 12.11->[13.12-13.12]   Dedup: 13.12->[]
I20260709 13:21:02.792016 13377 raft_consensus.cc:1240] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:21:02.785221 12629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 1843) took 1124 ms. Trace:
I20260709 13:21:02.786082 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:02.793804 12630 raft_consensus.cc:2410] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:21:02.785601 12700 raft_consensus.cc:1240] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:02.768349 20243 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.792708 12629 rpcz_store.cc:276] 0709 13:21:01.659049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:01.659337 (+   288us) service_pool.cc:224] Handling call
0709 13:21:01.659922 (+   585us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:02.600922 (+941000us) spinlock_profiling.cc:243] Waited 41.3 ms on lock 0x7b540023b2a0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d329dc0 00007f4c6d3b41c0 00007f4c6d3b5231 00007f4c6d3b1d8f 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c
0709 13:21:02.736366 (+135444us) spinlock_profiling.cc:243] Waited 64.7 ms on lock 0x7b58004271d0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d30e0aa 00007f4c6d390447 00007f4c6d3a8fe7 00007f4c6d3aa7de 00007f4c6d3b4b49 00007f4c6d3b5416 00007f4c6d3b1d8f 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4
0709 13:21:02.779271 (+ 42905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:02.782844 (+  3573us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":243903232}
W20260709 13:21:02.786159 19953 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:21:02.799906 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:02.779805 20233 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:02.822273 20233 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.805132 13348 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:02.824259 20273 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:02.824714 20273 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:02.694677 19880 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.825145 20273 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:02.794836 20212 consensus_queue.cc:1059] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:21:02.826627 20212 raft_consensus.cc:3055] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:02.827023 20212 raft_consensus.cc:740] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:02.827996 19880 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:02.749750 19941 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.827901 20212 consensus_queue.cc:260] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.829263 20212 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:02.799742 20092 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:02.789755 20109 raft_consensus.cc:740] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:02.810377 20275 consensus_queue.cc:1059] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:21:02.813059 20131 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.817487 20253 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:02.903551 20182 raft_consensus.cc:3055] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:21:02.788816 20185 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 13:21:02.903971 20182 raft_consensus.cc:740] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:02.805114 20132 consensus_queue.cc:1059] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:21:02.830328 19941 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:02.784457 20189 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:21:02.832113 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:02.905336 20132 raft_consensus.cc:3055] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:02.907194 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:02.907399 20132 raft_consensus.cc:740] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:02.908336 20132 consensus_queue.cc:260] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:02.908706 20087 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.832858 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:02.833976 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:02.835803 12706 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:21:02.835857 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:02.760226 20191 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.799530 19907 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.785835 20239 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:02.742509 19826 consensus_queue.cc:237] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.840886 20273 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.857455 20092 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:02.833279 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:02.803601 20252 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:21:02.842911 12517 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 612a4639a7a749be8856bbc60766e312 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER failed with leader 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:21:02.845077 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:02.845507 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:21:02.854835 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 57882.6us
W20260709 13:21:02.862059 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:02.801375 20243 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:02.865034 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 64500.3us
I20260709 13:21:02.870651 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:02.802690 20220 raft_consensus.cc:3055] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:02.882444 20229 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:02.872327 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:02.874279 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:02.875779 12633 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 13:21:02.878786 12707 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:21:02.881995 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:02.882645 12627 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:21:02.888545 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:02.890575 12778 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:21:02.892511 20147 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:21:02.895403 20180 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:02.897925 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:02.902135 20109 consensus_queue.cc:260] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:02.904835 20182 consensus_queue.cc:260] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:02.870196 20278 log.cc:927] Time spent T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:21:02.914677 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:02.917603 12777 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 21. Current term is 22. Ops: []
I20260709 13:21:02.918030 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:21:02.920243 12517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 77401.3us
I20260709 13:21:02.921240 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:02.930055 12746 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:21:02.979487 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 49516.1us
I20260709 13:21:02.909847 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:02.932101 12633 pending_rounds.cc:85] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 9
I20260709 13:21:02.980573 12633 pending_rounds.cc:107] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 13:21:02.980926 12633 raft_consensus.cc:2889] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:02.933807 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:02.981974 12627 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:21:02.929237 12598 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:02.929889 13385 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:21:02.927932 12775 raft_consensus.cc:1275] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:21:02.949150 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:02.934878 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:21:02.949952 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 66675.3us
I20260709 13:21:02.910476 12704 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:03.003638 13379 raft_consensus.cc:2468] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 22.
I20260709 13:21:02.979324 12628 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
W20260709 13:21:03.017551 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 98512.5us
W20260709 13:21:03.017998 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 88857.5us
I20260709 13:21:02.984715 12672 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:03.018798 13349 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:03.025385 12704 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:21:03.026283 12702 raft_consensus.cc:1240] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:21:03.028151 13377 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:21:03.028822 13349 leader_election.cc:400] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 17. Current term is 18.
W20260709 13:21:03.026993 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 92186.4us
I20260709 13:21:03.029888 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:03.031901 12528 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40528 (request call id 230) took 5569 ms. Trace:
I20260709 13:21:03.032263 12528 rpcz_store.cc:276] 0709 13:20:57.462276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:57.462509 (+   233us) service_pool.cc:224] Handling call
0709 13:21:03.031847 (+5569338us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:00.046960 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:00.047649 (+   689us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:00.047705 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:21:00.047726 (+    21us) tablet.cc:662] Decoding operations
0709 13:21:00.048901 (+  1175us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:00.049014 (+   113us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:00.049048 (+    34us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:21:00.186468 (+137420us) tablet.cc:701] Row locks acquired
0709 13:21:00.205852 (+ 19384us) write_op.cc:260] PREPARE: finished
0709 13:21:00.215018 (+  9166us) write_op.cc:270] Start()
0709 13:21:00.215193 (+   175us) write_op.cc:276] Timestamp: P: 1783603260214935 usec, L: 0
0709 13:21:00.215218 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:21:00.215982 (+   764us) log.cc:844] Serialized 4458 byte log entry
0709 13:21:01.112978 (+896996us) op_driver.cc:464] REPLICATION: finished
0709 13:21:01.278311 (+165333us) write_op.cc:301] APPLY: starting
0709 13:21:01.278421 (+   110us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:01.425111 (+146690us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:01.436623 (+ 11512us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:01.957161 (+520538us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:01.957416 (+   255us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=22,mrs_lookups=0
0709 13:21:01.957459 (+    43us) write_op.cc:312] APPLY: finished
0709 13:21:01.957815 (+   356us) log.cc:844] Serialized 105 byte log entry
0709 13:21:02.216956 (+259141us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:02.237682 (+ 20726us) write_op.cc:454] Released schema lock
0709 13:21:02.242217 (+  4535us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:02.242669 (+   452us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:02.251208 (+  8539us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":108,"reported_tablets":14,"child_traces":[["op",{"apply.queue_time_us":158239,"apply.run_cpu_time_us":92494,"apply.run_wall_time_us":964303,"cfile_cache_hit":46,"cfile_cache_hit_bytes":50151,"num_ops":11,"prepare.queue_time_us":306812,"prepare.run_cpu_time_us":15561,"prepare.run_wall_time_us":373101,"raft.queue_time_us":479170,"raft.run_cpu_time_us":16521,"raft.run_wall_time_us":171881,"replication_time_us":854214,"thread_start_us":1110357,"threads_started":5,"wal-append.queue_time_us":705528,"wal-append.run_cpu_time_us":27205,"wal-append.run_wall_time_us":1439210}]]}
I20260709 13:21:03.036983 12598 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:02.919113 20092 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:03.040385 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:03.041200 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:03.046810 13380 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:03.046373 13381 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:03.048167 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:03.048923 12702 raft_consensus.cc:3060] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:03.041392 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:03.043062 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:02.909530 20132 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:02.925366 20253 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:02.920020 20252 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:02.930759 20275 raft_consensus.cc:3055] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:02.923652 19958 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.121111 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:21:03.054509 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:03.037467 20242 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:02.949795 19953 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.913295 20191 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:03.055423 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 70837.2us
I20260709 13:21:03.057822 13381 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 24.
I20260709 13:21:03.059303 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:03.155763 12707 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:02.930033 20220 raft_consensus.cc:740] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:02.917785 20233 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:03.064937 12702 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
I20260709 13:21:02.919734 20186 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:03.066393 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:03.207531 12700 raft_consensus.cc:2393] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 16: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:03.066831 12597 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:03.042831 20190 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:21:03.074071 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:03.055402 20244 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:21:03.130586 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:03.044632 19889 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:02.913596 19907 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:03.141639 19958 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:03.142268 19972 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:21:03.143612 20168 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.148408 19917 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:03.036286 20116 consensus_queue.cc:1059] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.191s
I20260709 13:21:02.919262 20273 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:03.036972 19829 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:03.071183 12633 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 18. Current term is 19. Ops: []
W20260709 13:21:03.209972 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 143201us
I20260709 13:21:03.063244 20217 consensus_queue.cc:1059] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.182s
I20260709 13:21:03.054625 12704 raft_consensus.cc:2393] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 21: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:03.082715 20275 raft_consensus.cc:740] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:03.056031 20151 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:21:03.312266 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:03.236580 12627 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 24.
I20260709 13:21:03.063386 20238 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.187s
I20260709 13:21:03.243942 12707 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:21:03.085652 20252 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:03.042574 20221 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.185s
I20260709 13:21:03.055814 20179 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.186s
I20260709 13:21:03.296782 12708 raft_consensus.cc:1240] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:03.141387 13380 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
W20260709 13:21:03.304915 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:03.026991 19848 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.106s
I20260709 13:21:03.122624 20253 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:03.215427 20191 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:03.246019 20242 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:03.203713 19953 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:03.318634 20206 consensus_meta.cc:287] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.580s	user 0.011s	sys 0.002s
I20260709 13:21:03.306327 19829 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:03.261008 20147 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 13:21:03.313866 20151 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:03.302296 20168 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:03.326448 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 284477us
I20260709 13:21:03.313359 20098 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:21:03.328816 12627 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:21:03.326812 20242 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:03.321202 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 242431us
I20260709 13:21:03.336905 20063 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.337715 20257 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.339493 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:03.369890 12780 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:21:03.305508 19917 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:03.327142 20087 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:03.324314 20229 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:03.348186 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:03.348743 20116 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:03.351014 19907 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:03.353339 20092 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:03.356206 19889 raft_consensus.cc:3055] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:03.324290 20220 consensus_queue.cc:260] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:03.358011 20180 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:03.330055 20206 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:03.390215 19826 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:03.368137 13349 leader_election.cc:400] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 17. Current term is 18.
I20260709 13:21:03.332553 19829 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:03.371982 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 159692us
I20260709 13:21:03.335579 20151 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:03.342480 20243 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:03.412690 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 87456.3us
I20260709 13:21:03.435302 20131 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:03.491036 12597 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:21:03.492189 12597 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:03.490633 12672 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:03.327787 20191 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:03.371500 20063 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:03.523350 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:03.415990 13380 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:21:03.416452 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:03.388718 20257 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:03.524071 12746 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:03.358660 20242 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:03.451668 13378 raft_consensus.cc:2393] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 22: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:21:03.412079 19889 raft_consensus.cc:740] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:21:03.450090 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 82077.6us
I20260709 13:21:03.495214 20168 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:03.544922 19889 consensus_queue.cc:260] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:03.546097 19889 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:03.497692 20297 consensus_queue.cc:1048] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:21:03.489100 19826 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:03.548274 20192 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:03.506336 20289 log.cc:927] Time spent T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:21:03.451066 20220 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:03.564483 12531 catalog_manager.cc:5697] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
W20260709 13:21:03.517500 20281 log.cc:927] Time spent T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 13:21:03.413763 20229 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:03.519075 20206 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:03.519531 20186 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:03.521035 20253 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:03.519984 20219 raft_consensus.cc:2955] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Committing config change with OpId 14.11: config changed from index 10 to 11, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:03.525194 13383 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:21:03.331398 20105 consensus_queue.cc:1059] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 13:21:03.527248 12672 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:03.528545 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:03.529074 20243 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:03.529804 20191 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:03.530077 20185 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.624212 20185 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:03.626734 20185 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:03.628078 20092 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:03.629032 12531 catalog_manager.cc:5697] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 21 to 22, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 22 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:03.629961 20105 raft_consensus.cc:3055] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:03.630398 20105 raft_consensus.cc:740] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:03.631193 20105 consensus_queue.cc:260] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:03.631352 12531 catalog_manager.cc:5697] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 12 to 13, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 13 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:03.632547 20105 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:03.636871 20063 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:03.618607 20220 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.640369 20244 raft_consensus.cc:3055] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:03.640323 20220 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:03.640836 20244 raft_consensus.cc:740] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:03.530256 19953 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:03.601270 20233 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:03.641901 20233 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:03.644037 20233 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:03.533430 12746 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:21:03.646240 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 114226us
I20260709 13:21:03.646675 20229 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:03.647223 20229 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:21:03.541227 20293 raft_consensus.cc:2804] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:21:03.545931 20179 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:21:03.647786 20229 raft_consensus.cc:697] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 24 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:03.647996 20293 raft_consensus.cc:697] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 22 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:03.648876 20293 consensus_queue.cc:237] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:03.497378 20217 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:03.649904 20217 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:03.448493 20221 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.570072 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:03.648782 20229 consensus_queue.cc:237] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 22.20, Last appended by leader: 20, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:03.572176 20275 consensus_queue.cc:260] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:03.545280 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:21:03.662721 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 117509us
I20260709 13:21:03.664327 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:03.664891 13379 raft_consensus.cc:2376] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16. Re-sending same reply.
I20260709 13:21:03.574487 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:03.575523 19829 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:03.577749 19958 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:03.578311 19826 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:03.579703 20257 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:03.501477 20212 consensus_meta.cc:287] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.637s	user 0.004s	sys 0.004s
I20260709 13:21:03.673668 20275 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:03.583612 19848 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:21:03.609750 12774 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:21:03.675288 20308 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.675611 12774 raft_consensus.cc:606] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:21:03.676079 12774 raft_consensus.cc:614] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:03.675863 20308 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:03.534638 20169 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:03.621069 13384 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 22.
W20260709 13:21:03.623518 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 96373.6us
I20260709 13:21:03.582827 20081 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.672391 20116 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:03.677845 20081 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:03.678066 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:03.679709 13379 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:21:03.679841 20081 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:03.542586 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:03.680708 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:03.666518 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:03.682700 13377 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
W20260709 13:21:03.683290 20306 log.cc:927] Time spent T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:21:03.683423 12708 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:55238
I20260709 13:21:03.684054 12708 raft_consensus.cc:606] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:21:03.684437 13384 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:03.684513 12708 raft_consensus.cc:614] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:03.667322 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:03.669312 13349 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:03.692488 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:03.694577 13384 raft_consensus.cc:2410] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
W20260709 13:21:03.661224 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 227961us
I20260709 13:21:03.662120 20147 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:03.678135 20116 raft_consensus.cc:697] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:03.569425 12628 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 13:21:03.708158 12628 pending_rounds.cc:85] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 15
I20260709 13:21:03.708583 12628 pending_rounds.cc:107] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 21.16
W20260709 13:21:03.708839 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:03.708940 12628 raft_consensus.cc:2889] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:03.693513 12633 raft_consensus.cc:2410] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:21:03.633447 12531 catalog_manager.cc:5697] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 15 to 16, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 16 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:03.699196 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:03.701942 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:03.647152 12746 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:21:03.710696 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 63610.4us
I20260709 13:21:03.655617 20217 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:03.702986 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:03.705432 12672 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:03.695845 12625 raft_consensus.cc:2955] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 10 to 11, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:03.718230 12672 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:03.721191 12531 catalog_manager.cc:5697] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 20 to 21, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 21 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:03.723445 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:03.724414 12777 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 22.
I20260709 13:21:03.704838 20179 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:03.726408 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:03.725646 13382 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 11.
I20260709 13:21:03.720909 13349 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:03.718842 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:03.729373 12531 catalog_manager.cc:5697] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 23 to 24, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 24 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:03.730577 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:03.734043 13349 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:03.716845 19826 raft_consensus.cc:2804] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:03.737540 12597 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:21:03.738504 12598 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:03.741915 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:03.742132 20206 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:03.743472 12782 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:21:03.735044 12531 catalog_manager.cc:5697] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:03.719914 20081 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:21:03.719473 19958 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:03.708106 20186 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:21:03.750185 12777 raft_consensus.cc:1240] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:21:03.747881 12531 catalog_manager.cc:5697] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 19 to 20, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 20 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:03.674067 20244 consensus_queue.cc:260] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:03.750618 12598 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:03.670882 20221 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:03.747076 12782 raft_consensus.cc:606] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 25 LEADER]: Received request to transfer leadership to TS 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:03.752192 13349 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:03.707962 20169 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:21:03.757695 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:03.761926 12780 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:21:03.731873 20098 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:21:03.744969 20190 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:03.732079 20147 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.765590 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:21:03.730629 20179 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:03.767369 12598 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:03.768797 12633 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:21:03.750873 20238 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:03.732481 20273 consensus_queue.cc:1048] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 13:21:03.727542 19848 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
I20260709 13:21:03.771999 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:03.769210 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:03.777789 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:03.758713 20092 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:03.758380 20233 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:03.758342 19826 raft_consensus.cc:697] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:03.744585 19907 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:21:03.748394 20300 consensus_queue.cc:1059] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.034s
I20260709 13:21:03.783767 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:03.874809 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 91135.4us
I20260709 13:21:03.877934 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:03.879958 13384 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:21:03.882490 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:03.775695 19889 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:03.774950 20238 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:03.892958 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:03.858525 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:03.775624 20244 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:03.864555 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 108540us
I20260709 13:21:03.878710 12776 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:21:03.893682 12704 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
W20260709 13:21:03.850618 20311 log.cc:927] Time spent T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:21:03.887499 13383 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:21:03.892486 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:21:03.857342 20252 consensus_meta.cc:287] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.532s	user 0.005s	sys 0.005s
I20260709 13:21:03.895536 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:03.905687 12708 raft_consensus.cc:1240] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 24. Current term is 25. Ops: []
I20260709 13:21:03.905515 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:03.907605 12747 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:03.774102 19953 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:03.919296 12778 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:21:03.770085 20308 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:03.850612 20239 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.941411 13381 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 21. Current term is 22. Ops: []
I20260709 13:21:03.873881 20220 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:03.789491 12777 raft_consensus.cc:2955] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 10 to 11, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:03.772904 20179 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:21:03.932010 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 43573.7us
W20260709 13:21:03.949955 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:03.884750 19889 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:03.933655 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:03.930554 12746 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:03.916553 20238 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:03.923686 20189 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:03.884855 20190 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:03.771802 20116 consensus_queue.cc:237] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:03.845460 20221 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:03.774593 20147 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:03.950179 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:03.780989 20243 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:03.895948 20109 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:03.927052 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:03.904173 20172 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.926019 19953 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:03.939949 20253 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:03.935175 20252 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:03.968747 20314 log.cc:927] Time spent T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 13:21:03.976327 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:03.988358 12707 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:21:03.981873 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:03.957943 19889 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:03.952579 20180 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.991717 12628 raft_consensus.cc:2410] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:21:03.991791 12781 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:21:03.965014 20254 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.005470 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:03.943389 20292 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 13:21:03.945940 20191 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:21:03.951586 20239 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.013302 12774 raft_consensus.cc:2368] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12.
I20260709 13:21:03.985929 19953 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:21:04.018729 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 88257.1us
I20260709 13:21:04.025018 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:04.005591 20180 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.037096 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:04.039883 20182 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:03.972863 19826 consensus_queue.cc:237] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.17, Last appended by leader: 17, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.043265 20180 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:04.045984 20147 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:04.055102 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 98977.8us
I20260709 13:21:04.017603 19907 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:04.070873 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:04.072398 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:04.074700 12701 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:21:04.077445 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:04.082852 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:04.091226 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:04.092734 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:04.094395 12707 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:21:04.009465 20254 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.091863 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:04.006368 20172 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.095898 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:21:04.064581 20309 log.cc:927] Time spent T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.316s	user 0.000s	sys 0.001s
W20260709 13:21:04.079842 20314 log.cc:927] Time spent T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:21:04.111342 20253 raft_consensus.cc:697] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:04.121431 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:04.121511 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:04.122301 12709 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:21:04.122437 12706 raft_consensus.cc:2368] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 19. Current term is 20.
I20260709 13:21:04.078907 20180 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.125808 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:04.065444 19907 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:04.127267 12707 raft_consensus.cc:1240] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 16. Current term is 17. Ops: []
W20260709 13:21:04.122864 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.131630 12700 raft_consensus.cc:1240] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:21:04.133302 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:04.135118 13383 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
W20260709 13:21:04.119164 20317 log.cc:927] Time spent T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:21:04.141117 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:04.141726 19970 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:04.142802 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:04.143685 12709 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 20: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:04.144419 12672 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:04.146921 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:04.147316 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:04.147454 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:04.149621 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:04.154511 12780 raft_consensus.cc:2393] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 22: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
W20260709 13:21:04.155130 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.149646 19880 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:04.157622 20132 raft_consensus.cc:993] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2: : Instructing follower 639ab2db265f4747bb7afb93e833f530 to start an election
I20260709 13:21:04.158027 19825 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:21:04.160562 19889 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.161727 13385 raft_consensus.cc:2368] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16.
I20260709 13:21:04.137066 20180 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.164149 20180 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:04.164721 13377 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:21:04.167203 19889 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:04.174203 20315 log.cc:927] Time spent T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:21:04.180213 20189 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:04.132325 19907 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:04.146943 12672 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:04.151687 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:04.193648 12776 raft_consensus.cc:2393] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 22: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
W20260709 13:21:04.138908 20192 consensus_meta.cc:287] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.590s	user 0.003s	sys 0.008s
W20260709 13:21:04.202196 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 55320.3us
I20260709 13:21:04.203298 12672 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:04.136300 20098 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.136070 19902 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.206965 12672 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:04.208578 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:04.211112 13381 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 21.
I20260709 13:21:04.209177 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:04.212282 13378 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:21:04.212193 12778 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:21:04.217626 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:04.220391 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:04.221362 13382 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 21.
I20260709 13:21:04.142241 19970 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.227561 13382 raft_consensus.cc:1240] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:04.147961 20319 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.236971 12776 raft_consensus.cc:1240] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 22 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 13:21:04.254937 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:04.259342 13381 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:21:04.220319 20081 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.220130 20169 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:04.192019 20272 consensus_queue.cc:1059] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 13:21:04.149334 19818 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.091s
W20260709 13:21:04.240542 20151 consensus_meta.cc:287] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.727s	user 0.007s	sys 0.004s
I20260709 13:21:04.264240 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:04.266152 12746 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:04.268806 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:04.271276 12782 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 20.
I20260709 13:21:04.284600 13381 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:21:04.287148 13379 raft_consensus.cc:2955] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 12 to 13, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:04.233830 19902 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.225348 20098 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.336159 20169 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:04.352281 12779 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 21. Current term is 22. Ops: []
I20260709 13:21:04.359424 19970 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:04.363600 19953 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.293692 20252 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:04.244771 20319 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.367215 20172 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:04.366396 19902 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:04.393735 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.370448 20087 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:21:04.384219 20217 consensus_meta.cc:287] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.661s	user 0.012s	sys 0.000s
I20260709 13:21:04.281801 20081 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.285981 20253 consensus_queue.cc:237] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.382611 20243 raft_consensus.cc:697] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:04.392488 20105 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.392030 20182 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:04.293430 20151 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:04.420179 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 213289us
I20260709 13:21:04.400964 20212 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:04.384493 19907 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.413996 20239 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:04.430997 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.433621 20340 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:21:04.435863 20340 consensus_queue.cc:1059] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:04.445353 12776 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:21:04.410616 20217 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:04.419267 20105 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:04.466535 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.423162 20182 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.470475 12598 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:04.429394 20172 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:21:04.430423 20212 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.482649 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:04.467957 20221 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:04.485527 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:04.485594 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:04.470353 20239 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:04.491104 12628 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
W20260709 13:21:04.495536 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:04.482620 20242 consensus_meta.cc:287] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.908s	user 0.010s	sys 0.002s
I20260709 13:21:04.497257 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:04.497886 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:04.498894 12630 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 20: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:21:04.484318 20185 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:04.503116 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:04.504977 12630 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:04.504714 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:04.511500 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:04.503938 20098 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:04.520399 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:04.521700 12628 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:21:04.500030 20327 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.530230 20343 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:04.530584 19953 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:04.531916 20217 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:04.496505 20221 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.542949 20105 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:04.543481 20081 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:04.537489 20221 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:04.544806 20319 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:04.506521 20185 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:04.499193 20239 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:04.564975 20109 consensus_meta.cc:287] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.574s	user 0.010s	sys 0.001s
I20260709 13:21:04.565348 20239 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:04.566300 20337 log.cc:927] Time spent T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.168s	user 0.001s	sys 0.001s
I20260709 13:21:04.567566 20185 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:04.485729 20243 consensus_queue.cc:237] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:04.503556 20242 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.570525 12632 raft_consensus.cc:2368] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16.
I20260709 13:21:04.572788 12627 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 25.
I20260709 13:21:04.575991 12673 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:04.530504 19907 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:04.526824 20340 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:21:04.573200 20179 consensus_meta.cc:287] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.612s	user 0.011s	sys 0.001s
I20260709 13:21:04.599617 12630 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:21:04.576258 20116 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:04.602366 12517 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 612a4639a7a749be8856bbc60766e312 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER failed with leader 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:21:04.604718 20238 consensus_meta.cc:287] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.627s	user 0.011s	sys 0.001s
I20260709 13:21:04.605180 20238 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.606393 12633 raft_consensus.cc:2368] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 22. Current term is 23.
I20260709 13:21:04.606793 12781 raft_consensus.cc:606] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Received request to transfer leadership to TS 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:04.607333 20238 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:04.607573 12774 raft_consensus.cc:1240] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:21:04.530823 20343 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.602006 20340 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.610090 20298 raft_consensus.cc:3055] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:21:04.610555 20298 raft_consensus.cc:740] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:04.611436 20298 consensus_queue.cc:260] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.612545 20298 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:04.608174 20116 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.615640 20116 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:04.608052 20339 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:04.616942 20339 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:04.617301 20339 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:04.601743 19880 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:04.624053 12630 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:21:04.624442 12630 pending_rounds.cc:85] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 12
I20260709 13:21:04.624640 12630 pending_rounds.cc:107] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:21:04.624848 12630 raft_consensus.cc:2889] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:21:04.627698 20244 consensus_meta.cc:287] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.704s	user 0.007s	sys 0.004s
I20260709 13:21:04.628398 19829 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:04.631004 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:04.632103 12630 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:21:04.630972 12597 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:04.634032 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:04.632907 13349 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:04.635213 12630 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 20: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:21:04.636243 12597 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:04.638015 13385 consensus_queue.cc:237] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.639082 13349 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:04.552138 20327 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.640323 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:04.608659 20151 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:04.641503 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.644999 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:04.645637 12630 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:21:04.646767 12630 raft_consensus.cc:1240] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:04.646643 13349 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:04.601863 20179 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.632380 20244 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.649686 12597 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:04.649734 19842 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:04.640611 20352 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:04.652726 20352 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:04.635246 20220 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:04.653163 20352 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:04.642634 20116 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:21:04.653398 20220 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.655304 19917 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:04.656111 20343 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:04.656464 20169 raft_consensus.cc:3055] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:21:04.656884 20169 raft_consensus.cc:740] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:04.656771 19902 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:04.657898 20244 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.659296 20340 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:04.659904 20244 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:04.661281 20327 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:04.662714 20219 consensus_queue.cc:1059] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:21:04.663615 20219 raft_consensus.cc:3055] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:04.663849 20244 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:04.663921 20219 raft_consensus.cc:740] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:04.664821 20219 consensus_queue.cc:260] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:04.666561 20254 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:04.668195 20219 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:04.668869 20244 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.637431 20242 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:04.669360 20244 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:04.671381 20179 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:04.669365 20185 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:04.673130 12625 raft_consensus.cc:1240] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:21:04.675082 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:21:04.675365 20190 consensus_meta.cc:287] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.698s	user 0.009s	sys 0.001s
I20260709 13:21:04.675871 20190 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:04.676182 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:04.641660 20109 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:21:04.677830 20109 raft_consensus.cc:697] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:04.678332 20092 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:21:04.678750 20109 consensus_queue.cc:237] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:04.679636 20336 log.cc:927] Time spent T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.215s	user 0.002s	sys 0.000s
I20260709 13:21:04.680039 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:04.677107 20212 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:04.677289 12672 leader_election.cc:400] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16.
I20260709 13:21:04.680315 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:04.680572 13381 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:04.685883 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:04.687237 13349 leader_election.cc:400] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 19. Current term is 20.
I20260709 13:21:04.695080 13377 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:21:04.695307 13349 leader_election.cc:403] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:21:04.681250 12672 leader_election.cc:403] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:21:04.693252 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:04.692975 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:04.703637 13384 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
W20260709 13:21:04.696192 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 45727.8us
I20260709 13:21:04.704995 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:04.705892 12707 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:21:04.706712 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:04.707182 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:04.707337 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:04.708235 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:04.708671 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:04.681074 12625 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
I20260709 13:21:04.708925 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:04.709303 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:04.691687 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:04.681066 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:04.709959 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.709398 12709 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 19.
I20260709 13:21:04.711366 12705 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:21:04.710121 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:04.691144 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:04.707842 12707 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 23.
I20260709 13:21:04.712324 13380 raft_consensus.cc:2393] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 16: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
W20260709 13:21:04.682140 20353 log.cc:927] Time spent T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.075s	user 0.002s	sys 0.000s
I20260709 13:21:04.712800 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:04.710762 12673 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:04.713269 12597 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:04.715341 13383 raft_consensus.cc:2393] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 14: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:21:04.716698 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:04.682832 20185 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:04.671424 20244 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:04.718729 13381 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 20.
I20260709 13:21:04.727191 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:04.693342 20087 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.683230 20212 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:04.683360 20221 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.705787 20190 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 19. Current term is 20.
I20260709 13:21:04.731155 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:21:04.775105 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 45375.8us
I20260709 13:21:04.776705 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:04.779196 13383 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:21:04.776755 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:04.778249 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:04.752583 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:04.764411 12598 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:04.787467 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:04.790171 13378 raft_consensus.cc:2410] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 25 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 13:21:04.790983 13381 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:21:04.787074 12598 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:04.792960 12598 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:04.780557 13376 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
I20260709 13:21:04.728698 20238 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:04.726128 20172 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:04.729940 20340 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:04.711977 20219 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:04.747390 12701 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:21:04.734460 20343 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:21:04.801709 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 69166.1us
I20260709 13:21:04.802314 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:04.802814 12672 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:04.774060 20338 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:04.801328 13349 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:04.801525 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:04.814239 13381 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 20.
I20260709 13:21:04.812706 12634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 4757) took 1122 ms. Trace:
I20260709 13:21:04.784752 20297 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:04.796027 13376 raft_consensus.cc:2955] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 10 to 11, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:04.657904 20169 consensus_queue.cc:260] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.802268 20238 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:04.817481 12598 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:04.807178 20340 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:04.820968 13377 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 23.
I20260709 13:21:04.789774 20179 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:04.799851 20185 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:21:04.814455 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.803099 20302 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:04.800823 20244 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:04.850870 20244 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:04.851279 20244 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:04.821866 19818 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:04.818342 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:04.752041 20087 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:04.822439 12634 rpcz_store.cc:276] 0709 13:21:03.690382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:03.696670 (+  6288us) service_pool.cc:224] Handling call
0709 13:21:03.697018 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:03.780836 (+ 83818us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:03.780867 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:03.843930 (+ 63063us) log.cc:844] Serialized 26 byte log entry
0709 13:21:03.951506 (+107576us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:03.951546 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:04.806278 (+854732us) spinlock_profiling.cc:243] Waited 83.4 ms on lock 0x7b5400408b54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:21:04.806345 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:04.806407 (+    62us) raft_consensus.cc:1685] finished
0709 13:21:04.806422 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:04.812634 (+  6212us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":287,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":128,"spinlock_wait_cycles":191912832,"thread_start_us":107102,"threads_started":1,"wal-append.queue_time_us":164921}
I20260709 13:21:04.834323 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:04.875233 13381 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 21.
I20260709 13:21:04.775784 20212 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:04.835933 12597 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:04.774121 20221 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.840085 20185 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:04.850245 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:04.882591 12628 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:04.816645 20351 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:04.866703 20169 raft_consensus.cc:3060] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:04.868600 12701 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 18.
I20260709 13:21:04.833858 20190 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:04.834700 20238 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:04.839710 20340 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:04.838346 20179 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:04.847411 20242 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:04.833704 20343 raft_consensus.cc:697] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:04.814455 20172 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:04.874186 20272 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.829089 20297 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:04.914738 12630 raft_consensus.cc:1275] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:21:04.825393 20338 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.883528 20239 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:21:04.904428 20190 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:04.889482 20185 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:04.899353 19818 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:04.905756 20298 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.930819 12628 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:21:04.918344 20179 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:04.926630 20242 raft_consensus.cc:697] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:04.936111 19818 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:04.928274 20272 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:04.978835 19818 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:04.979457 20272 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:04.951589 20343 consensus_queue.cc:237] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.930757 20190 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:04.982002 20352 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.983139 19826 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:04.983662 19826 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:04.985344 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:04.986019 20132 raft_consensus.cc:1081] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 25 LEADER]: Signalling peer 639ab2db265f4747bb7afb93e833f530 to start an election
I20260709 13:21:04.986740 12627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56832 (request call id 4750) took 1412 ms. Trace:
I20260709 13:21:04.987191 20352 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:04.987116 12627 rpcz_store.cc:276] 0709 13:21:03.574046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:03.574184 (+   138us) service_pool.cc:224] Handling call
0709 13:21:04.986593 (+1412409us) spinlock_profiling.cc:243] Waited 902 ms on lock 0x7b54002eb8d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:21:04.986685 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2075088256}
I20260709 13:21:04.988445 20244 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:04.990815 20293 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:04.991317 20293 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:04.992466 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 726405us
I20260709 13:21:04.993587 20324 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:21:04.939592 20298 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:04.997728 20298 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:04.998761 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.999863 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:05.000876 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:05.000496 13384 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:05.001731 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:05.002260 13377 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:05.002418 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:05.003180 13385 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:04.932814 20239 raft_consensus.cc:697] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 23 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:05.007126 12672 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:05.008512 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:05.009203 12709 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 21.
I20260709 13:21:04.997143 12597 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:04.956106 20172 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:05.012281 12597 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:04.957441 20179 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.962942 20221 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:04.964241 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:04.964775 20219 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16.
I20260709 13:21:04.966709 20087 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:04.973511 20334 log.cc:927] Time spent T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.476s	user 0.000s	sys 0.001s
I20260709 13:21:04.975369 20339 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:04.976126 12633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39590 (request call id 1861) took 1198 ms. Trace:
I20260709 13:21:05.009598 13377 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
I20260709 13:21:05.010280 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:05.014569 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:05.079612 12774 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:21:05.014775 20063 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:05.015408 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:05.082491 12701 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:21:05.015756 20105 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:05.018126 20242 consensus_queue.cc:237] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:05.029021 12597 leader_election.cc:400] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 11 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12.
W20260709 13:21:05.090898 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 77271.4us
I20260709 13:21:05.039263 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:05.093925 12781 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:05.028788 13384 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 12.
W20260709 13:21:05.022567 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 172148us
W20260709 13:21:05.069238 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 194941us
W20260709 13:21:05.096681 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:05.098305 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:05.037423 20252 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:05.042047 20169 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:05.075479 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:05.110381 12778 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 22.
I20260709 13:21:05.077205 12633 rpcz_store.cc:276] 0709 13:21:03.777325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:03.777492 (+   167us) service_pool.cc:224] Handling call
0709 13:21:04.975888 (+1198396us) spinlock_profiling.cc:243] Waited 826 ms on lock 0x7b54000de154. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:21:04.976048 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1899758848}
I20260709 13:21:05.096292 13349 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:05.112663 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:05.081478 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:05.113330 12774 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:05.021420 20302 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.085827 12598 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:05.114329 13349 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:05.115043 12598 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:05.037694 20295 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:05.100409 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:05.101720 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:05.000027 19826 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:05.108783 13385 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:21:05.117164 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:05.120925 12701 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 21.
I20260709 13:21:05.121124 13349 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:05.118323 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:05.119410 12700 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:21:05.119597 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:05.081977 20293 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:05.117029 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:05.125098 12626 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 20.
I20260709 13:21:05.126780 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:05.128857 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:05.130259 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:05.087798 20105 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:05.102337 20098 raft_consensus.cc:3055] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:05.132084 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:05.150600 12775 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:21:05.150774 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:05.133625 13384 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:21:05.151422 12778 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 21.
I20260709 13:21:05.148408 12781 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 20.
I20260709 13:21:05.158825 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:05.118835 20295 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:05.161517 12672 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:05.121738 20352 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:05.167161 12704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 3554) took 1239 ms. Trace:
I20260709 13:21:05.130460 19842 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:05.130664 19917 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:05.088732 20063 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:05.149391 20105 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:05.118155 20252 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:05.150144 20098 raft_consensus.cc:740] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:05.167963 12704 rpcz_store.cc:276] 0709 13:21:03.921613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:03.921723 (+   110us) service_pool.cc:224] Handling call
0709 13:21:03.922061 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:04.141271 (+219210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:04.141301 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:04.142753 (+  1452us) log.cc:844] Serialized 26 byte log entry
0709 13:21:04.383290 (+240537us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:21:04.488843 (+105553us) log.cc:844] Serialized 14 byte log entry
0709 13:21:04.489517 (+   674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:04.660417 (+170900us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b180041dfa0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2932a6 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419
0709 13:21:04.662046 (+  1629us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b5400249ad4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:21:04.662103 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:04.972932 (+310829us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:21:05.160321 (+187389us) raft_consensus.cc:1685] finished
0709 13:21:05.160344 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:05.160473 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5015,"prepare.queue_time_us":148537,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":6308,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":5138,"spinlock_wait_cycles":508428928,"thread_start_us":277886,"threads_started":2,"wal-append.queue_time_us":279398}
I20260709 13:21:05.159610 20116 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:05.166491 20295 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:05.122234 20302 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:05.185160 20302 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:05.171500 20352 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:05.185843 20352 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:21:05.186826 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:05.170904 20244 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:05.173398 19842 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:05.171821 20109 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:21:05.182822 12673 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:05.203087 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:05.208653 12631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 1862) took 1258 ms. Trace:
I20260709 13:21:05.174216 20365 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:05.179847 19970 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:05.219333 19902 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:21:05.209093 12631 rpcz_store.cc:276] 0709 13:21:03.950642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:03.950898 (+   256us) service_pool.cc:224] Handling call
0709 13:21:03.975036 (+ 24138us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:04.379550 (+404514us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:04.379597 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:04.393895 (+ 14298us) log.cc:844] Serialized 26 byte log entry
0709 13:21:04.829695 (+435800us) spinlock_profiling.cc:243] Waited 4.44 ms on lock 0x7b1800403f00. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf520d1 00007f4c6d36da0f
0709 13:21:04.899684 (+ 69989us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:05.120672 (+220988us) log.cc:844] Serialized 14 byte log entry
0709 13:21:05.202331 (+ 81659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:05.202423 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:05.202473 (+    50us) raft_consensus.cc:1685] finished
0709 13:21:05.208461 (+  5988us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:05.208590 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":397515,"prepare.queue_time_us":255679,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":460147,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":8712,"spinlock_wait_cycles":10213120,"thread_start_us":371542,"threads_started":3,"wal-append.queue_time_us":531666}
W20260709 13:21:05.247231 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 132280us
I20260709 13:21:05.226995 20063 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:05.228448 20098 consensus_queue.cc:260] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:05.206058 19842 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:05.131170 20239 consensus_queue.cc:237] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:05.240413 20252 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:05.240839 12704 raft_consensus.cc:1275] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:21:05.246852 20334 log.cc:927] Time spent T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:21:05.220120 20298 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:05.247287 20365 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:05.248808 19902 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:05.249720 12774 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 23.
W20260709 13:21:05.253414 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 120413us
I20260709 13:21:05.254165 12598 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:05.208397 12673 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:21:05.284714 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 76394.8us
I20260709 13:21:05.288259 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:05.289129 12629 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:05.283941 13349 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:05.290208 12598 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:05.256788 20190 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:05.259852 20298 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:21:05.281822 20318 log.cc:927] Time spent T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 1.190s	user 0.006s	sys 0.000s
I20260709 13:21:05.305259 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:05.307992 12704 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:05.294274 19970 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:05.309059 19970 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:05.312529 19970 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:05.318382 12704 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:21:05.275131 20365 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:05.281582 19902 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:05.281833 20098 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:05.321885 12598 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:05.282454 20179 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:05.325718 20339 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:05.325795 20343 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:05.293524 20169 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:05.331671 12629 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:21:05.306522 20244 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:21:05.335131 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 51264.8us
I20260709 13:21:05.318863 20298 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:05.338374 13349 leader_election.cc:400] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 22. Current term is 23.
W20260709 13:21:05.342175 20383 log.cc:927] Time spent T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:21:05.332389 20343 raft_consensus.cc:697] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:05.298924 20376 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:05.336822 20169 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:05.357064 13349 leader_election.cc:403] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:21:05.357815 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 1.20579e+06us
I20260709 13:21:05.298884 20238 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:05.337917 20244 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:21:05.389134 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 102658us
I20260709 13:21:05.390407 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:05.391582 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 53411.1us
I20260709 13:21:05.367415 20179 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:05.396180 12625 raft_consensus.cc:1240] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:21:05.399183 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:05.400002 12782 raft_consensus.cc:1240] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:21:05.396965 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:05.384087 20323 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:05.403961 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:05.402465 12779 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 20.
I20260709 13:21:05.404959 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:21:05.405877 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 246767us
I20260709 13:21:05.407833 12629 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:05.407966 12707 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:05.365787 20376 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:05.392932 20244 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:05.406644 12777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 4271) took 1633 ms. Trace:
I20260709 13:21:05.414541 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:05.414769 12777 rpcz_store.cc:276] 0709 13:21:03.773129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:03.788517 (+ 15388us) service_pool.cc:224] Handling call
0709 13:21:03.788998 (+   481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:03.789312 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:21:04.371478 (+582166us) log.cc:844] Serialized 14 byte log entry
0709 13:21:05.204439 (+832961us) log.cc:844] Serialized 14 byte log entry
0709 13:21:05.406376 (+201937us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:05.406419 (+    43us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:05.406458 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:05.406507 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:05.406601 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":994731,"prepare.queue_time_us":184916,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":704275,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":3093,"thread_start_us":327855,"threads_started":2,"wal-append.queue_time_us":279931}
I20260709 13:21:05.390559 20169 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:05.393796 20385 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 22. Current term is 23.
W20260709 13:21:05.488549 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:05.464347 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 76251.2us
I20260709 13:21:05.386471 20343 consensus_queue.cc:237] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:05.493129 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:05.414465 20376 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:05.494010 12782 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 21.
I20260709 13:21:05.498136 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:05.408842 20300 consensus_queue.cc:1059] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.080s
I20260709 13:21:05.472437 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:05.504215 12777 raft_consensus.cc:2393] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 17: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:21:05.383890 20151 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.019s
I20260709 13:21:05.474718 12783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 4268) took 1727 ms. Trace:
W20260709 13:21:05.474924 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 66508us
I20260709 13:21:05.393251 20185 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:05.483825 12707 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 17.
I20260709 13:21:05.390966 20105 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:05.415059 20323 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:05.476670 19907 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:05.413986 20192 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.487381 12781 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:21:05.489207 12629 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 17.
I20260709 13:21:05.497092 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:05.407369 20352 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:05.550232 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:05.550459 12781 raft_consensus.cc:606] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:21:05.490252 20179 raft_consensus.cc:697] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:05.506414 12746 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:05.551960 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 45638.8us
I20260709 13:21:05.552942 12746 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:05.555438 12629 raft_consensus.cc:2368] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 23. Current term is 24.
I20260709 13:21:05.510622 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:05.556210 20151 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:05.493526 20220 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:05.556802 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:05.556898 20352 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:05.557571 12626 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
I20260709 13:21:05.558341 20323 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:05.558995 20179 consensus_queue.cc:237] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:05.529006 20190 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:05.486194 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 126612us
I20260709 13:21:05.523301 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:21:05.495980 20380 log.cc:927] Time spent T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:21:05.504770 12783 rpcz_store.cc:276] 0709 13:21:03.747623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:03.748062 (+   439us) service_pool.cc:224] Handling call
0709 13:21:03.763640 (+ 15578us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:03.775892 (+ 12252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:03.775927 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:03.851780 (+ 75853us) log.cc:844] Serialized 52 byte log entry
0709 13:21:03.969328 (+117548us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:04.135799 (+166471us) log.cc:844] Serialized 14 byte log entry
0709 13:21:04.137033 (+  1234us) log.cc:844] Serialized 14 byte log entry
0709 13:21:04.138750 (+  1717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:04.138832 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:04.682867 (+544035us) spinlock_profiling.cc:243] Waited 39.5 ms on lock 0x7b18003d14a0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2932a6 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c722dbaa1 00007f4c722d2b37 00007f4c722d29ca 00007f4c723ca26f
0709 13:21:04.684361 (+  1494us) spinlock_profiling.cc:243] Waited 7.57 ms on lock 0x7b4800042258. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c722dbaa1 00007f4c722d2b37 00007f4c722d29ca 00007f4c723ca26f 00007f4c723c9f37 00007f4c723de55c 00007f4c723de508 00007f4c723de488 00007f4c723de443 00007f4c723dd6ec 00007f4c71307a96
0709 13:21:04.982951 (+298590us) spinlock_profiling.cc:243] Waited 3.74 ms on lock 0x7b18003d14a0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a23fc0a 00007f4c7397f219 00007f4c7128658f 00007f4c6a2c5571 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd
0709 13:21:05.255393 (+272442us) spinlock_profiling.cc:243] Waited 30.1 ms on lock 0x7b24001f9f88. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a14b9c4 00007f4c6a149acb 00007f4c6a304dc9 00007f4c6a304a55 00007f4c6a30409d 00007f4c6cf3610e 00007f4c6cf4d88c 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221
0709 13:21:05.474347 (+218954us) raft_consensus.cc:1685] finished
0709 13:21:05.474372 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:05.474504 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":735911,"prepare.queue_time_us":463435,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":5840,"raft.queue_time_us":11953,"raft.run_cpu_time_us":4117,"raft.run_wall_time_us":619264,"spinlock_wait_cycles":186286464,"thread_start_us":193090,"threads_started":2,"wal-append.queue_time_us":146232}
I20260709 13:21:05.564479 12634 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 25.
I20260709 13:21:05.565421 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
W20260709 13:21:05.540040 20356 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.932s	user 0.014s	sys 0.000s
I20260709 13:21:05.506783 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:05.344841 20340 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:05.548467 19907 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:05.557091 12633 raft_consensus.cc:2393] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 22: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:21:05.511899 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:05.512604 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:05.493229 20365 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:05.574057 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 71333.5us
I20260709 13:21:05.576004 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:05.576428 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:05.576992 13382 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:05.582294 12634 raft_consensus.cc:1240] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:05.527848 20063 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.061s
W20260709 13:21:05.575537 20356 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.968s	user 0.015s	sys 0.000s
W20260709 13:21:05.573508 20357 log.cc:927] Time spent T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:21:05.595505 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:05.598664 12776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36478 (request call id 4276) took 1423 ms. Trace:
I20260709 13:21:05.599129 12776 rpcz_store.cc:276] 0709 13:21:04.174949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:04.290165 (+115216us) service_pool.cc:224] Handling call
0709 13:21:04.290651 (+   486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:04.447436 (+156785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:05.598402 (+1150966us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b540011ce54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:21:05.598590 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":350859136}
I20260709 13:21:05.574869 20190 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.600428 13348 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:05.600570 12746 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:05.604470 13382 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:21:05.604815 12746 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:05.584182 20172 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:21:05.606769 20252 raft_consensus.cc:2764] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
W20260709 13:21:05.638296 12632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56832 (request call id 4748) took 2069 ms (client timeout 1688 ms). Trace:
W20260709 13:21:05.638728 12632 rpcz_store.cc:269] 0709 13:21:03.569286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:03.569816 (+   530us) service_pool.cc:224] Handling call
0709 13:21:05.638078 (+2068262us) spinlock_profiling.cc:243] Waited 911 ms on lock 0x7b54002fe9d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:21:05.638231 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2095083392}
W20260709 13:21:05.639623 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:05.601780 20294 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:05.602916 20305 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:05.641795 20172 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:05.642097 20305 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:05.642331 20172 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:05.644034 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:21:05.621820 20331 log.cc:927] Time spent T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 1.152s	user 0.005s	sys 0.000s
I20260709 13:21:05.621963 20388 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.623224 12776 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac"
dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
mode: GRACEFUL
new_leader_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:33732
I20260709 13:21:05.672405 12776 raft_consensus.cc:606] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Received request to transfer leadership to TS 19cd864c053744bebd96093a7982537e
I20260709 13:21:05.555006 20192 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:05.578944 20273 consensus_queue.cc:1048] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:21:05.624492 20212 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:05.607513 20147 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:05.608649 20319 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:05.626988 20238 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:05.627981 19902 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:05.633507 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:05.613685 19826 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:05.637308 12780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 1221) took 1499 ms. Trace:
I20260709 13:21:05.643914 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:05.605865 12746 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:21:05.687783 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 81990.4us
I20260709 13:21:05.670795 20376 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:05.610316 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:05.668223 20393 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:21:05.672399 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 3555) took 1750 ms. Trace:
I20260709 13:21:05.610008 19842 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:05.686673 12780 rpcz_store.cc:276] 0709 13:21:04.138224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:04.155062 (+ 16838us) service_pool.cc:224] Handling call
0709 13:21:04.155368 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:05.041797 (+886429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:05.041828 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:05.079090 (+ 37262us) log.cc:844] Serialized 26 byte log entry
0709 13:21:05.282979 (+203889us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:05.283018 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:05.595435 (+312417us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b540040e554. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:21:05.595485 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:05.595530 (+    45us) raft_consensus.cc:1685] finished
0709 13:21:05.595546 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:05.637219 (+ 41673us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":929251,"raft.queue_time_us":642350,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":6281,"spinlock_wait_cycles":564992768,"thread_start_us":198209,"threads_started":1,"wal-append.queue_time_us":258261}
I20260709 13:21:05.611385 19907 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:05.684036 12631 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:21:05.696708 12705 raft_consensus.cc:2393] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 23: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:05.635671 20190 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:05.615656 20105 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:05.645109 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 44764us
I20260709 13:21:05.688817 12779 raft_consensus.cc:2410] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:21:05.614383 20185 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:05.685218 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:21:05.678515 20388 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:05.613433 20295 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:05.644182 20305 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:05.678869 20147 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:05.686623 19826 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:05.692245 12702 rpcz_store.cc:276] 0709 13:21:03.921884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:03.922261 (+   377us) service_pool.cc:224] Handling call
0709 13:21:03.922579 (+   318us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:03.922977 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:03.923007 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:03.923619 (+   612us) log.cc:844] Serialized 52 byte log entry
0709 13:21:04.065043 (+141424us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b1800346a20. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2a7e07 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c6cf4c20b 00007f4c6cf51d23 00007f4c6cf520d1 00007f4c6d36da0f
0709 13:21:04.133580 (+ 68537us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:04.243758 (+110178us) log.cc:844] Serialized 14 byte log entry
0709 13:21:04.604903 (+361145us) log.cc:844] Serialized 14 byte log entry
0709 13:21:04.618845 (+ 13942us) spinlock_profiling.cc:243] Waited 5.68 ms on lock 0x7b1800346a20. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c5522 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:21:04.635893 (+ 17048us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:04.635962 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:04.658863 (+ 22901us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b1800346a20. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c53f3 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:21:05.621176 (+962313us) spinlock_profiling.cc:243] Waited 1.14 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:21:05.672168 (+ 50992us) raft_consensus.cc:1685] finished
0709 13:21:05.672189 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:05.672339 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":116359,"prepare.queue_time_us":314258,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":72,"raft.queue_time_us":144776,"raft.run_cpu_time_us":489,"raft.run_wall_time_us":28637,"spinlock_wait_cycles":73458560,"thread_start_us":527293,"threads_started":3,"wal-append.queue_time_us":478156}
I20260709 13:21:05.694505 12746 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:05.714380 12746 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:05.715499 12746 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:05.704248 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:05.705739 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:05.706161 13377 raft_consensus.cc:1240] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:05.706805 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:05.706743 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:05.708766 20394 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:21:05.698184 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 87943us
I20260709 13:21:05.712291 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:05.734875 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:05.735350 13382 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 20.
I20260709 13:21:05.703886 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:21:05.713366 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:05.691464 20294 raft_consensus.cc:2764] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:21:05.738494 12702 raft_consensus.cc:2368] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 25. Current term is 26.
I20260709 13:21:05.744947 12597 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:05.742915 12705 raft_consensus.cc:2393] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 21: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:05.747433 12628 raft_consensus.cc:2368] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18.
I20260709 13:21:05.701344 20293 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:05.753242 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:05.754231 13384 raft_consensus.cc:2393] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 20: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:21:05.754062 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:05.756868 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:05.758410 13381 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:21:05.761500 12597 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:05.758471 12702 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82"
dest_uuid: "639ab2db265f4747bb7afb93e833f530"
 from {username='slave'} at 127.0.0.1:42490
I20260709 13:21:05.730120 19826 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:05.754769 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:05.751606 12747 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:21:05.762848 12702 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:05.765933 12702 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:05.766657 12597 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:05.741778 19907 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:05.769097 12597 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:05.771116 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:21:05.744186 20206 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:05.773371 12597 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:05.746773 20132 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:05.744604 20323 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:21:05.713537 20385 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 13:21:05.756740 20365 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:05.759348 19958 consensus_queue.cc:1059] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:21:05.770874 20105 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:05.818392 13378 raft_consensus.cc:1240] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:05.775236 20394 raft_consensus.cc:697] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 17 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:05.776557 20319 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:05.818557 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:21:05.819116 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 48519.4us
I20260709 13:21:05.826792 12776 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:05.819729 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:05.750746 20375 consensus_queue.cc:1059] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:21:05.821731 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:05.828771 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:05.829149 12778 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:21:05.770987 20220 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:05.773985 20233 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:05.773051 20172 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:05.818531 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 47511.2us
I20260709 13:21:05.852299 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:05.829533 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:05.830147 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:05.834525 20132 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:05.835664 19842 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:05.835690 20388 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:05.836091 20378 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:05.839861 20169 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:05.818291 20242 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:05.836126 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:05.888463 20377 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:21:05.824193 20360 log.cc:927] Time spent T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.060s	user 0.000s	sys 0.004s
I20260709 13:21:05.830651 20298 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:05.824676 20105 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:05.854697 19907 raft_consensus.cc:697] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:05.899025 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:05.824291 20294 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:21:05.885072 12778 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 24.
I20260709 13:21:05.904238 12625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56832 (request call id 4773) took 1236 ms. Trace:
I20260709 13:21:05.916869 12634 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 24.
I20260709 13:21:05.826498 20319 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:05.879318 20340 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:05.886174 20132 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:05.886162 20189 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 13:21:05.833827 12702 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:05.899618 12782 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:05.904147 12631 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:21:05.912642 12633 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:21:05.914966 12776 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
W20260709 13:21:05.905288 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 53077.2us
I20260709 13:21:05.898289 20190 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:05.905110 20407 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:05.903354 20377 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:05.901639 20402 log.cc:927] Time spent T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:21:05.898664 19902 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 26 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:05.889773 20192 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:05.914614 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 141373us
I20260709 13:21:05.918329 19826 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:05.919234 13349 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:05.885855 13376 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:21:05.921833 12707 raft_consensus.cc:1240] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:05.928820 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:21:05.931286 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:05.917989 12625 rpcz_store.cc:276] 0709 13:21:04.667970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:04.748909 (+ 80939us) service_pool.cc:224] Handling call
0709 13:21:04.749478 (+   569us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:05.398373 (+648895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:05.903987 (+505614us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b54000dd754. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:21:05.904179 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1486679424}
I20260709 13:21:05.926761 20132 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:05.903496 12630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39590 (request call id 1874) took 1104 ms. Trace:
I20260709 13:21:05.878360 20206 raft_consensus.cc:697] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:05.929991 13349 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:05.889528 20323 raft_consensus.cc:697] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:05.921996 20319 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:05.894457 20376 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:05.916368 20105 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:05.899117 20378 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:05.899032 20147 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:05.934087 12702 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 26 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:05.952170 20378 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:05.898165 20295 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:05.899279 20244 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:05.957170 20244 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:05.958567 12626 raft_consensus.cc:1240] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:21:05.940665 20327 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:21:05.978677 20323 consensus_queue.cc:237] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:05.982210 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:05.983027 12778 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:21:05.984759 20333 log.cc:927] Time spent T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 1.522s	user 0.000s	sys 0.010s
I20260709 13:21:05.927232 20377 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:05.993534 13379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51762 (request call id 1484) took 2117 ms. Trace:
I20260709 13:21:05.935479 20406 raft_consensus.cc:2764] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:21:05.920562 20294 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:05.994668 20294 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:05.994951 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:05.941429 20206 consensus_queue.cc:237] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:05.941728 19907 consensus_queue.cc:237] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:05.941435 20252 consensus_queue.cc:1059] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:21:05.998749 20252 raft_consensus.cc:3055] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:05.999192 20252 raft_consensus.cc:740] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:06.000613 20220 raft_consensus.cc:3055] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:21:06.000263 20252 consensus_queue.cc:260] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.001856 20252 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:05.941761 20192 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.002s
I20260709 13:21:05.944043 20182 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:05.898402 20394 consensus_queue.cc:237] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:05.932701 20190 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:21:05.994129 13379 rpcz_store.cc:276] 0709 13:21:03.875842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:03.875981 (+   139us) service_pool.cc:224] Handling call
0709 13:21:03.895879 (+ 19898us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:03.896305 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:03.896334 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:03.905894 (+  9560us) log.cc:844] Serialized 494 byte log entry
0709 13:21:04.286942 (+381048us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:04.403798 (+116856us) log.cc:844] Serialized 14 byte log entry
0709 13:21:04.404187 (+   389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:04.404362 (+   175us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:05.983968 (+1579606us) spinlock_profiling.cc:243] Waited 5.34 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:21:05.991221 (+  7253us) raft_consensus.cc:1685] finished
0709 13:21:05.993236 (+  2015us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:05.993415 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2014,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":10336,"spinlock_wait_cycles":12282240,"thread_start_us":315858,"threads_started":1,"wal-append.queue_time_us":470363}
W20260709 13:21:05.946741 20396 log.cc:927] Time spent T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.245s	user 0.002s	sys 0.000s
I20260709 13:21:05.937779 20254 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:06.040105 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:06.040403 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:06.040789 12702 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:06.041116 12701 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 24.
I20260709 13:21:06.042012 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:06.042356 12701 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:21:05.997949 20355 log.cc:927] Time spent T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 1.392s	user 0.004s	sys 0.000s
I20260709 13:21:06.001215 20220 raft_consensus.cc:740] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:21:06.031718 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 59210us
I20260709 13:21:06.047966 12778 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 21.
W20260709 13:21:06.032264 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 96844.5us
I20260709 13:21:06.051347 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:06.051550 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:06.052037 13378 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:06.036356 19826 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:06.040781 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:06.052389 12597 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:05.933408 12630 rpcz_store.cc:276] 0709 13:21:04.799304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:04.856118 (+ 56814us) service_pool.cc:224] Handling call
0709 13:21:04.856457 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:04.926757 (+ 70300us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:05.903207 (+976450us) spinlock_profiling.cc:243] Waited 57.8 ms on lock 0x7b5400048b54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:21:05.903420 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":133046528}
I20260709 13:21:06.047547 12702 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 20.
I20260709 13:21:06.048473 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:06.053321 12705 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:05.938153 12747 leader_election.cc:400] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 23. Current term is 24.
I20260709 13:21:06.054505 12747 leader_election.cc:403] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:21:06.055001 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 117066us
I20260709 13:21:06.056058 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:06.057523 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:06.058060 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:06.058581 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:06.058818 12778 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 20: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:21:06.059721 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:06.059876 12672 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:06.062740 12597 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:06.063949 12782 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 22.
I20260709 13:21:06.064491 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:06.065079 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:06.065722 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:06.068217 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:06.068912 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:06.069077 12780 raft_consensus.cc:3055] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:21:06.069458 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:06.069638 12780 raft_consensus.cc:740] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:06.070164 12702 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:21:06.071008 13383 raft_consensus.cc:1240] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:21:06.032963 20190 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:06.076315 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: true dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:06.077987 13382 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:06.039515 20340 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.081441 12702 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:21:06.084271 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:06.039740 20351 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:06.086498 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:06.088269 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:06.091399 13379 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:06.092489 13378 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 21.
I20260709 13:21:06.072989 20182 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:06.095606 20220 consensus_queue.cc:260] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.096774 20220 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:06.069945 20343 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:21:06.108664 20298 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:06.094967 20340 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.109946 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:06.096936 12780 consensus_queue.cc:260] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.130223 12780 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:06.103293 20351 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.113713 20408 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 23. Current term is 24.
I20260709 13:21:06.116523 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:06.117393 12705 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 22.
I20260709 13:21:06.117259 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:06.072036 20254 raft_consensus.cc:697] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:06.117806 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:06.082664 20190 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:06.103405 12672 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:06.132769 13382 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 26.
I20260709 13:21:06.133544 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:06.136448 12747 leader_election.cc:400] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 pre-election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 25. Current term is 26.
I20260709 13:21:06.140132 12629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:06.158324 12629 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:21:06.158774 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:06.159780 19826 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:21:06.160279 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:06.150955 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:06.153929 13379 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 20.
I20260709 13:21:06.154978 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:06.161530 12631 raft_consensus.cc:2393] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 16: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:21:06.082082 20219 consensus_queue.cc:1059] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 13:21:06.162281 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:06.175776 12702 raft_consensus.cc:1275] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
W20260709 13:21:06.163054 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 70107.1us
W20260709 13:21:06.176124 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 72803.4us
I20260709 13:21:06.177414 12672 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:06.163421 12701 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 24.
I20260709 13:21:06.166208 20063 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:21:06.174157 20169 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:06.174393 12747 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:06.176122 12705 raft_consensus.cc:2393] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 14: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:21:06.178790 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:06.176687 13349 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:06.183861 19826 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:06.180536 12780 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 26.
I20260709 13:21:06.181026 12672 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:06.184650 12747 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:21:06.189255 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:06.191078 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:06.192301 12776 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 20: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:21:06.192551 12782 raft_consensus.cc:2368] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 20. Current term is 21.
I20260709 13:21:06.193549 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:06.194963 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:06.195586 12782 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:06.180588 19848 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:06.199339 20351 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:06.189692 20394 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:21:06.200767 20394 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:06.102550 20376 consensus_queue.cc:1175] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "522fd2ba0e9f402691b5ef29af8579cd" responder_term: 12 status { last_received { term: 10 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12. Ops: [11.11-11.11]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:21:06.157249 20172 raft_consensus.cc:2764] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:21:06.209227 19848 raft_consensus.cc:697] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 21 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:06.188896 20385 consensus_queue.cc:1048] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:21:06.210502 19848 consensus_queue.cc:237] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.187263 20298 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:06.202448 20378 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.209313 20401 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:06.214334 13348 leader_election.cc:400] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 20. Current term is 21.
I20260709 13:21:06.202385 20394 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:06.223592 13348 leader_election.cc:403] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:21:06.223788 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:06.225852 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:06.228242 12672 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:06.236831 12783 raft_consensus.cc:1240] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:06.232172 13348 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:06.238808 12630 raft_consensus.cc:1240] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:21:06.238489 12782 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:21:06.237190 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:06.240732 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:06.245596 12780 raft_consensus.cc:2393] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 16: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:21:06.214303 20408 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:06.209271 20063 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:06.151511 20294 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:06.247066 12672 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:06.252257 13348 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:06.259196 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:06.237907 20242 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:06.255918 19907 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:06.276319 20378 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:06.238102 20295 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:06.279006 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:06.279690 12775 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:06.286798 13382 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:06.256633 12672 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:06.280108 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:06.238056 20393 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 20. Current term is 21.
I20260709 13:21:06.237854 20401 raft_consensus.cc:697] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 18 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:06.285516 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:06.288133 13349 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:06.289104 12598 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:06.300679 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:06.299422 12782 raft_consensus.cc:2393] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 16: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:21:06.301802 12598 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:06.301898 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:06.301911 13382 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 21.
I20260709 13:21:06.255753 20179 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:06.252502 20365 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:06.213918 20254 consensus_queue.cc:237] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.255556 20369 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:06.281939 20272 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:21:06.304265 12597 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:06.291920 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:06.253860 20063 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:06.306288 12598 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:06.289906 20323 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:06.291046 19842 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:21:06.337481 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 45631.7us
I20260709 13:21:06.340996 20220 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:06.341157 19842 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.341661 20393 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.342125 19907 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:06.315279 20063 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:06.342413 20413 log.cc:927] Time spent T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:21:06.343462 20105 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.346053 20340 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:06.304942 20183 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:06.350389 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:06.383337 13384 raft_consensus.cc:2410] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 21 because replica has last-logged OpId of term: 18 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 16.
I20260709 13:21:06.299973 20298 raft_consensus.cc:2804] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:06.352289 20272 raft_consensus.cc:697] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 26 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:06.355891 20185 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:21:06.357676 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:06.358125 20220 raft_consensus.cc:697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 21 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:06.358595 20393 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.359632 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:06.288077 20242 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:06.400360 20242 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:06.283493 20408 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.360764 19842 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:06.307988 20238 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:06.305794 20378 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:21:06.366349 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:06.299540 20295 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:06.387141 12775 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 26.
W20260709 13:21:06.351634 20243 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.781s	user 0.043s	sys 0.043s
I20260709 13:21:06.396185 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:06.395907 20220 consensus_queue.cc:237] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.404383 12632 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:06.407662 12672 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:06.396965 20393 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:06.409667 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 105449us
I20260709 13:21:06.381829 20206 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:06.415288 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:06.405853 20408 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:06.417269 12706 raft_consensus.cc:2410] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:21:06.418681 20206 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.419535 20295 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:06.420678 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:06.394495 20183 raft_consensus.cc:697] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:06.425531 20206 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:06.426607 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 120386us
I20260709 13:21:06.425598 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:06.428042 12598 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:21:06.411345 20243 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.841s	user 0.043s	sys 0.043s
I20260709 13:21:06.407073 20185 raft_consensus.cc:697] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:06.434485 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:06.427696 12708 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:21:06.429385 12598 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:06.395004 20298 raft_consensus.cc:697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 20 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:06.435412 12705 raft_consensus.cc:1275] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 14. (term mismatch)
I20260709 13:21:06.437613 12705 pending_rounds.cc:85] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 13
I20260709 13:21:06.438123 12705 pending_rounds.cc:107] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 13:21:06.438428 12705 raft_consensus.cc:2889] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:06.440886 12778 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 14.
I20260709 13:21:06.441978 20298 consensus_queue.cc:237] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.442680 20116 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:06.443204 20116 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:06.444139 20185 consensus_queue.cc:237] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:06.412989 20401 consensus_queue.cc:237] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.419644 20319 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.452654 12701 raft_consensus.cc:1240] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:06.453632 20319 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:06.453897 20272 consensus_queue.cc:237] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.454473 12777 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 20.
W20260709 13:21:06.454628 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:06.421869 20378 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:06.455979 12701 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 21. Current term is 22. Ops: []
I20260709 13:21:06.426606 20340 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:06.459158 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:06.461043 13349 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:06.463115 20378 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.436429 20295 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:21:06.464573 20190 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Starting leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.436390 20238 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:21:06.467700 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:06.468972 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:06.482636 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:06.483865 12777 raft_consensus.cc:2393] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 16: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:21:06.473896 12625 raft_consensus.cc:2955] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 3 to 11, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:06.464016 20327 raft_consensus.cc:2955] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 LEADER]: Committing config change with OpId 15.11: config changed from index 3 to 11, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:06.486178 20105 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:06.474933 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:21:06.478142 12700 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42500 (request call id 3563) took 1764 ms (client timeout 1953 ms). Trace:
I20260709 13:21:06.488811 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:06.478235 12708 raft_consensus.cc:1275] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:21:06.459273 20243 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.496104 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:06.489035 12700 rpcz_store.cc:269] 0709 13:21:04.707311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:04.708543 (+  1232us) service_pool.cc:224] Handling call
0709 13:21:06.471922 (+1763379us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b540011c1d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:21:06.472032 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":842253056}
W20260709 13:21:06.497787 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:06.499785 12628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:56832 (request call id 4779) took 1401 ms. Trace:
I20260709 13:21:06.458359 20242 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:06.478258 19880 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.503091 19880 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.491514 20295 raft_consensus.cc:697] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 22 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:06.506049 20294 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:06.505955 20295 consensus_queue.cc:237] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:06.506955 12709 raft_consensus.cc:1275] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:21:06.506789 20243 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.510317 20063 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.512496 20063 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:06.512625 20378 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:06.506312 20369 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:06.514452 12782 raft_consensus.cc:1240] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:06.516083 12633 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:21:06.519414 13348 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:06.520390 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:06.524883 12630 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:21:06.525315 13384 raft_consensus.cc:2393] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 20: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:06.500313 12628 rpcz_store.cc:276] 0709 13:21:05.097855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:05.097985 (+   130us) service_pool.cc:224] Handling call
0709 13:21:06.499403 (+1401418us) spinlock_profiling.cc:243] Waited 648 ms on lock 0x7b54000de654. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:21:06.499547 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1490863360}
I20260709 13:21:06.480538 20254 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:06.521267 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:06.428851 20183 consensus_queue.cc:237] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.531163 20254 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:06.527482 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:06.528182 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:06.528455 12775 raft_consensus.cc:1275] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:21:06.530167 12634 raft_consensus.cc:1275] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:21:06.532692 20369 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.533751 20238 raft_consensus.cc:697] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 24 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:06.534446 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:06.540947 12634 pending_rounds.cc:85] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e: Aborting all ops after (but not including) 10
I20260709 13:21:06.541234 12634 pending_rounds.cc:107] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:21:06.528730 20393 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:06.535275 12777 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:21:06.535374 20377 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:06.541447 12634 raft_consensus.cc:2889] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:06.466358 20388 consensus_queue.cc:1059] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:21:06.542969 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:06.550184 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:06.550472 12777 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:06.521639 20243 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:06.538885 20254 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:06.559739 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:06.563794 12782 raft_consensus.cc:1275] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:21:06.525120 20242 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:06.577710 12777 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 20.
I20260709 13:21:06.549809 20393 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.595631 12628 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 25.
I20260709 13:21:06.596750 12775 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:21:06.608219 12779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36478 (request call id 4288) took 1130 ms. Trace:
I20260709 13:21:06.608716 12779 rpcz_store.cc:276] 0709 13:21:05.473908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:05.474315 (+   407us) service_pool.cc:224] Handling call
0709 13:21:06.603912 (+1129597us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b54001eb754. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:21:06.604120 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":431761280}
I20260709 13:21:06.582829 20388 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.021s
W20260709 13:21:06.617057 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 81072.7us
I20260709 13:21:06.562811 20092 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
W20260709 13:21:06.631675 12598 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.953s (SENT) [suppressed 5 similar messages]
I20260709 13:21:06.679409 20063 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:06.784564 20369 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:06.612689 20238 consensus_queue.cc:237] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:06.797986 12703 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 26 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 25. Current term is 26. Ops: []
I20260709 13:21:06.798940 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:06.799588 12631 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:06.803355 12628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:06.784586 20393 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:06.787113 20297 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:06.825235 12631 raft_consensus.cc:2468] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 25.
I20260709 13:21:06.829247 13349 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:06.794697 20190 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:06.786723 20063 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:06.835047 12631 raft_consensus.cc:1240] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:21:06.839965 12631 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:21:06.833062 20393 raft_consensus.cc:2804] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Leader election won for term 25
W20260709 13:21:06.843890 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 212243us
W20260709 13:21:06.851280 20429 log.cc:927] Time spent T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:21:06.852734 20369 consensus_queue.cc:1059] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:06.853178 19825 raft_consensus.cc:993] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2: : Instructing follower 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:21:06.853878 19825 raft_consensus.cc:1081] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Signalling peer 19cd864c053744bebd96093a7982537e to start an election
I20260709 13:21:06.854100 20441 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:21:06.862521 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:06.843724 20242 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:06.864182 20394 consensus_meta.cc:287] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd: Time spent flushing consensus metadata: real 0.616s	user 0.000s	sys 0.009s
W20260709 13:21:06.848656 20428 log.cc:927] Time spent T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:21:06.838593 20116 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.849939 20437 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.884198 20437 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.886833 20437 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:06.887374 20437 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:06.866400 20393 raft_consensus.cc:697] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 25 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:21:06.891954 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:06.892923 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:06.893934 12634 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 12.
I20260709 13:21:06.896132 13349 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:06.895015 20394 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.898576 20116 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:06.887833 20437 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.900380 20394 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:06.905289 13376 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:21:06.888909 20393 consensus_queue.cc:237] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 22.20, Last appended by leader: 20, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.912017 20440 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:06.912588 20440 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:06.913000 20440 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:06.914572 20437 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:06.919241 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:06.920224 12634 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 19.
I20260709 13:21:06.929276 20063 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:06.930099 20323 consensus_meta.cc:287] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.580s	user 0.009s	sys 0.004s
I20260709 13:21:06.930543 20323 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:06.910938 20243 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.021s
I20260709 13:21:06.931878 20440 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.932825 20254 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.935271 12626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:56838 (request call id 2982) took 1412 ms. Trace:
I20260709 13:21:06.935732 12626 rpcz_store.cc:276] 0709 13:21:05.522784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:05.558300 (+ 35516us) service_pool.cc:224] Handling call
0709 13:21:05.558593 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:05.959156 (+400563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:06.935009 (+975853us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b54000b7f54. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:21:06.935204 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":919245568}
W20260709 13:21:06.922405 20436 log.cc:927] Time spent T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
W20260709 13:21:06.941756 20239 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.305s	user 0.009s	sys 0.023s
I20260709 13:21:06.948873 20394 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:06.958771 20239 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.322s	user 0.009s	sys 0.024s
I20260709 13:21:06.963438 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:06.966508 12626 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:06.968215 20254 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:06.977797 20440 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:06.983155 12626 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 13.
I20260709 13:21:06.989717 13349 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:06.992553 20431 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:06.993391 20431 raft_consensus.cc:697] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:06.994894 20172 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.961853 20394 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:06.969811 20327 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:52424 (request call id 3) took 2336 ms. Trace:
I20260709 13:21:07.003291 12781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 4012) took 1183 ms. Trace:
I20260709 13:21:07.003733 12781 rpcz_store.cc:276] 0709 13:21:05.819655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:05.819801 (+   146us) service_pool.cc:224] Handling call
0709 13:21:06.604279 (+784478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:06.604708 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:21:06.604766 (+    58us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:06.604824 (+    58us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:06.604868 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:06.604945 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:07.003122 (+398177us) spinlock_profiling.cc:243] Waited 784 ms on lock 0x7b54001eb750. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:21:07.003231 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":657,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":5868,"spinlock_wait_cycles":1803244800}
I20260709 13:21:06.998973 20233 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:07.007939 20296 raft_consensus.cc:3055] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:07.008430 20296 raft_consensus.cc:740] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:07.008312 20233 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.009470 20296 consensus_queue.cc:260] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.010812 20296 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:06.995499 20431 consensus_queue.cc:237] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.012662 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:07.007673 12630 raft_consensus.cc:1240] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:21:07.018616 20394 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:06.995515 20172 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.022440 20233 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:07.043538 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:07.040926 20448 log.cc:927] Time spent T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:21:07.000407 20327 rpcz_store.cc:276] 0709 13:21:04.631563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:04.631883 (+   320us) service_pool.cc:224] Handling call
0709 13:21:04.651102 (+ 19219us) log.cc:844] Serialized 494 byte log entry
0709 13:21:05.910332 (+1259230us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b5400730b20. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d32b142 00007f4c6d30beaa 00007f4c6d312d53 00007f4c6d312cd9 00007f4c6d312c69 00007f4c6d312c31 00007f4c6d311dfd 000000000038f354 000000000038f199 00007f4c6a2c5475
0709 13:21:05.989221 (+ 78889us) spinlock_profiling.cc:243] Waited 7.56 ms on lock 0x7b1800479fe0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2932a6 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d332b26 00007f4c6d32c49a 00007f4c6d32b714
0709 13:21:05.992260 (+  3039us) spinlock_profiling.cc:243] Waited 4.63 ms on lock 0x7b5400730b20. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d332b26 00007f4c6d32c49a 00007f4c6d32b714 00007f4c6d30beaa 00007f4c6d312d53 00007f4c6d312cd9 00007f4c6d312c69 00007f4c6d312c31 00007f4c6d311dfd 000000000038f354
0709 13:21:06.968204 (+975944us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28997,"raft.queue_time_us":34701,"raft.run_cpu_time_us":9309,"raft.run_wall_time_us":2208986,"spinlock_wait_cycles":52274688,"thread_start_us":157042,"threads_started":1,"wal-append.queue_time_us":191840,"wal-append.run_cpu_time_us":1704,"wal-append.run_wall_time_us":1004334}
I20260709 13:21:07.054868 12781 raft_consensus.cc:1275] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 13:21:07.059360 20395 log.cc:927] Time spent T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 1.419s	user 0.013s	sys 0.000s
I20260709 13:21:07.060814 12704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 3575) took 1706 ms. Trace:
I20260709 13:21:07.061236 12704 rpcz_store.cc:276] 0709 13:21:05.354079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:05.354249 (+   170us) service_pool.cc:224] Handling call
0709 13:21:05.354666 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:05.354904 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:05.354929 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:05.355412 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 13:21:05.551968 (+196556us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:05.552011 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:05.552086 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:07.058700 (+1506614us) spinlock_profiling.cc:243] Waited 4.85 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:21:07.060487 (+  1787us) raft_consensus.cc:1685] finished
0709 13:21:07.060511 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:07.060749 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11148160,"thread_start_us":157135,"threads_started":1,"wal-append.queue_time_us":212347}
I20260709 13:21:07.069375 12781 raft_consensus.cc:1275] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:21:07.068269 20444 log.cc:927] Time spent T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:21:07.072199 20338 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:07.073930 20172 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:07.076843 20384 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:07.077505 20384 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.080431 20384 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:07.093485 12781 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:07.103087 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:07.103384 13382 raft_consensus.cc:1240] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:07.101239 12597 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:07.109474 12597 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:07.111683 13382 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 25. Current term is 26. Ops: []
I20260709 13:21:07.119429 12673 leader_election.cc:400] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18.
I20260709 13:21:07.119318 13382 raft_consensus.cc:1240] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 24. Current term is 25. Ops: [24.18-24.18]
I20260709 13:21:07.119980 12673 leader_election.cc:403] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:07.120417 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:07.123062 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:07.123920 12631 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 22. Current term is 23. Ops: []
I20260709 13:21:07.125833 12781 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:21:07.126796 12630 raft_consensus.cc:1240] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:07.128273 20243 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:07.128399 20238 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:21:07.128885 20243 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.120388 20338 consensus_queue.cc:1059] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 13:21:07.131377 20243 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:07.131685 20338 raft_consensus.cc:3055] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:07.132122 20338 raft_consensus.cc:740] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:07.126085 20368 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18.
I20260709 13:21:07.133832 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:07.134541 12626 raft_consensus.cc:2368] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16.
I20260709 13:21:07.135931 20424 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.137552 20424 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.136822 13379 raft_consensus.cc:1240] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:07.140157 12597 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:07.140411 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:07.117664 20406 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:07.143330 12597 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:07.144011 20239 consensus_queue.cc:1048] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:07.144143 20375 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:21:07.143970 20233 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:21:07.148869 20452 log.cc:927] Time spent T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:21:07.149271 12702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 3273) took 1037 ms. Trace:
I20260709 13:21:07.149796 19970 raft_consensus.cc:3055] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:07.194139 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:07.194345 19970 raft_consensus.cc:740] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:07.194455 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:07.194803 12626 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 22.
I20260709 13:21:07.195010 12627 raft_consensus.cc:3060] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:07.195235 19970 consensus_queue.cc:260] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.196261 19970 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:07.196175 20352 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:07.132931 20338 consensus_queue.cc:260] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.193832 12702 rpcz_store.cc:276] 0709 13:21:06.111767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:06.121797 (+ 10030us) service_pool.cc:224] Handling call
0709 13:21:06.122347 (+   550us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:06.176195 (+ 53848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:07.149016 (+972821us) spinlock_profiling.cc:243] Waited 53.1 ms on lock 0x7b54001b1554. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:21:07.149209 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":122081152}
W20260709 13:21:07.163105 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:07.163851 12632 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac"
dest_uuid: "19cd864c053744bebd96093a7982537e"
 from {username='slave'} at 127.0.0.1:56838
I20260709 13:21:07.164186 12630 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:21:07.167138 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:07.202189 12778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54346 (request call id 1231) took 1145 ms. Trace:
I20260709 13:21:07.202584 12778 rpcz_store.cc:276] 0709 13:21:06.057115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:06.059514 (+  2399us) service_pool.cc:224] Handling call
0709 13:21:07.202000 (+1142486us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b54003985d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:21:07.202121 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":841901952}
I20260709 13:21:07.203006 12630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:07.204253 12777 raft_consensus.cc:1275] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:21:07.204337 12630 raft_consensus.cc:3060] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:07.172725 19880 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:07.175997 20434 log.cc:927] Time spent T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.359s	user 0.001s	sys 0.001s
I20260709 13:21:07.207712 20273 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:07.208256 20273 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.210574 12626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:07.211402 12776 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:21:07.211691 12626 raft_consensus.cc:2393] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 20: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:21:07.210450 20273 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:07.212796 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:07.189162 12628 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 24. Current term is 25. Ops: []
I20260709 13:21:07.190012 12634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:07.214123 12625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:07.214890 12633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:07.215720 12633 raft_consensus.cc:2393] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 22: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:21:07.216881 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:07.190716 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:07.218969 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:07.219161 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:07.219635 12778 raft_consensus.cc:2376] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 26. Re-sending same reply.
I20260709 13:21:07.220520 12627 raft_consensus.cc:2468] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 20.
W20260709 13:21:07.192939 20320 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.088s	user 0.009s	sys 0.000s
I20260709 13:21:07.201821 12632 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:07.226272 12632 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:07.213105 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e"
I20260709 13:21:07.227154 20368 consensus_queue.cc:1059] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:07.228803 20368 raft_consensus.cc:3055] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:21:07.229159 20368 raft_consensus.cc:740] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:07.163097 20298 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:07.229455 20105 consensus_queue.cc:1048] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:07.230024 20298 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.231137 12673 leader_election.cc:400] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Vote denied by peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603) with higher term. Message: Invalid argument: T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16.
I20260709 13:21:07.232106 20298 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:07.234699 20172 consensus_queue.cc:1059] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:21:07.225389 12630 raft_consensus.cc:2468] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 26.
W20260709 13:21:07.190819 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 47562.2us
I20260709 13:21:07.221498 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:07.237691 12780 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 22.
I20260709 13:21:07.224004 12781 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:21:07.225265 12778 raft_consensus.cc:1275] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:21:07.225401 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:07.227504 20320 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.127s	user 0.010s	sys 0.000s
I20260709 13:21:07.219419 12598 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:07.238014 12597 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:07.240671 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:07.241523 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:07.242323 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:07.237654 20273 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.240971 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:07.236809 12673 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:06.841442 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:07.247997 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:07.252790 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:07.255244 20425 consensus_queue.cc:1048] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:21:07.294528 20384 consensus_queue.cc:1059] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:21:07.251292 20432 log.cc:927] Time spent T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.417s	user 0.003s	sys 0.000s
I20260709 13:21:07.297631 20384 raft_consensus.cc:3055] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:07.256567 12516 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 981fb2f435e24d06a81fc02cf4591cc5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 14)
I20260709 13:21:07.259164 12627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:07.264446 13382 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:21:07.299289 12627 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 23: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:21:07.242549 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:21:07.266438 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:07.300027 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 57539.1us
I20260709 13:21:07.266366 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:07.300951 13381 raft_consensus.cc:2376] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 26. Re-sending same reply.
I20260709 13:21:07.300098 13382 raft_consensus.cc:1240] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 25 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 24. Current term is 25. Ops: []
I20260709 13:21:07.301945 13385 raft_consensus.cc:1275] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:21:07.272532 12746 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:07.285435 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:07.288916 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:07.304001 12706 raft_consensus.cc:2468] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:21:07.288420 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:07.258850 12631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:07.246768 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:21:07.278734 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:07.268797 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:07.268097 13376 raft_consensus.cc:1275] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:21:07.286279 12673 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:07.274691 12781 raft_consensus.cc:2393] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 16: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:21:07.310750 12634 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 22 index: 20. Preceding OpId from leader: term: 25 index: 21. (index mismatch)
I20260709 13:21:07.266908 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:07.312147 13377 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 22.
I20260709 13:21:07.286731 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:07.245533 12632 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.353724 12707 raft_consensus.cc:2368] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 25. Current term is 26.
W20260709 13:21:07.354014 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 59737.8us
I20260709 13:21:07.246299 20168 raft_consensus.cc:3055] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:07.285117 12523 catalog_manager.cc:5697] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 19 to 20, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 20 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:07.406196 12632 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:07.250444 20406 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:07.256199 20426 log.cc:927] Time spent T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.637s	user 0.002s	sys 0.000s
I20260709 13:21:07.406103 12523 catalog_manager.cc:5697] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 15 to 16, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 16 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:07.355360 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:21:07.408648 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 51750.3us
W20260709 13:21:07.417234 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 140078us
I20260709 13:21:07.267190 20105 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:07.261157 20424 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:07.553686 12701 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 13:21:07.554162 12701 pending_rounds.cc:85] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 14
I20260709 13:21:07.312922 20323 raft_consensus.cc:2764] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Leader election decision vote started in defunct term 25: won
I20260709 13:21:07.554546 12701 pending_rounds.cc:107] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 13:21:07.554873 12701 raft_consensus.cc:2889] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:07.272092 20151 consensus_queue.cc:1059] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.282s
I20260709 13:21:07.354939 20233 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.543238 12531 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54792 (request call id 345) took 9353 ms. Trace:
I20260709 13:21:07.267895 20431 consensus_queue.cc:1059] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.282s
I20260709 13:21:07.252215 20408 consensus_queue.cc:1048] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.287s
I20260709 13:21:07.550781 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:07.561010 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 205558us
W20260709 13:21:07.442585 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 128398us
I20260709 13:21:07.273972 20293 consensus_queue.cc:1059] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.280s
I20260709 13:21:07.565100 20063 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:07.589478 20393 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:07.442492 20168 raft_consensus.cc:740] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:07.592227 19825 raft_consensus.cc:3055] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:07.357889 20257 consensus_queue.cc:1059] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.206s
I20260709 13:21:07.246641 20368 consensus_queue.cc:260] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.591028 12673 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:07.539721 20219 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:07.259068 20377 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:07.262450 19917 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.250595 20320 raft_consensus.cc:3055] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:07.312151 20384 raft_consensus.cc:740] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:07.565196 20394 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:07.564280 12531 rpcz_store.cc:276] 0709 13:20:58.189646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:58.189915 (+   269us) service_pool.cc:224] Handling call
0709 13:20:58.284697 (+ 94782us) spinlock_profiling.cc:243] Waited 915 us on lock 0x7b540011eed4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31c090 00007f4c6d3baf3a 00007f4c731711cc 00007f4c7326f37e 00007f4c717302a4 00007f4c71730232 00007f4c71730161 00007f4c717300dc 00007f4c7172f392 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:21:07.543149 (+9258452us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:04.092205 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:04.292993 (+200788us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:04.297977 (+  4984us) write_op.cc:435] Acquired schema lock
0709 13:21:04.334646 (+ 36669us) tablet.cc:662] Decoding operations
0709 13:21:04.364154 (+ 29508us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:04.364216 (+    62us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:04.364254 (+    38us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:21:04.365027 (+   773us) tablet.cc:701] Row locks acquired
0709 13:21:04.365049 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:21:04.365221 (+   172us) write_op.cc:270] Start()
0709 13:21:04.365347 (+   126us) write_op.cc:276] Timestamp: P: 1783603264365160 usec, L: 0
0709 13:21:04.365372 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:21:04.408572 (+ 43200us) log.cc:844] Serialized 3215 byte log entry
0709 13:21:04.607260 (+198688us) op_driver.cc:464] REPLICATION: finished
0709 13:21:05.497979 (+890719us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6a2a80ac 00007f4c73102ab9 00007f4c6a2c469b 00007f4c6a2c293f 00007f4c6a2c4c02 00007f4c70de8ef0
0709 13:21:05.602675 (+104696us) write_op.cc:301] APPLY: starting
0709 13:21:05.645549 (+ 42874us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:05.905667 (+260118us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:05.905699 (+    32us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:05.964384 (+ 58685us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:05.964734 (+   350us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=16,mrs_lookups=0
0709 13:21:05.964809 (+    75us) write_op.cc:312] APPLY: finished
0709 13:21:05.965350 (+   541us) log.cc:844] Serialized 81 byte log entry
0709 13:21:06.085589 (+120239us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:06.102413 (+ 16824us) write_op.cc:454] Released schema lock
0709 13:21:06.117951 (+ 15538us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:06.175047 (+ 57096us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:06.178882 (+  3835us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":100,"reported_tablets":21,"spinlock_wait_cycles":2105600,"child_traces":[["op",{"apply.queue_time_us":971637,"apply.run_cpu_time_us":26961,"apply.run_wall_time_us":561758,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37599,"num_ops":8,"prepare.queue_time_us":135077,"prepare.run_cpu_time_us":9562,"prepare.run_wall_time_us":413757,"raft.queue_time_us":44842,"raft.run_cpu_time_us":16149,"raft.run_wall_time_us":999505,"replication_time_us":241722,"spinlock_wait_cycles":56694656,"thread_start_us":1291493,"threads_started":5,"wal-append.queue_time_us":231658,"wal-append.run_cpu_time_us":13570,"wal-append.run_wall_time_us":2111276}]]}
I20260709 13:21:07.444222 19880 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.317593 20388 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.283378 20401 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.622814 20424 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:07.593442 20456 log.cc:927] Time spent T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.363s	user 0.001s	sys 0.000s
I20260709 13:21:07.625989 20169 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:21:07.613758 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:07.599279 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 48553.2us
I20260709 13:21:07.314935 20182 consensus_queue.cc:1048] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.285s
I20260709 13:21:07.545745 20105 raft_consensus.cc:697] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:07.623430 12673 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:07.404970 19907 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:07.275923 20297 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:07.627498 12634 raft_consensus.cc:1275] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:21:07.443886 20298 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.563906 12523 catalog_manager.cc:5697] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:07.313447 20254 consensus_queue.cc:1048] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.309s
I20260709 13:21:07.605688 20365 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:21:07.605245 19825 raft_consensus.cc:740] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:07.595706 20063 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:07.365526 20425 consensus_queue.cc:1048] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.270s
I20260709 13:21:07.275978 20338 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:07.267480 20273 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.621999 19970 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:07.603276 12746 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:07.639197 19880 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:07.667011 20219 raft_consensus.cc:2804] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:21:07.589404 20233 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.628216 20320 raft_consensus.cc:740] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:07.609063 20393 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.662544 12632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "19cd864c053744bebd96093a7982537e" is_pre_election: true
I20260709 13:21:07.677699 20063 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:07.638372 20252 raft_consensus.cc:3055] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:21:07.630239 20394 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.670481 20408 consensus_queue.cc:1059] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.021s
I20260709 13:21:07.632781 20151 consensus_queue.cc:1059] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 13:21:07.667433 12523 catalog_manager.cc:5697] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:07.641168 20323 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:21:07.628407 20377 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.673318 20298 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.629768 19917 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.645345 20401 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:07.686909 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 83717.5us
I20260709 13:21:07.442080 20375 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.224s
I20260709 13:21:07.655520 20424 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.663590 12630 raft_consensus.cc:1240] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:07.664997 12626 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 2
I20260709 13:21:07.722307 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:07.687800 19907 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:07.705464 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 87953.4us
I20260709 13:21:07.708642 12523 catalog_manager.cc:5697] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 17 to 18, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 18 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:07.637995 20388 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.656903 20168 consensus_queue.cc:260] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.717458 12746 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:07.734028 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:07.670356 20384 consensus_queue.cc:260] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.694101 20219 raft_consensus.cc:697] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 26 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:21:07.690824 19970 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.730880 13377 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:21:07.733042 13379 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:21:07.700243 20179 consensus_queue.cc:1059] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 13:21:07.699604 20252 raft_consensus.cc:740] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:07.731240 12523 catalog_manager.cc:5697] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
W20260709 13:21:07.734071 20458 log.cc:927] Time spent T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:21:07.728312 20424 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:07.744976 20257 raft_consensus.cc:3055] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:21:07.759083 20257 raft_consensus.cc:740] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:07.759924 20257 consensus_queue.cc:260] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 24.21, Last appended by leader: 21, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.761016 20257 raft_consensus.cc:3060] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:07.728071 19880 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:07.742285 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:07.742769 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: true dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:07.765897 13382 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:07.711515 20327 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.023s
W20260709 13:21:07.745319 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:07.746578 19917 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:07.748024 20252 consensus_queue.cc:260] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:07.749085 20457 log.cc:927] Time spent T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:21:07.751368 19970 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:07.751829 11517 tablet_server.cc:179] TabletServer@127.11.63.65:0 shutting down...
I20260709 13:21:07.718524 19825 consensus_queue.cc:260] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.753300 20273 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:07.754822 20298 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:07.724924 20151 consensus_queue.cc:1243] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 522fd2ba0e9f402691b5ef29af8579cd log is divergent from this leader: its last log entry 20.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
W20260709 13:21:07.739444 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 131554us
I20260709 13:21:07.767745 13377 raft_consensus.cc:2468] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
W20260709 13:21:07.735949 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:07.735740 20425 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.943588 20252 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:07.945271 20297 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:07.950775 20401 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:07.953204 20377 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:07.711262 20105 consensus_queue.cc:237] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:07.954769 20233 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:07.967262 13382 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 20.
W20260709 13:21:07.978048 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:07.987352 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 269941us
I20260709 13:21:07.995012 19825 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:07.725948 20320 consensus_queue.cc:260] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:07.981249 20327 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:07.987367 20464 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:21:07.987584 20273 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:07.999019 20151 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:08.026800 20063 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:08.023706 20388 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:08.009457 20401 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.006991 20298 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:08.009538 19902 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:08.011183 20465 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:07.998647 19917 consensus_queue.cc:1048] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 13:21:08.004650 20460 consensus_queue.cc:1048] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:21:08.012852 20323 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.029781 20320 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:08.004866 20425 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:08.060266 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 403938us
I20260709 13:21:08.069659 12672 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:08.063748 20393 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:08.047189 20365 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:08.024722 20327 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.016952 20219 consensus_queue.cc:237] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:08.038820 20151 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:08.045499 20401 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:08.035758 20273 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.049002 19902 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:08.086678 12672 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:21:08.088056 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 73500.1us
I20260709 13:21:08.049172 20298 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.090747 12746 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:08.055414 20323 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.083493 20297 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:08.052034 20465 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.088032 20468 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.088794 20394 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:08.088130 20169 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.016s
I20260709 13:21:08.089237 20464 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:08.081465 20365 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.111589 20464 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.098356 20466 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.104648 19970 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:08.112200 20425 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:08.084611 20463 consensus_queue.cc:1048] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
W20260709 13:21:08.134850 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 45306.4us
W20260709 13:21:08.135349 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 44663.1us
I20260709 13:21:08.139225 20464 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:08.141283 20401 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:08.141836 20063 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:08.143076 20327 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:08.152128 20327 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.154966 20327 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.155405 20465 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:08.157444 20327 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:08.162760 19902 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:08.165297 20473 consensus_queue.cc:1048] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.014s
I20260709 13:21:08.110200 20468 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.183492 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:08.129544 20466 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.183408 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:08.185308 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:08.202708 20466 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:08.184511 13376 raft_consensus.cc:2468] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:21:08.175129 20273 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:08.126941 20474 consensus_queue.cc:1059] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 13:21:08.189679 20151 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:08.191390 20365 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:08.194303 20298 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:08.194811 20185 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:08.197325 20468 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:08.199667 20293 raft_consensus.cc:3055] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:21:08.207008 20293 raft_consensus.cc:740] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:08.207968 20293 consensus_queue.cc:260] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:08.209218 20293 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:08.186869 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:08.211576 20425 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:08.211790 20298 consensus_queue.cc:1059] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:21:08.212620 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:08.232550 20425 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:08.214087 19880 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:21:08.215366 20463 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:08.235600 20425 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:08.235363 20463 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:08.236693 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 49898.8us
I20260709 13:21:08.215870 20464 consensus_queue.cc:1059] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:08.237699 12597 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:08.238094 20220 raft_consensus.cc:2804] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:08.238574 20257 raft_consensus.cc:3055] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:21:08.239061 20257 raft_consensus.cc:740] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:08.239310 20296 raft_consensus.cc:3055] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:08.239764 20296 raft_consensus.cc:740] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:08.240017 20257 consensus_queue.cc:260] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.241304 20273 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.241377 12597 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:08.242053 19907 raft_consensus.cc:3055] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:08.241978 20273 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.242635 19907 raft_consensus.cc:740] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:21:08.243618 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:08.245404 20168 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:08.230937 13381 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:21:08.253047 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:08.216588 20475 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:21:08.219820 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 71831.6us
I20260709 13:21:08.222389 20474 raft_consensus.cc:3055] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:21:08.223686 19825 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.224133 20063 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.224291 20408 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:08.226789 19917 consensus_queue.cc:1048] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:08.230877 19970 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:08.231930 20239 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.212734 20368 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:08.276961 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:08.298058 13384 raft_consensus.cc:1813] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Handling vote request from an unknown peer 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:08.298725 13384 raft_consensus.cc:2410] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:21:08.278004 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:08.299476 13382 raft_consensus.cc:2410] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
W20260709 13:21:08.278645 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:08.257545 20476 consensus_queue.cc:1059] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.077s
I20260709 13:21:08.238596 20233 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:08.238555 20220 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:08.240826 20296 consensus_queue.cc:260] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:08.300593 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:08.270905 20406 raft_consensus.cc:2804] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:08.281970 20273 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:08.280459 20474 raft_consensus.cc:740] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:08.317425 19829 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.318208 19829 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:08.320979 19829 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:08.324786 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:08.332978 12777 pending_rounds.cc:85] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 10
I20260709 13:21:08.333535 12777 pending_rounds.cc:107] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:21:08.336584 12777 raft_consensus.cc:2889] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:08.336115 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: true dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:08.341832 12781 raft_consensus.cc:3055] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:08.342237 12781 raft_consensus.cc:740] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:21:08.343760 13348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:08.343148 12781 consensus_queue.cc:260] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.347780 12781 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:08.359187 12775 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:21:08.361306 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:08.362301 12775 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:21:08.365974 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:08.311527 20220 raft_consensus.cc:3060] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:08.387190 13384 raft_consensus.cc:1240] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:08.379491 20252 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:08.294381 19825 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:08.312228 20406 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:08.296650 20063 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.298858 20408 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:08.404701 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:08.402005 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 315365us
I20260709 13:21:08.389448 20296 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:08.303884 20239 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:08.426352 12782 raft_consensus.cc:1275] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 13:21:08.388824 20471 log.cc:927] Time spent T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:21:08.390863 20252 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:08.411461 12672 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:08.396649 20406 raft_consensus.cc:3060] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:08.408355 20293 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.439327 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:08.450523 13381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 1922) took 1181 ms. Trace:
W20260709 13:21:08.452198 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:08.452253 20425 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:08.453742 19825 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:08.453632 20425 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.454380 19825 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.454892 19825 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.456362 20425 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:08.454201 20063 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:08.457033 19825 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:08.467126 20252 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:08.470842 20408 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:08.473187 20239 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:08.473982 20474 consensus_queue.cc:260] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:08.431694 20320 consensus_queue.cc:1059] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:21:08.475538 20320 consensus_queue.cc:1243] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 522fd2ba0e9f402691b5ef29af8579cd log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:21:08.476006 20320 raft_consensus.cc:3055] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:08.476353 20320 raft_consensus.cc:740] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:08.435400 20365 raft_consensus.cc:2764] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 26 LEADER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 13:21:08.450904 13381 rpcz_store.cc:276] 0709 13:21:07.269553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:07.301493 (+ 31940us) service_pool.cc:224] Handling call
0709 13:21:08.230459 (+928966us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:08.252010 (+ 21551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:08.450326 (+198316us) spinlock_profiling.cc:243] Waited 929 ms on lock 0x7b5400670a50. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:21:08.450464 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2135648000}
I20260709 13:21:08.477154 20320 consensus_queue.cc:260] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.482419 20320 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:08.485718 20352 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.486397 20352 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.488763 20352 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:08.489885 20293 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.491247 20384 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:08.491992 20293 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:08.495380 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:08.496006 13381 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 26.
I20260709 13:21:08.497784 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:08.498413 13381 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 21.
I20260709 13:21:08.498495 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:08.497880 12746 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:08.499866 20168 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.500479 20463 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:08.500598 20168 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.502408 20463 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:21:08.502113 12746 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:21:08.503930 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 287157us
W20260709 13:21:08.507169 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:08.504361 13384 raft_consensus.cc:2410] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
W20260709 13:21:08.553617 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 417866us
W20260709 13:21:08.554068 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 45460.4us
W20260709 13:21:08.556005 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:08.444341 19829 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:21:08.568058 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:08.501796 20239 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:08.569990 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:08.502913 20463 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:08.574097 12747 leader_election.cc:400] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for earlier term 25. Current term is 26.
I20260709 13:21:08.577190 12775 raft_consensus.cc:1275] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 24 index: 21. Preceding OpId from leader: term: 25 index: 21. (term mismatch)
I20260709 13:21:08.578734 12775 pending_rounds.cc:85] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 20
I20260709 13:21:08.567021 20473 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:08.578013 12705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:49378 (request call id 33) took 1298 ms. Trace:
I20260709 13:21:08.503436 20293 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.071s
I20260709 13:21:08.434499 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:08.566051 20408 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.578982 12775 pending_rounds.cc:107] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 24.21
I20260709 13:21:08.587831 12775 raft_consensus.cc:2889] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:08.589356 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:08.575574 20463 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:21:08.583277 12705 rpcz_store.cc:276] 0709 13:21:07.279439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:07.284121 (+  4682us) service_pool.cc:224] Handling call
0709 13:21:08.573257 (+1289136us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54003ec1d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b8735 00007f4c6d3b09fa 00007f4c723784c9 00007f4c6b8f4c54 00007f4c6b8f4be2 00007f4c6b8f4b11 00007f4c6b8f4a8c 00007f4c6b8f3d42 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:21:08.577955 (+  4698us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2764801792}
I20260709 13:21:08.590032 12775 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:21:08.595238 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:08.584484 12781 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 20.
W20260709 13:21:08.594763 12517 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 612a4639a7a749be8856bbc60766e312 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER failed with leader 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 639ab2db265f4747bb7afb93e833f530 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:08.607331 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:08.608572 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 174136us
I20260709 13:21:08.610520 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.610845 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.611526 12701 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 22.
I20260709 13:21:08.611325 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:08.614890 12701 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:21:08.573064 19825 consensus_queue.cc:1048] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.019s
I20260709 13:21:08.583243 20473 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:08.621690 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:08.622551 12705 raft_consensus.cc:2393] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 22: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:08.585235 20252 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:08.625658 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:08.571158 19829 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:08.628508 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 332994us
I20260709 13:21:08.577222 20239 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.594153 20408 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.631740 20473 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:08.633340 20063 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 13:21:08.657545 20474 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:08.617295 19917 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:08.662465 19917 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.645498 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:08.664984 19917 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:08.627363 19880 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:08.666123 19880 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:08.666456 19880 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:08.658999 19907 consensus_queue.cc:260] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.670970 19907 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:08.672871 19829 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:08.673062 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:08.679150 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:08.679977 12701 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:08.646768 20257 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:08.627592 20477 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.687516 20477 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.687832 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:08.678721 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:08.689901 12705 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:08.656733 20168 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:08.695142 12702 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:21:08.696461 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.697432 12706 raft_consensus.cc:2468] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 20.
I20260709 13:21:08.698683 12703 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:21:08.699250 20239 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:08.700325 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.701180 20408 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:08.701825 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:08.701869 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.702383 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.702646 12709 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 19.
I20260709 13:21:08.702929 12706 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:21:08.704131 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.704655 12706 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 17.
I20260709 13:21:08.705374 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:08.706288 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.706911 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.705663 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.707717 12700 raft_consensus.cc:2410] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:21:08.710501 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:21:08.714205 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:08.715803 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.717005 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.717648 12704 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 22.
W20260709 13:21:08.717761 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 448996us
I20260709 13:21:08.719204 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.720876 12705 raft_consensus.cc:2410] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 25 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 17.
I20260709 13:21:08.608125 20323 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:08.722817 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.724366 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:08.723814 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.726812 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.727377 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.727938 12700 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 24.
I20260709 13:21:08.729183 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:08.730014 13382 raft_consensus.cc:2410] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:21:08.729955 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.737083 12701 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 19.
I20260709 13:21:08.738276 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.738798 12702 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:21:08.742100 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.757752 13384 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:21:08.762034 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:21:08.766731 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:08.729655 20185 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:08.730732 20352 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:08.747368 20375 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:08.753312 20384 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.722162 20254 consensus_queue.cc:1048] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.043s
W20260709 13:21:08.771037 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:08.775647 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:08.754299 20467 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:08.800024 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:21:08.803154 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:08.788931 12703 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 23.
I20260709 13:21:08.806466 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.769284 20293 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:08.807255 12709 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 20.
I20260709 13:21:08.800815 20368 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:08.813009 12782 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:21:08.779922 20185 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.805516 20477 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:08.829689 20487 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:21:08.759380 20220 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.795519 20468 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:08.783138 20482 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:08.785198 20384 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.795974 20473 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.796995 20467 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.835870 20239 consensus_queue.cc:1048] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:21:08.836876 12775 raft_consensus.cc:1275] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:21:08.837828 20425 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:21:08.876638 12598 leader_election.cc:336] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.953s (SENT)
I20260709 13:21:08.804687 19970 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.849283 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.852954 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.853672 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:08.901669 12706 raft_consensus.cc:2376] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16. Re-sending same reply.
I20260709 13:21:08.853612 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.785388 20406 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.783402 20375 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:08.868880 13379 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.010s	sys 0.018s
I20260709 13:21:08.870687 12775 pending_rounds.cc:85] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 10
I20260709 13:21:08.905112 12775 pending_rounds.cc:107] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:21:08.905421 12775 raft_consensus.cc:2889] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:08.875494 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.877164 12598 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:08.906324 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:08.782855 20352 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.878659 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:08.841049 20293 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.868820 20233 raft_consensus.cc:697] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:08.841243 19880 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.844743 20368 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.890522 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:08.890726 20425 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.892283 12781 raft_consensus.cc:2393] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 22: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
W20260709 13:21:09.005227 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:08.892899 20473 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:08.895462 20493 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:08.895843 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:08.899075 20467 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:08.906955 13382 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:21:08.907232 13379 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.010s	sys 0.018s
I20260709 13:21:09.011669 12778 raft_consensus.cc:2468] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 14.
I20260709 13:21:08.925169 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.928323 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:08.930240 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:08.862792 20482 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:08.859732 20468 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:08.999718 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 73014us
I20260709 13:21:09.040966 13382 raft_consensus.cc:2468] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 22.
I20260709 13:21:09.042814 12709 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:21:09.043511 12746 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:09.006723 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:09.009692 12703 raft_consensus.cc:1275] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:21:08.915622 20463 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:08.910844 19970 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:09.017021 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:09.046020 12782 raft_consensus.cc:2410] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:21:09.001955 12774 raft_consensus.cc:2468] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:21:09.008045 20298 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.072610 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:09.081875 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:09.060946 20473 raft_consensus.cc:2804] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:21:09.123009 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:09.125748 12709 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:21:09.132081 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:09.130729 20467 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.137929 20467 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:09.142609 20368 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:09.125552 20233 consensus_queue.cc:237] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:09.131331 20487 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:09.123185 12703 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 23.
I20260709 13:21:09.148622 20487 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:09.153594 20487 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:09.157609 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:09.031721 20493 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:09.169252 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:09.170276 12703 raft_consensus.cc:2393] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 21: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:09.176115 19970 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:09.180091 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:09.078413 20298 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:09.158895 20239 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.196776 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:09.199901 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:09.200709 12781 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 27 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 26. Current term is 27. Ops: []
I20260709 13:21:09.201159 12703 raft_consensus.cc:2410] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 21 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:21:09.206130 13377 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 21. Current term is 22. Ops: []
I20260709 13:21:09.208164 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:09.215750 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:09.219624 13381 raft_consensus.cc:1275] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:21:09.220831 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:09.221704 12704 raft_consensus.cc:1275] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:21:09.223477 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:09.184551 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:09.139943 20473 raft_consensus.cc:697] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:09.188441 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:21:09.223721 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:09.224669 12781 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 24.
I20260709 13:21:09.259258 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:09.188050 20254 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:09.183581 20424 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.009s	sys 0.026s
W20260709 13:21:09.183107 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 468143us
I20260709 13:21:09.228979 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:09.263290 20424 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.009s	sys 0.026s
I20260709 13:21:09.218645 20239 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:09.295943 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:09.270216 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:09.298054 12781 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
W20260709 13:21:09.267102 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 491255us
I20260709 13:21:09.299644 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:21:09.242856 12672 connection.cc:256] RPC call timeout handler was delayed by -1.2562522319999516s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1564s for a response
I20260709 13:21:09.256345 12709 raft_consensus.cc:2410] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:21:09.257887 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:09.261590 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:09.190534 19880 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:09.222864 20480 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.035s	sys 0.054s
I20260709 13:21:09.304579 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:09.305392 13385 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:21:09.266255 13381 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:21:09.344161 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:09.307294 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:09.346211 13385 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
W20260709 13:21:09.323355 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:09.346271 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:09.326216 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:09.320693 12672 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:09.303704 13380 raft_consensus.cc:2393] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 19: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:21:09.346292 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:09.354991 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:09.355857 13380 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 22.
I20260709 13:21:09.357717 12777 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:21:09.276799 20500 log.cc:927] Time spent T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
W20260709 13:21:09.278829 20490 log.cc:927] Time spent T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.195s	user 0.000s	sys 0.000s
I20260709 13:21:09.362155 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:09.362383 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:09.363571 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:09.364120 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:21:09.279368 20495 log.cc:927] Time spent T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
W20260709 13:21:09.278306 20494 log.cc:927] Time spent T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.195s	user 0.000s	sys 0.008s
I20260709 13:21:09.372188 13381 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:09.370751 12709 raft_consensus.cc:2376] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 19. Re-sending same reply.
I20260709 13:21:09.339529 12747 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:09.340024 20480 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.035s	sys 0.054s
W20260709 13:21:09.360431 20489 log.cc:927] Time spent T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:21:09.370962 12700 raft_consensus.cc:2393] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 22: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:09.291579 20185 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:09.291378 20293 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:09.284956 20467 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:09.289695 20220 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:09.275244 20424 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.281746 20497 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.287062 20298 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:09.290716 20352 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:09.411890 12783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 4034) took 1096 ms. Trace:
I20260709 13:21:09.412807 13381 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 23.
I20260709 13:21:09.263739 20499 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:09.382361 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:09.292271 20425 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:09.400554 13384 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.009s	sys 0.012s
W20260709 13:21:09.421902 13384 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.009s	sys 0.013s
I20260709 13:21:09.359928 20368 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:09.367357 20463 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 27 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:09.303895 20254 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:09.290323 20482 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:09.387730 19829 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:09.367053 20219 consensus_queue.cc:1048] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:21:09.454425 13376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 1923) took 1180 ms. Trace:
I20260709 13:21:09.425989 20475 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:09.351636 19970 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.051s
I20260709 13:21:09.457206 13379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51754 (request call id 1701) took 1194 ms. Trace:
I20260709 13:21:09.458941 13379 rpcz_store.cc:276] 0709 13:21:08.262760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:08.264034 (+  1274us) service_pool.cc:224] Handling call
0709 13:21:08.270117 (+  6083us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:08.270496 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:08.270525 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:08.289070 (+ 18545us) log.cc:844] Serialized 26 byte log entry
0709 13:21:09.012069 (+722999us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:09.012107 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:09.012171 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:09.456892 (+444721us) raft_consensus.cc:1685] finished
0709 13:21:09.456924 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:09.457128 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10288,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":4695,"thread_start_us":708025,"threads_started":1,"wal-append.queue_time_us":909342}
I20260709 13:21:09.427711 20352 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:09.454892 13376 rpcz_store.cc:276] 0709 13:21:08.274549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:08.275823 (+  1274us) service_pool.cc:224] Handling call
0709 13:21:08.276276 (+   453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:08.281008 (+  4732us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:08.281057 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:08.293546 (+ 12489us) log.cc:844] Serialized 26 byte log entry
0709 13:21:08.731763 (+438217us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:08.731818 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:08.731904 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:09.454167 (+722263us) raft_consensus.cc:1685] finished
0709 13:21:09.454218 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:09.454362 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5240,"raft.queue_time_us":4902,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":9078,"spinlock_wait_cycles":8448,"thread_start_us":419270,"threads_started":1,"wal-append.queue_time_us":713565}
I20260709 13:21:09.409860 20467 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 13:21:09.406471 19880 consensus_queue.cc:1059] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.044s
I20260709 13:21:09.438793 20425 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:09.418303 20220 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 13:21:09.411476 20424 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:09.472019 20467 consensus_queue.cc:1243] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer 3d539f8d95d343d8b1696803f18b30d2 log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:21:09.417429 20497 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:09.407008 20473 consensus_queue.cc:237] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:09.479336 20477 raft_consensus.cc:3055] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:21:09.471747 20352 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:09.486565 20477 raft_consensus.cc:740] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:09.475353 20425 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:09.490860 20477 consensus_queue.cc:260] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:09.491925 20425 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:09.492147 20477 raft_consensus.cc:3060] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:09.501525 20497 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:09.504069 20493 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:09.550047 12598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.009s	sys 0.025s
W20260709 13:21:09.550992 12598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.009s	sys 0.025s
W20260709 13:21:09.553861 12599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.014s	sys 0.013s
W20260709 13:21:09.554488 12599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.014s	sys 0.014s
I20260709 13:21:09.559710 12776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 4041) took 1202 ms. Trace:
I20260709 13:21:09.562659 12780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 4040) took 1228 ms. Trace:
I20260709 13:21:09.564610 12782 raft_consensus.cc:2955] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 3 to 11, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:21:09.550050 12600 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.001s	sys 0.017s
W20260709 13:21:09.569294 12600 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.002s	sys 0.018s
W20260709 13:21:09.570102 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:09.412364 12783 rpcz_store.cc:276] 0709 13:21:08.315948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:08.323643 (+  7695us) service_pool.cc:224] Handling call
0709 13:21:08.324036 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:08.372086 (+ 48050us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:21:08.372134 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:08.372696 (+   562us) log.cc:844] Serialized 26 byte log entry
0709 13:21:08.790790 (+418094us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:21:08.790844 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:08.790942 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:09.411629 (+620687us) raft_consensus.cc:1685] finished
0709 13:21:09.411659 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:09.411822 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47532,"raft.queue_time_us":52573,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":81,"thread_start_us":417753,"threads_started":1,"wal-append.queue_time_us":687680}
I20260709 13:21:09.563258 12780 rpcz_store.cc:276] 0709 13:21:08.334348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:08.339289 (+  4941us) service_pool.cc:224] Handling call
0709 13:21:08.339727 (+   438us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:21:08.369893 (+ 30166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:08.369941 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:21:08.370771 (+   830us) log.cc:844] Serialized 52 byte log entry
0709 13:21:08.791199 (+420428us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:08.842507 (+ 51308us) log.cc:844] Serialized 14 byte log entry
0709 13:21:08.843034 (+   527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:08.843128 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:09.561849 (+718721us) raft_consensus.cc:1685] finished
0709 13:21:09.561909 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:09.562370 (+   461us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40613,"prepare.queue_time_us":94547,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":36639,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":6114,"thread_start_us":419962,"threads_started":1,"wal-append.queue_time_us":580587}
I20260709 13:21:09.576011 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:09.575865 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:09.579375 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:09.581532 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:09.592525 12780 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:21:09.593503 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 290817us
I20260709 13:21:09.595655 20352 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:09.560611 12776 rpcz_store.cc:276] 0709 13:21:08.356754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:08.356943 (+   189us) service_pool.cc:224] Handling call
0709 13:21:08.357430 (+   487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:08.747970 (+390540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:08.748008 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:08.748510 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 13:21:09.157320 (+408810us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:21:09.157390 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:09.157588 (+   198us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:09.558826 (+401238us) raft_consensus.cc:1685] finished
0709 13:21:09.558879 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:09.559301 (+   422us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17512,"raft.queue_time_us":19555,"raft.run_cpu_time_us":65,"raft.run_wall_time_us":65,"thread_start_us":780869,"threads_started":2,"wal-append.queue_time_us":666523}
I20260709 13:21:09.598683 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:09.599040 20510 log.cc:927] Time spent T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:21:09.599650 12776 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:21:09.603160 12779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36488 (request call id 4045) took 1238 ms. Trace:
I20260709 13:21:09.422276 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:09.619065 19829 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:09.619354 12777 raft_consensus.cc:2468] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 13.
I20260709 13:21:09.602540 20425 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.606001 20487 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:09.620524 20425 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:09.620910 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:09.619634 19829 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:09.621042 20487 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:09.629770 19829 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:21:09.601768 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 725079us
I20260709 13:21:09.628747 20425 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:09.606413 12780 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 23.
W20260709 13:21:09.632436 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:09.601531 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:09.629858 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:09.635567 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:09.611268 20406 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:09.638212 20468 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:09.639636 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:09.646747 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:09.647694 12783 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
W20260709 13:21:09.651279 20503 log.cc:927] Time spent T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.186s	user 0.000s	sys 0.000s
W20260709 13:21:09.618870 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 196521us
I20260709 13:21:09.609175 20375 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:09.663327 20518 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:09.676816 20515 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.677042 20518 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:09.678020 20515 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:09.679682 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:09.680191 20518 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:09.680614 20515 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:09.681981 20498 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.682864 20487 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:09.682799 20498 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:09.684008 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:09.685310 20482 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.617858 12779 rpcz_store.cc:276] 0709 13:21:08.364184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:08.364312 (+   128us) service_pool.cc:224] Handling call
0709 13:21:08.364680 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:08.740073 (+375393us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:21:09.185477 (+445404us) log.cc:844] Serialized 14 byte log entry
0709 13:21:09.574783 (+389306us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:09.576015 (+  1232us) log.cc:844] Serialized 26 byte log entry
0709 13:21:09.576334 (+   319us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:21:09.602128 (+ 25794us) log.cc:844] Serialized 14 byte log entry
0709 13:21:09.602442 (+   314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:09.602570 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:09.602645 (+    75us) raft_consensus.cc:1685] finished
0709 13:21:09.602665 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:09.602929 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11002,"prepare.queue_time_us":161372,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":56,"raft.queue_time_us":36776,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"thread_start_us":1170913,"threads_started":3,"wal-append.queue_time_us":313338}
I20260709 13:21:09.685701 20493 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:09.686090 20482 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:09.686827 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:09.686640 20493 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:09.688954 20482 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:09.719360 12779 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 22. Current term is 23. Ops: []
I20260709 13:21:09.725961 12597 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:09.727349 12778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 1266) took 1146 ms. Trace:
I20260709 13:21:09.689513 20493 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:09.742878 20467 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.708477 20498 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:09.756145 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 65130.8us
W20260709 13:21:09.757587 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 291474us
W20260709 13:21:09.758383 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 163568us
W20260709 13:21:09.773831 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 62166.2us
I20260709 13:21:09.766237 20482 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:21:09.782539 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 395020us
I20260709 13:21:09.782927 20532 raft_consensus.cc:2804] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:09.783643 20532 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:09.784121 20532 raft_consensus.cc:3060] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:09.784540 12774 raft_consensus.cc:1240] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:21:09.786655 12672 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:09.787976 20530 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:09.788354 20467 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:09.791422 20467 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:09.793799 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:09.799281 20384 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:09.800022 20502 log.cc:927] Time spent T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.536s	user 0.001s	sys 0.001s
I20260709 13:21:09.800506 20105 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:09.801393 20105 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:09.802265 12673 consensus_peers.cc:597] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:09.804442 20105 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:09.804859 12673 consensus_peers.cc:597] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:09.759765 12778 rpcz_store.cc:276] 0709 13:21:08.581226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:09.014516 (+433290us) service_pool.cc:224] Handling call
0709 13:21:09.014772 (+   256us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:09.020419 (+  5647us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:09.020463 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:09.571960 (+551497us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b2c00160c88. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6bcaba11 00007f4c6bcbb372 00007f4c6d3eda01 00007f4c6d3b243a 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2
0709 13:21:09.572921 (+   961us) log.cc:844] Serialized 26 byte log entry
0709 13:21:09.668600 (+ 95679us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:09.698696 (+ 30096us) log.cc:844] Serialized 14 byte log entry
0709 13:21:09.712723 (+ 14027us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:09.721126 (+  8403us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:09.726797 (+  5671us) raft_consensus.cc:1685] finished
0709 13:21:09.726913 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:09.727134 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15601,"prepare.queue_time_us":35726,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":81,"raft.queue_time_us":34888,"raft.run_cpu_time_us":3746,"raft.run_wall_time_us":35076,"spinlock_wait_cycles":5087488,"thread_start_us":95151,"threads_started":1,"wal-append.queue_time_us":149398}
I20260709 13:21:09.825206 20477 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:09.830150 12673 leader_election.cc:336] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:09.831386 19829 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:09.821686 20384 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:09.833912 20463 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:09.843948 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:09.843544 20532 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:09.841274 12673 consensus_peers.cc:597] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:09.840013 20151 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:09.851317 12777 raft_consensus.cc:1240] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 13:21:09.853013 20463 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:09.857695 20532 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:09.853252 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 127783us
I20260709 13:21:09.865263 13349 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:09.864192 20530 consensus_queue.cc:1048] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.002s
I20260709 13:21:09.865551 20535 consensus_queue.cc:1048] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:21:09.866791 20384 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:09.867369 20384 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:09.867831 20384 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:09.868433 20477 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:09.867664 13349 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:09.872465 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:09.870010 12673 consensus_peers.cc:597] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:09.876279 20473 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.873194 20151 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:09.883618 20151 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:09.881069 20482 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:09.884541 20487 raft_consensus.cc:2804] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:09.892858 19829 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:09.893080 20487 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:09.893745 20487 raft_consensus.cc:3060] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:09.895215 20477 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:09.886345 20493 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:21:09.898139 12598 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:09.886984 20254 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
I20260709 13:21:09.888257 12775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 1267) took 1297 ms. Trace:
I20260709 13:21:09.899642 12775 rpcz_store.cc:276] 0709 13:21:08.591169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:09.016498 (+425329us) service_pool.cc:224] Handling call
0709 13:21:09.047102 (+ 30604us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:09.049315 (+  2213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:09.049364 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:09.049984 (+   620us) log.cc:844] Serialized 26 byte log entry
0709 13:21:09.339691 (+289707us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:21:09.411584 (+ 71893us) log.cc:844] Serialized 14 byte log entry
0709 13:21:09.412012 (+   428us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:09.412194 (+   182us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:09.508040 (+ 95846us) spinlock_profiling.cc:243] Waited 7.33 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:21:09.614294 (+106254us) raft_consensus.cc:1685] finished
0709 13:21:09.614487 (+   193us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:09.850115 (+235628us) spinlock_profiling.cc:243] Waited 4.68 ms on lock 0x7b1800484cc0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c53f3 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:21:09.887909 (+ 37794us) spinlock_profiling.cc:243] Waited 30.3 ms on lock 0x7b5400127850. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b18ad 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:21:09.888078 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16534,"prepare.queue_time_us":50905,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":14036,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":6651,"spinlock_wait_cycles":97219968,"thread_start_us":216206,"threads_started":1,"wal-append.queue_time_us":404121}
I20260709 13:21:09.902279 20424 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:09.902781 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:09.888916 12782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54346 (request call id 1269) took 1195 ms. Trace:
I20260709 13:21:09.904855 12782 rpcz_store.cc:276] 0709 13:21:08.693785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:09.079118 (+385333us) service_pool.cc:224] Handling call
0709 13:21:09.079915 (+   797us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:09.080370 (+   455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:09.080410 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:09.113284 (+ 32874us) log.cc:844] Serialized 494 byte log entry
0709 13:21:09.563982 (+450698us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:09.812230 (+248248us) log.cc:844] Serialized 14 byte log entry
0709 13:21:09.864303 (+ 52073us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:09.888657 (+ 24354us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:09.888700 (+    43us) raft_consensus.cc:1685] finished
0709 13:21:09.888710 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:09.888862 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":309,"raft.queue_time_us":15587,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":155,"thread_start_us":440531,"threads_started":1,"wal-append.queue_time_us":381174}
I20260709 13:21:09.870853 20239 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:09.904201 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:09.907547 20482 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:09.862182 20463 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:09.911731 20493 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:21:09.913139 20493 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:21:09.920512 20424 consensus_queue.cc:1048] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:09.839063 20467 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:21:09.924666 12598 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:09.925127 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 60289.6us
I20260709 13:21:09.931316 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:21:09.925755 12673 leader_election.cc:336] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:09.933323 12597 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:21:09.935796 12597 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:21:09.936466 20239 raft_consensus.cc:2804] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:09.936956 20239 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:09.937361 20239 raft_consensus.cc:3060] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:09.937536 20425 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:09.938144 20425 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:09.938577 20425 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:09.936159 12673 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:09.877305 20473 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:09.943364 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 79787.2us
W20260709 13:21:09.951900 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 84411.9us
I20260709 13:21:09.953544 12672 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:09.943894 20384 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:09.954375 20298 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:09.956859 20477 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:09.968791 20477 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:09.969466 20477 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:09.969769 20530 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:09.970373 20530 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:09.970784 20530 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:09.968708 20528 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:09.966967 20532 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:09.983124 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:09.981156 20105 consensus_queue.cc:1059] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 13:21:09.986593 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:09.987498 20105 raft_consensus.cc:3055] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:21:09.981336 20528 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:09.988816 13349 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:21:09.992491 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 96171.3us
I20260709 13:21:09.994783 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:09.994967 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:09.995452 12780 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 22.
I20260709 13:21:09.995819 12774 raft_consensus.cc:2393] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 23: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:21:09.984244 20532 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:09.999047 12672 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:09.978914 20487 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:09.993813 20528 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:10.003516 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:10.002780 12673 leader_election.cc:336] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:10.011380 20533 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:10.012076 20533 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:10.012533 20254 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:10.013087 20254 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:10.013520 20254 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:10.014462 20533 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:10.015108 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:10.018122 19970 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:10.018452 13349 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:10.018743 12673 leader_election.cc:336] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:10.018911 19970 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:10.019457 20533 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:21:10.019708 12673 consensus_peers.cc:597] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:10.021003 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:10.022296 20473 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:10.022742 20467 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:10.023303 20467 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:10.020947 12673 consensus_peers.cc:597] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:10.024595 12673 consensus_peers.cc:597] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:10.025240 12673 leader_election.cc:336] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:10.001286 20532 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:09.989886 20105 raft_consensus.cc:740] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:10.025342 20467 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:10.032590 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:10.036746 19970 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:10.033214 20105 consensus_queue.cc:260] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:10.045876 20384 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:10.047205 20298 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:10.047786 20298 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:10.048278 20298 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:10.052014 12672 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:10.048458 20487 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:10.056026 20467 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:10.062892 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:10.056113 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:10.063767 12706 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 25.
I20260709 13:21:10.063100 20467 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:10.038298 13348 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:10.039242 12673 leader_election.cc:336] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:10.066143 12775 raft_consensus.cc:2393] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 15: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:21:10.065665 12673 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:10.071733 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:10.072481 12706 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 22.
I20260709 13:21:10.080603 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:10.081408 12776 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 22.
I20260709 13:21:10.081733 19970 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:21:10.082206 19970 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:10.082958 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:10.083048 20384 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:21:10.083396 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 57055.7us
I20260709 13:21:10.084355 19970 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:21:10.091317 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:10.067483 20467 raft_consensus.cc:3060] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:10.094298 20105 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:10.104382 20220 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:10.105278 20220 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:10.107569 20220 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:10.117661 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:10.123114 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:10.135258 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:10.138598 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:10.130810 20473 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:10.144901 12597 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:21:10.149547 12775 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 20.
W20260709 13:21:10.153093 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:10.156203 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 231723us
W20260709 13:21:10.163194 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 131120us
W20260709 13:21:10.165329 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:10.159837 20375 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:21:10.181165 12673 leader_election.cc:336] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:10.174721 20375 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:10.189924 12778 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:21:10.178092 19829 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:10.210305 19829 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:10.192921 12673 consensus_peers.cc:597] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:10.195922 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:10.196388 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:10.196836 20368 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:10.201905 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:10.184160 20239 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:10.193616 20375 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:10.204656 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:10.211668 12598 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:10.217427 12776 raft_consensus.cc:2393] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 23: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:10.221079 19829 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:10.222925 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:10.223891 13378 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 25.
I20260709 13:21:10.224337 12747 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:10.170673 20473 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:10.221186 13376 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:21:10.235417 13378 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:21:10.241020 12707 raft_consensus.cc:1275] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 16. (term mismatch)
I20260709 13:21:10.225283 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:10.238054 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:10.209136 20530 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:10.253338 12707 pending_rounds.cc:85] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 15
W20260709 13:21:10.229552 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 47022.8us
I20260709 13:21:10.240460 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:10.219517 20425 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:10.260545 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:10.260792 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:10.261837 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:10.262115 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:10.262603 12782 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:21:10.263136 20425 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:10.263610 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:10.263867 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:10.264576 12704 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:21:10.238651 12673 leader_election.cc:336] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:10.267050 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 50137.4us
W20260709 13:21:10.230175 20544 log.cc:927] Time spent T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:21:10.261837 20239 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:10.267993 20528 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:10.271483 20528 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:10.272612 12705 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 21.
I20260709 13:21:10.273972 13377 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 23.
I20260709 13:21:10.274463 20477 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:10.277112 20477 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:10.256898 12707 pending_rounds.cc:107] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 13:21:10.229434 20482 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:10.291563 12783 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 24. Current term is 25. Ops: []
I20260709 13:21:10.296864 12704 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 23.
I20260709 13:21:10.288465 12707 raft_consensus.cc:2889] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:10.299769 13376 raft_consensus.cc:2468] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 27.
I20260709 13:21:10.303488 20151 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:10.304234 20151 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:10.306555 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:10.307379 12780 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 20.
I20260709 13:21:10.305876 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:10.267975 12673 consensus_peers.cc:597] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:10.311815 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 43895.6us
I20260709 13:21:10.230271 19829 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:21:10.231410 20368 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:10.309744 12523 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54814 (request call id 312) took 11665 ms (client timeout 15000 ms). Trace:
I20260709 13:21:10.229825 20493 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:21:10.318584 12782 pending_rounds.cc:85] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 13
I20260709 13:21:10.320120 12782 pending_rounds.cc:107] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 13:21:10.320465 12782 raft_consensus.cc:2889] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:10.294027 20463 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:10.321601 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:10.285868 20384 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:10.342576 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:10.292856 20482 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:10.320595 12673 leader_election.cc:336] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:10.321655 20463 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:10.316951 19829 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:10.347074 12673 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:10.323077 20493 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:10.279625 19970 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:10.355284 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:10.360319 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:10.362608 12598 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:10.365978 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:10.367865 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:10.281697 20254 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:10.355875 19829 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:10.352283 20463 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:21:10.378283 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:10.380596 12746 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:10.359383 20493 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:10.381392 12705 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 22.
I20260709 13:21:10.390060 20239 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:10.403260 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:10.401180 12747 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:21:10.406167 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 55461.1us
I20260709 13:21:10.385314 20482 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:10.369446 20384 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:10.421693 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:10.422874 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:10.358820 20368 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:10.443809 12598 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:10.445904 20220 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:10.450340 13377 raft_consensus.cc:1240] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 27 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 26. Current term is 27. Ops: [26.18-26.18]
W20260709 13:21:10.452873 13349 leader_election.cc:336] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:10.457397 20384 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:10.406840 20239 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.051s
I20260709 13:21:10.414170 20151 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:10.440394 20468 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:10.461304 20468 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:10.461699 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:10.461643 20425 consensus_queue.cc:1059] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:21:10.462729 20425 raft_consensus.cc:3055] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:21:10.463124 20425 raft_consensus.cc:740] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:10.464027 20477 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:10.465979 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:10.466291 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:10.467247 20473 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:10.468825 20219 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:10.469125 20482 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:10.471223 20368 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:10.471503 20482 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:10.471722 19970 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:10.472182 20530 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:10.319710 12523 rpcz_store.cc:269] 0709 13:20:58.644076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:58.644294 (+   218us) service_pool.cc:224] Handling call
0709 13:21:10.307626 (+11663332us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:08.056107 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:08.133348 (+ 77241us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:08.136054 (+  2706us) write_op.cc:435] Acquired schema lock
0709 13:21:08.150266 (+ 14212us) tablet.cc:662] Decoding operations
0709 13:21:08.192307 (+ 42041us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:08.192366 (+    59us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:08.192394 (+    28us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:21:08.192913 (+   519us) tablet.cc:701] Row locks acquired
0709 13:21:08.192932 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:21:08.193085 (+   153us) write_op.cc:270] Start()
0709 13:21:08.193197 (+   112us) write_op.cc:276] Timestamp: P: 1783603268193033 usec, L: 0
0709 13:21:08.193219 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:21:08.193964 (+   745us) log.cc:844] Serialized 2444 byte log entry
0709 13:21:09.612032 (+1418068us) op_driver.cc:464] REPLICATION: finished
0709 13:21:09.794762 (+182730us) write_op.cc:301] APPLY: starting
0709 13:21:09.794955 (+   193us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:09.868102 (+ 73147us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:09.868284 (+   182us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:09.890428 (+ 22144us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b540011eed4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3af32a 00007f4c6d334f14 00007f4c6d334ec3 00007f4c6d334e38 00007f4c6d334df3 00007f4c6d3340dc 00007f4c6d35de16 00007f4c6d345cbb 00007f4c6d33039d 00007f4c6d333d75
0709 13:21:10.110398 (+219970us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:10.110759 (+   361us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=12,mrs_lookups=0
0709 13:21:10.110822 (+    63us) write_op.cc:312] APPLY: finished
0709 13:21:10.111257 (+   435us) log.cc:844] Serialized 65 byte log entry
0709 13:21:10.111533 (+   276us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:10.111720 (+   187us) write_op.cc:454] Released schema lock
0709 13:21:10.111788 (+    68us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:10.112492 (+   704us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:10.112521 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":39173,"reported_tablets":23,"child_traces":[["op",{"apply.queue_time_us":182289,"apply.run_cpu_time_us":14934,"apply.run_wall_time_us":317696,"cfile_cache_hit":26,"cfile_cache_hit_bytes":29231,"num_ops":6,"prepare.queue_time_us":283334,"prepare.run_cpu_time_us":43928,"prepare.run_wall_time_us":390270,"raft.queue_time_us":824397,"raft.run_cpu_time_us":37804,"raft.run_wall_time_us":409567,"replication_time_us":1418395,"spinlock_wait_cycles":235998592,"thread_start_us":1323044,"threads_started":4,"wal-append.queue_time_us":428963}]]}
I20260709 13:21:10.409154 20532 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:10.479249 20507 log.cc:927] Time spent T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:21:10.486083 13378 raft_consensus.cc:2410] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:21:10.465289 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:10.486166 12528 catalog_manager.cc:5697] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 12 to 13, VOTER 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) evicted. New cstate: current_term: 17 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:10.487761 12776 consensus_queue.cc:237] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:10.490676 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:10.491410 13382 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:21:10.485749 20467 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:10.467283 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:10.492390 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:10.489054 12528 catalog_manager.cc:5697] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:10.460907 20375 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:10.477393 20532 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:10.567481 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 114668us
I20260709 13:21:10.568492 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:21:10.567297 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 166207us
I20260709 13:21:10.571339 12747 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:10.523803 20468 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:10.492229 20530 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:10.576609 13382 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:21:10.577375 12704 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:21:10.576412 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 179993us
I20260709 13:21:10.579283 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:10.580215 12598 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:10.466212 20477 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:10.574717 20473 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:21:10.592839 12705 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
W20260709 13:21:10.592779 13349 consensus_peers.cc:597] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:10.464042 20425 consensus_queue.cc:260] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:10.610497 20425 raft_consensus.cc:3060] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:10.613935 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:21:10.614432 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:10.624750 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 56511.1us
I20260709 13:21:10.585819 12747 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:10.613646 20530 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:10.603415 20532 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:21:10.651273 13349 consensus_peers.cc:597] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:10.665998 20558 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:10.666862 20558 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:10.671870 12672 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:10.672847 20552 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:10.675587 20552 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:10.673398 20368 consensus_queue.cc:1059] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:21:10.678282 20368 raft_consensus.cc:3055] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:21:10.678719 20368 raft_consensus.cc:740] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:10.679710 20368 consensus_queue.cc:260] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:10.682641 20550 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:10.686699 20552 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:10.689965 20558 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:10.681320 20368 raft_consensus.cc:3060] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:21:10.692984 20564 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
I20260709 13:21:10.685288 12672 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:10.698416 19829 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:10.699111 12672 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:10.700378 20552 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:10.653115 12528 catalog_manager.cc:5697] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:10.716096 12598 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:10.701371 19829 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 26 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:10.734165 20384 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:10.735251 20384 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:10.737830 20384 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:10.755031 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:10.764782 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 65777.7us
W20260709 13:21:10.762205 13349 consensus_peers.cc:597] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:10.770480 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 455020us
I20260709 13:21:10.785746 20352 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:10.793462 20477 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:10.794902 20375 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:10.754628 20473 raft_consensus.cc:697] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:21:10.804352 13349 consensus_peers.cc:597] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:10.810111 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 154053us
I20260709 13:21:10.811223 20254 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:10.772198 20482 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:10.729655 20564 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
W20260709 13:21:10.829748 20298 consensus_meta.cc:287] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.781s	user 0.012s	sys 0.002s
I20260709 13:21:10.830466 12672 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:21:10.838366 20567 log.cc:927] Time spent T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:21:10.766275 12528 catalog_manager.cc:5697] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 17 to 18, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 18 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:10.839154 20467 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:10.800733 20463 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:10.803181 20473 consensus_queue.cc:237] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:10.829818 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 56532.4us
I20260709 13:21:10.855671 12704 raft_consensus.cc:2468] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 25.
W20260709 13:21:10.856253 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:10.855512 13349 leader_election.cc:336] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:10.865681 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 279901us
I20260709 13:21:10.855141 20482 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:10.881129 12598 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:10.865238 20552 raft_consensus.cc:2764] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 13:21:10.876955 12528 catalog_manager.cc:5697] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:21:10.854496 20468 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:10.872097 20482 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:10.860452 20298 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:10.902855 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 52875.9us
I20260709 13:21:10.888393 12747 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:10.859391 20564 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:10.904515 13348 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:21:10.907287 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:10.918977 20468 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:10.915949 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 85594.4us
W20260709 13:21:10.917126 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 56161.3us
I20260709 13:21:10.906679 12528 catalog_manager.cc:5697] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 14 to 15, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 15 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:10.893150 20561 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:21:10.925652 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 322988us
I20260709 13:21:10.902567 20375 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:21:10.929575 13349 leader_election.cc:336] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:10.931984 13348 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:10.932585 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:10.939250 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:10.939607 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:10.941318 12704 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
I20260709 13:21:10.926028 20151 raft_consensus.cc:2804] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:21:10.954068 20298 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:10.955013 20220 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:21:10.957345 20493 consensus_meta.cc:287] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.557s	user 0.005s	sys 0.005s
I20260709 13:21:10.958253 20493 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:10.961962 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:10.962729 12702 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:21:10.966404 20570 log.cc:927] Time spent T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:21:10.961251 20493 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:10.949517 12747 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:21:10.973618 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 45267.6us
I20260709 13:21:10.935624 20554 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:21:10.975850 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 50026us
I20260709 13:21:10.987426 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:10.988106 12704 raft_consensus.cc:3060] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:10.944269 20467 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:21:10.993372 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:10.994288 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 47260.3us
I20260709 13:21:10.993701 20151 raft_consensus.cc:697] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 25 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:21:10.997915 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 48530.1us
W20260709 13:21:10.998117 12748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:10.973408 20535 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:11.002275 12672 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:10.977777 20554 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:10.991566 20530 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:11.006611 13349 consensus_peers.cc:597] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:11.022152 20572 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:11.022871 20572 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:11.025722 20559 raft_consensus.cc:2764] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
W20260709 13:21:11.022819 20566 log.cc:927] Time spent T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 13:21:11.028391 20572 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:11.030436 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:11.030030 20151 consensus_queue.cc:237] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:11.003298 20468 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:11.016081 20463 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:11.035724 20564 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:11.010818 20554 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:11.038096 20425 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.038793 20425 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:11.003981 20535 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:11.041808 20535 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:21:11.045886 13349 leader_election.cc:336] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.033375 20468 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:11.049361 20468 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:11.051375 19829 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.052013 19829 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:10.986230 20375 consensus_queue.cc:1048] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:21:11.067991 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:11.068718 12706 raft_consensus.cc:3060] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:11.041242 20425 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:11.011855 12528 catalog_manager.cc:5697] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 15 to 16, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 16 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:21:11.111029 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 221671us
W20260709 13:21:11.119411 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 297417us
W20260709 13:21:11.129962 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 84121.9us
W20260709 13:21:11.130074 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 135180us
W20260709 13:21:11.134187 12673 consensus_peers.cc:597] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:11.147838 13350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:11.154066 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:11.160398 12704 raft_consensus.cc:2468] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 22.
W20260709 13:21:11.153232 13349 leader_election.cc:336] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.164090 13349 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:21:11.171766 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 159959us
W20260709 13:21:11.172070 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 156728us
I20260709 13:21:11.181118 12746 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:11.186352 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 234734us
I20260709 13:21:11.178073 20254 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:11.196244 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:11.196853 12775 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:11.192062 12528 catalog_manager.cc:5697] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 10 to 11, term changed from 14 to 16, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 16 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:11.195683 12747 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:21:11.197095 12673 leader_election.cc:336] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.200559 12673 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:11.195444 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:11.209724 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:11.210350 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:11.210525 12774 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:21:11.207756 12707 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:21:11.213665 12702 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 26.
W20260709 13:21:11.215216 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 77041.4us
I20260709 13:21:11.195778 19829 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:11.223064 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:11.198509 20151 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:11.223977 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:21:11.233244 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 64487.5us
I20260709 13:21:11.235726 12781 raft_consensus.cc:2393] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 23: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:21:11.209729 20220 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.236800 12747 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:21:11.246673 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 49622.6us
I20260709 13:21:11.240702 20563 raft_consensus.cc:2804] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:21:11.247361 12673 leader_election.cc:336] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.261821 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:11.262164 20151 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:11.262614 20151 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:11.263015 20151 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:11.270395 12774 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
W20260709 13:21:11.272181 12673 leader_election.cc:336] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.279628 20151 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:11.287675 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 48039.3us
I20260709 13:21:11.294796 12706 raft_consensus.cc:2468] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 23.
I20260709 13:21:11.298563 12775 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:21:11.252116 20368 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:11.317740 20530 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:21:11.318732 13349 leader_election.cc:336] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:11.319844 13349 leader_election.cc:336] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.320196 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:11.322573 12672 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:11.251937 20220 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:11.325084 20220 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:11.325870 12747 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:11.326725 20564 raft_consensus.cc:2804] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:21:11.326790 12747 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:11.327323 20564 raft_consensus.cc:697] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 26 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:11.327538 20554 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:11.328265 20564 consensus_queue.cc:237] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:11.293030 20563 raft_consensus.cc:697] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:11.330960 20581 raft_consensus.cc:2804] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:21:11.331604 20581 raft_consensus.cc:697] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:11.331241 20563 consensus_queue.cc:237] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:11.332533 20581 consensus_queue.cc:237] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:11.333859 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:11.305917 12528 catalog_manager.cc:5697] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 16 to 18, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 18 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:11.336287 12528 catalog_manager.cc:5697] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:11.321394 13349 leader_election.cc:336] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.338114 13349 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:11.339766 20467 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:21:11.343207 12528 catalog_manager.cc:5697] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 21 to 22, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 22 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:11.348791 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 62233.3us
I20260709 13:21:11.340445 20467 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:11.344923 20468 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:11.345198 12528 catalog_manager.cc:5697] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:11.273308 20535 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:11.355713 12597 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:21:11.346028 20530 raft_consensus.cc:697] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 23 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:11.355027 20554 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 19cd864c053744bebd96093a7982537e (127.11.63.65:43603)
W20260709 13:21:11.368903 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 46396us
I20260709 13:21:11.371140 20151 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:11.363098 20425 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:11.374759 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:21:11.377835 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 176259us
I20260709 13:21:11.378285 12702 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:11.376776 20482 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:11.385835 12746 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:21:11.374262 12673 leader_election.cc:336] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.366497 20550 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:11.397903 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:11.397755 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:11.394459 12673 leader_election.cc:336] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.401983 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:11.405545 12673 leader_election.cc:336] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.405830 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:11.406013 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:11.406617 13379 raft_consensus.cc:3060] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:11.407393 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:11.408641 12746 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:11.409775 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:11.389977 20582 log.cc:927] Time spent T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:21:11.415007 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:11.406371 12673 consensus_peers.cc:597] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:11.413733 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:11.406822 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:11.414537 13385 raft_consensus.cc:1275] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:21:11.419202 13376 raft_consensus.cc:2393] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 17: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:21:11.429674 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:11.430361 13383 raft_consensus.cc:1240] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:11.430552 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:11.432142 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:11.436759 13378 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:21:11.432750 13377 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 21.
I20260709 13:21:11.445813 20473 raft_consensus.cc:2804] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 13:21:11.446216 20580 log.cc:927] Time spent T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:21:11.449887 20239 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:21:11.450178 12672 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:11.451333 12672 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:11.451200 20530 consensus_queue.cc:237] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:11.398963 20573 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:21:11.457832 20573 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:11.458325 20573 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:11.399494 20151 consensus_queue.cc:1048] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.061s
I20260709 13:21:11.462961 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:11.464124 20590 consensus_queue.cc:1059] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:11.465260 20590 raft_consensus.cc:2764] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:21:11.465740 20558 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:11.395468 20425 raft_consensus.cc:697] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:11.466647 20298 raft_consensus.cc:3055] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:21:11.467201 20473 raft_consensus.cc:697] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 27 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:21:11.437947 12673 leader_election.cc:336] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.487882 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:11.488009 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:21:11.488487 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 55898.3us
I20260709 13:21:11.488888 12777 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:11.468201 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 66334.2us
I20260709 13:21:11.470808 13381 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:21:11.490520 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:11.491291 13376 raft_consensus.cc:3060] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:11.486932 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:11.486135 13378 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 24.
W20260709 13:21:11.493430 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:11.486755 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:11.497725 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:11.498041 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 158679us
W20260709 13:21:11.497059 12747 leader_election.cc:336] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Timed out: RequestConsensusVote RPC to 127.11.63.65:43603 timed out after 1.611s (SENT)
I20260709 13:21:11.500546 12747 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:11.502187 12702 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
W20260709 13:21:11.504424 12747 consensus_peers.cc:597] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:11.505290 12747 leader_election.cc:336] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.505864 20571 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:11.508945 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:11.509814 19829 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:21:11.511121 12747 leader_election.cc:336] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:11.512506 12747 leader_election.cc:336] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.517766 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:11.518198 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:11.514611 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:11.498646 12673 leader_election.cc:336] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.518573 12705 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:11.513669 12781 raft_consensus.cc:2368] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 17. Current term is 18.
W20260709 13:21:11.520953 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 121873us
I20260709 13:21:11.521078 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:11.522745 13381 raft_consensus.cc:2393] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 17: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
W20260709 13:21:11.523362 12673 leader_election.cc:336] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.474391 20558 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:11.531311 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:11.532007 12706 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:11.525827 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:21:11.530376 13349 leader_election.cc:336] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.532795 13383 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:11.532817 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:11.534420 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:11.496814 20298 raft_consensus.cc:740] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:21:11.528373 12673 leader_election.cc:336] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.542389 12597 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:11.539528 12673 leader_election.cc:336] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:11.545478 12673 leader_election.cc:336] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.548151 12702 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 21.
I20260709 13:21:11.553107 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:11.536624 20558 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:11.561517 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:11.565601 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:11.566112 13381 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:21:11.541091 20571 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:11.540934 20586 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:11.575892 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:11.577159 12704 raft_consensus.cc:2410] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
W20260709 13:21:11.574123 13349 consensus_peers.cc:597] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:11.580456 12597 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:11.585141 12706 raft_consensus.cc:2468] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 21.
I20260709 13:21:11.589385 13376 raft_consensus.cc:2468] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
I20260709 13:21:11.589756 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:11.597908 20482 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:11.598933 20468 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:11.603996 13383 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
W20260709 13:21:11.612183 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 99739.9us
I20260709 13:21:11.624125 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:11.556171 20564 consensus_queue.cc:1048] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 13:21:11.633948 20239 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:11.578137 20220 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:11.556538 20563 consensus_queue.cc:1059] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.033s
W20260709 13:21:11.612854 13349 consensus_peers.cc:597] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:11.642082 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 48776.9us
W20260709 13:21:11.643967 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:11.640293 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:11.645195 12706 raft_consensus.cc:2393] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 23: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
W20260709 13:21:11.655490 12598 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 2.003s (SENT) [suppressed 101 similar messages]
I20260709 13:21:11.650674 20239 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:11.673954 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:11.676127 20239 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:11.591953 20425 consensus_queue.cc:237] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:11.679171 20588 log.cc:927] Time spent T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:21:11.670342 20535 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:11.667887 13349 consensus_peers.cc:597] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:11.681321 13349 consensus_peers.cc:597] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:11.682189 13349 leader_election.cc:336] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.682585 13349 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:11.683164 20571 raft_consensus.cc:3055] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:11.683591 20571 raft_consensus.cc:740] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:21:11.683925 13349 consensus_peers.cc:597] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:11.684464 20571 consensus_queue.cc:260] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:11.686241 20589 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:11.687999 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:11.689707 13349 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:21:11.694839 12747 leader_election.cc:336] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.695868 12777 raft_consensus.cc:2468] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:21:11.703234 13381 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 23.
I20260709 13:21:11.700448 12747 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:11.669066 20352 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:11.714808 20600 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.715548 20600 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:11.647105 20473 consensus_queue.cc:237] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:11.720646 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:11.721513 12702 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:21:11.649447 20298 consensus_queue.cc:260] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:11.707895 20384 raft_consensus.cc:2764] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 13:21:11.731531 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:11.734661 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:11.736828 12707 raft_consensus.cc:1240] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 13:21:11.742942 13379 raft_consensus.cc:2468] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 26.
W20260709 13:21:11.745491 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 165114us
I20260709 13:21:11.747371 12597 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:11.718421 20564 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:11.760848 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:11.761593 12704 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:21:11.762132 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:11.762739 12709 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:11.766881 20352 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:11.770155 20596 log.cc:927] Time spent T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:21:11.771595 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 101884us
W20260709 13:21:11.772406 12747 leader_election.cc:336] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:11.773398 12747 leader_election.cc:336] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:11.774410 12747 leader_election.cc:336] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.774803 12747 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:11.775626 20468 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:21:11.776062 12747 leader_election.cc:336] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.776516 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:11.777279 20468 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:11.726020 20563 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.777889 20563 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:11.780155 20563 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:11.790236 20298 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:11.790400 12709 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:21:11.792135 20600 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:11.773367 20571 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:21:11.778477 20598 log.cc:927] Time spent T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 13:21:11.796218 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:11.808110 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 122604us
I20260709 13:21:11.814810 13349 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:11.810581 12705 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 23.
I20260709 13:21:11.818197 12702 raft_consensus.cc:1240] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 24. Current term is 25. Ops: [24.15-24.15]
W20260709 13:21:11.823086 12747 leader_election.cc:336] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.829870 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:11.824307 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 66830.9us
I20260709 13:21:11.828063 12746 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:11.835763 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:11.835003 12746 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:11.841035 13379 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 23.
I20260709 13:21:11.843361 20384 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:11.848055 20581 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:11.838894 20564 raft_consensus.cc:2804] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:11.849014 20564 raft_consensus.cc:697] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:11.850311 20564 consensus_queue.cc:237] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:11.859174 12746 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:21:11.864480 12747 leader_election.cc:336] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.865110 12747 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:11.865394 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:11.866432 12702 raft_consensus.cc:2393] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 23: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:21:11.868553 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:11.869650 12705 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 24: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:11.840907 20473 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:21:11.872082 12597 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:11.878307 12706 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 23.
W20260709 13:21:11.879632 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 64982.7us
I20260709 13:21:11.837864 20573 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:11.886027 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:11.894204 12708 raft_consensus.cc:2393] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 17: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:11.889853 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:11.895339 12701 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:21:11.891316 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:11.880559 20602 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:11.891673 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:11.892151 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:11.897454 12704 raft_consensus.cc:2468] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 23.
I20260709 13:21:11.898972 12597 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:11.899801 20604 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:11.900214 20604 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:11.900552 20604 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:11.910702 20602 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:11.911541 20602 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:11.911898 20573 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:11.913810 20602 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:11.916399 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:11.916029 20558 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:11.918640 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 26 candidate_status { last_received { term: 25 index: 21 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:11.918777 20600 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:11.872483 20571 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:21:11.922549 12673 leader_election.cc:336] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.922111 20571 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:11.923339 20571 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:11.926523 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:11.927199 12704 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 21.
W20260709 13:21:11.927358 13349 connection.cc:256] RPC call timeout handler was delayed by -1.1663221730002533s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15200000000000002s for a response
I20260709 13:21:11.920704 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:11.927830 13349 connection.cc:256] RPC call timeout handler was delayed by -1.0898221730001296s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1605s for a response
I20260709 13:21:11.928229 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:21:11.928778 13349 connection.cc:256] RPC call timeout handler was delayed by -1.0025221730002158s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17020000000000002s for a response
I20260709 13:21:11.929141 13379 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:11.931172 12597 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:11.934376 20352 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:11.938479 12597 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:11.907383 20473 raft_consensus.cc:697] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 23 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:11.926034 20425 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:11.944671 20220 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:11.949115 20468 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:11.949846 20468 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:11.951293 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 86836.1us
I20260709 13:21:11.938113 20600 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:11.956709 12597 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:11.967792 12597 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:11.968575 20239 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:11.969353 13381 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:21:11.969637 20239 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:21:11.968621 12597 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:11.971035 20563 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:21:11.963646 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:11.970329 20239 raft_consensus.cc:697] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 24 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:11.962260 20473 consensus_queue.cc:237] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:11.980192 20600 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:21:11.961714 20468 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:11.940503 20564 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.982802 12597 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:11.984750 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:11.992154 13379 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 24.
I20260709 13:21:11.976083 20239 consensus_queue.cc:237] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:11.993444 12746 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:11.995137 20468 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:21:11.995929 20468 raft_consensus.cc:697] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:12.002925 13381 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:21:11.995041 20564 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:12.007738 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:12.002369 12747 leader_election.cc:336] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:12.009039 12747 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:11.983522 13349 leader_election.cc:336] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:12.010143 13349 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:11.982757 20558 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:12.004982 20425 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:12.011617 13378 raft_consensus.cc:2955] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Committing config change with OpId 19.15: config changed from index 14 to 15, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:21:12.004436 20612 log.cc:927] Time spent T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:21:12.022245 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:12.022887 13379 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 26.
I20260709 13:21:12.022089 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:12.023792 12706 raft_consensus.cc:2368] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 16. Current term is 17.
I20260709 13:21:12.025610 12747 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:12.027441 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 53557.2us
W20260709 13:21:12.028133 13349 leader_election.cc:336] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:12.028621 13349 leader_election.cc:336] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:12.013522 20573 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:12.042376 13349 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:12.051769 20604 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:12.055642 12704 raft_consensus.cc:1275] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:21:12.055838 12701 raft_consensus.cc:1275] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 13:21:12.056316 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:12.056838 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:12.061981 20468 consensus_queue.cc:237] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:12.063407 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:12.063634 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:12.064055 12704 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 19.
I20260709 13:21:12.064278 12707 raft_consensus.cc:2368] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 23. Current term is 24.
I20260709 13:21:12.065325 12704 raft_consensus.cc:1240] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 20. Current term is 21. Ops: [18.14-20.15]
I20260709 13:21:12.065836 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:12.066273 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:12.066378 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:12.067019 12709 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:21:12.068053 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:12.070856 20564 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:12.071837 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:12.073117 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:12.073832 12701 raft_consensus.cc:2368] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 25. Current term is 26.
I20260709 13:21:12.076373 12703 raft_consensus.cc:1275] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:21:12.079597 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:12.080199 12703 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:21:12.079883 20563 raft_consensus.cc:2955] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Committing config change with OpId 19.15: config changed from index 14 to 15, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:12.048602 20581 raft_consensus.cc:2804] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:12.090163 20581 raft_consensus.cc:697] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:12.092016 20604 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:12.117895 20581 consensus_queue.cc:237] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:12.126928 20425 raft_consensus.cc:697] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:21:12.133802 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 103260us
W20260709 13:21:12.134629 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 106014us
W20260709 13:21:12.138562 13349 leader_election.cc:336] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:12.142304 13349 leader_election.cc:336] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:12.153091 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:12.135116 20554 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:12.160430 13348 leader_election.cc:400] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 17. Current term is 18.
I20260709 13:21:12.160862 13348 leader_election.cc:403] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:21:12.141698 20589 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:12.163440 13380 raft_consensus.cc:2410] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:21:12.131659 20425 consensus_queue.cc:237] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:12.152061 20571 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:12.163187 13348 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:12.167229 20254 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 17. Current term is 18.
I20260709 13:21:12.160010 20579 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:21:12.170804 13349 leader_election.cc:336] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.520s (SENT)
I20260709 13:21:12.178951 20571 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:12.178807 20579 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:12.180272 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:12.183051 13349 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:12.206996 20579 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:12.208067 20579 raft_consensus.cc:2804] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:12.210100 20579 raft_consensus.cc:697] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:12.211428 20579 consensus_queue.cc:237] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:12.238032 12705 raft_consensus.cc:1275] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:21:12.245874 20586 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.247047 20589 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:12.246800 20586 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:12.258088 20586 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:12.259423 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 97299us
W20260709 13:21:12.261770 13349 leader_election.cc:336] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.605s (SENT)
I20260709 13:21:12.262352 13349 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:12.267164 20586 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:21:12.269134 13349 leader_election.cc:336] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.702s (SENT)
I20260709 13:21:12.277770 13349 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:12.282073 13349 leader_election.cc:336] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.720s (SENT)
I20260709 13:21:12.282491 13349 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:12.287849 20586 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:12.288682 20586 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:21:12.290091 13349 leader_election.cc:336] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.795s (SENT)
I20260709 13:21:12.290395 13349 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:12.290962 20579 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:21:12.291373 13349 leader_election.cc:336] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.839s (SENT)
I20260709 13:21:12.291801 13349 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:12.292346 20579 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:21:12.292779 13349 leader_election.cc:336] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Timed out: RequestConsensusVote RPC to 127.11.63.66:45173 timed out after 1.857s (SENT)
W20260709 13:21:12.289393 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 278833us
I20260709 13:21:12.322695 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:12.330227 20622 consensus_queue.cc:1059] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:21:12.333781 12597 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:12.336187 13349 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:12.336510 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:21:12.337738 12674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:12.338582 20579 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:21:12.338976 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 46248.3us
I20260709 13:21:12.343160 20623 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.343796 20623 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:12.346318 20623 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:12.341948 20622 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:21:12.347144 20622 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:12.347519 20622 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:21:12.348444 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:12.349363 12705 raft_consensus.cc:2393] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 17: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:12.353150 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:12.353938 12703 raft_consensus.cc:2368] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 17. Current term is 18.
I20260709 13:21:12.355595 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:12.361404 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:12.362696 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:21:12.366726 11517 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 12659)
I20260709 13:21:12.373929 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:12.382555 20563 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:50664 (request call id 14) took 2123 ms. Trace:
I20260709 13:21:12.383162 20563 rpcz_store.cc:276] 0709 13:21:10.258688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.264039 (+  5351us) service_pool.cc:224] Handling call
0709 13:21:10.359470 (+ 95431us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b54001f7fa0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d32f520 00007f4c6d3b8b42 00007f4c6d3b7800 00007f4c6d3b09fa 00007f4c723784c9 00007f4c6b8f4c54 00007f4c6b8f4be2 00007f4c6b8f4b11 00007f4c6b8f4a8c 00007f4c6b8f3d42
0709 13:21:10.568084 (+208614us) log.cc:844] Serialized 494 byte log entry
0709 13:21:12.382451 (+1814367us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":33273,"raft.run_cpu_time_us":1201,"raft.run_wall_time_us":1200,"spinlock_wait_cycles":43112192,"thread_start_us":164241,"threads_started":1,"wal-append.queue_time_us":271577,"wal-append.run_cpu_time_us":3353,"wal-append.run_wall_time_us":1107819}
I20260709 13:21:12.392969 12781 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:12.394371 20622 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:12.394495 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:12.401710 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:12.397044 12516 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 20db0f202ea546699789c2325d241b35 with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:21:12.404253 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:12.411273 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:12.394620 12703 raft_consensus.cc:2468] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 20.
I20260709 13:21:12.394992 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:12.422282 12700 raft_consensus.cc:2393] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 17: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:12.422392 12776 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:12.427927 20622 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 27 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:12.395473 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:12.430889 12600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:12.402014 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:12.431593 12774 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:21:12.404721 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:12.436244 12702 raft_consensus.cc:2410] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 28 because replica has last-logged OpId of term: 26 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 17.
I20260709 13:21:12.439479 12705 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:21:12.440416 12779 raft_consensus.cc:1240] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:12.443943 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:12.447944 12776 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
W20260709 13:21:12.440963 12747 leader_election.cc:336] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:12.449301 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:12.450419 12747 leader_election.cc:336] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:12.451469 12747 leader_election.cc:336] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:12.451884 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:12.455518 12747 leader_election.cc:336] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:12.459281 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:12.460423 12747 leader_election.cc:336] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:12.462188 12598 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:12.461326 12747 leader_election.cc:336] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:12.463581 12747 leader_election.cc:336] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:12.467089 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:12.468524 12747 leader_election.cc:336] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:12.469406 12747 leader_election.cc:336] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:12.471879 20622 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:12.472541 12747 leader_election.cc:336] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:12.473616 12747 leader_election.cc:336] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:12.474984 20581 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:12.465248 20564 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:12.399865 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:12.485010 13380 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:12.480274 12747 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:12.476706 12598 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:12.487643 12747 leader_election.cc:336] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:12.488231 12747 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:12.489614 12747 consensus_peers.cc:597] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:12.479671 20581 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:21:12.494922 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:21:12.484966 20580 log.cc:927] Time spent T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.060s	user 0.000s	sys 0.003s
I20260709 13:21:12.496178 20622 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:21:12.503957 20473 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:12.495851 12598 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:12.515619 20622 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:12.504926 20581 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:12.518159 12598 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:12.519960 20622 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:12.520381 20622 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:12.520735 20622 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:12.531805 13380 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:21:12.532483 12598 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:12.530230 20571 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:21:12.537139 20622 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:12.538918 12598 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:12.541100 12598 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:12.542654 12672 connection.cc:256] RPC call timeout handler was delayed by -1.8270707340000172s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1506s for a response
I20260709 13:21:12.543119 12598 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:21:12.493419 12747 leader_election.cc:336] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:12.544499 20621 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:21:12.547497 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 54123.8us
W20260709 13:21:12.548147 12747 leader_election.cc:336] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:12.548910 12747 leader_election.cc:336] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:12.549273 12747 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:12.538808 20635 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.543776 20220 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:12.555605 20220 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:12.556013 20220 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:12.556842 13376 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:21:12.405957 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:12.406963 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:12.564714 13379 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 24: Already voted for candidate 19cd864c053744bebd96093a7982537e in this term.
I20260709 13:21:12.410460 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:12.566244 13381 raft_consensus.cc:2393] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 17: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:21:12.566711 20622 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:12.567371 20622 consensus_queue.cc:1059] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:12.570426 20220 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:12.572769 20220 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:12.583554 12598 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:12.585960 12598 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:12.551282 12747 leader_election.cc:336] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:12.550801 20602 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:12.554437 20623 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:12.587390 20623 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:12.587960 20623 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:12.588034 13383 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 23. Current term is 24. Ops: []
I20260709 13:21:12.411273 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:12.553436 20635 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:12.589309 13385 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:21:12.590692 20635 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:12.544761 20425 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:12.596108 20623 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:12.597918 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:12.598817 20623 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:12.596942 12747 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:12.599655 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:12.599946 20602 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:12.600972 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 49757.1us
I20260709 13:21:12.606292 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:12.608966 13376 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:21:12.610808 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:12.613595 20604 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:12.622051 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:12.629805 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:12.656507 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 70619.3us
I20260709 13:21:12.656803 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:12.657284 12598 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:12.648010 20493 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:21:12.648527 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 51085.3us
I20260709 13:21:12.648622 20621 consensus_queue.cc:1059] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:21:12.648664 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:12.650048 12776 raft_consensus.cc:1275] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:21:12.654134 13349 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:12.617308 12780 raft_consensus.cc:1240] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 23. Current term is 24. Ops: []
I20260709 13:21:12.630792 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:12.657442 13376 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:12.411885 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:12.663511 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:12.664739 12776 raft_consensus.cc:2393] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 23: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:21:12.665205 13383 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15.
I20260709 13:21:12.669075 12779 raft_consensus.cc:2368] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16.
I20260709 13:21:12.596522 13385 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 27.
W20260709 13:21:12.673458 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 60104.9us
I20260709 13:21:12.673213 13376 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 24.
I20260709 13:21:12.675956 13377 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 27 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 13:21:12.679225 13380 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:21:12.681815 13385 raft_consensus.cc:1275] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:21:12.684188 20493 raft_consensus.cc:3055] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:21:12.684669 20493 raft_consensus.cc:740] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:12.685273 20467 consensus_queue.cc:1059] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:21:12.686400 20254 raft_consensus.cc:3055] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:21:12.686779 20254 raft_consensus.cc:740] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:12.688024 12598 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:12.687757 20254 consensus_queue.cc:260] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:12.687776 20425 raft_consensus.cc:2804] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:21:12.689869 20425 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:12.673986 20220 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.690289 20425 raft_consensus.cc:3060] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:12.690412 20220 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:12.691211 20622 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:12.691615 20622 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:12.691974 20622 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:12.692725 20220 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:12.677765 20554 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:12.694223 20473 consensus_queue.cc:1048] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:21:12.677997 20604 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:21:12.708386 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 161438us
I20260709 13:21:12.708467 12746 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:12.713467 13349 leader_election.cc:400] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Vote denied by peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) with higher term. Message: Invalid argument: T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for earlier term 17. Current term is 18.
I20260709 13:21:12.715564 19829 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:21:12.724285 19829 raft_consensus.cc:697] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:12.725291 19829 consensus_queue.cc:237] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:12.698443 20493 consensus_queue.cc:260] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:12.727854 20239 consensus_queue.cc:1059] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:21:12.732630 20604 raft_consensus.cc:3055] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:21:12.733093 20604 raft_consensus.cc:740] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:12.734061 20604 consensus_queue.cc:260] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:12.735579 20239 consensus_queue.cc:1243] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Peer 3d539f8d95d343d8b1696803f18b30d2 log is divergent from this leader: its last log entry 24.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 13:21:12.710502 20600 consensus_queue.cc:1048] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:21:12.716599 20581 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 13:21:12.721547 20564 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 13:21:12.717500 13349 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:12.741433 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:21:12.741832 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 62095.2us
I20260709 13:21:12.742666 12672 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:12.747390 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:12.746093 13349 leader_election.cc:336] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:12.755921 12672 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:12.755236 13349 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:12.759704 12747 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:12.763630 13349 leader_election.cc:336] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:12.764124 12672 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:12.764099 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:12.754046 20560 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:12.754278 20530 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:21:12.799053 13349 leader_election.cc:336] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:12.802376 13349 leader_election.cc:336] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:12.802783 13349 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:12.756968 20586 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:21:12.791252 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 56001.7us
I20260709 13:21:12.762682 20584 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:12.805215 12597 leader_election.cc:400] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16.
I20260709 13:21:12.785638 20579 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:12.809206 20579 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:12.810626 12597 leader_election.cc:403] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:21:12.397662 20573 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.849979 20573 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:12.851317 12774 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:12.852715 20573 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:12.859395 12774 raft_consensus.cc:2468] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 21.
I20260709 13:21:12.717376 20473 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.075s
W20260709 13:21:12.909837 12517 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 612a4639a7a749be8856bbc60766e312 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER failed with leader 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 639ab2db265f4747bb7afb93e833f530 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:21:12.910877 12517 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 105019us
W20260709 13:21:13.367250 11517 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 12659)
I20260709 13:21:12.887364 20647 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:21:13.885232 12563 debug-util.cc:398] Leaking SignalData structure 0x7b080036a620 after lost signal to thread 11520
I20260709 13:21:14.249924 20652 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:14.250478 20653 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:14.250648 20652 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:14.251129 20653 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.253580 20652 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:14.253942 20653 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:14.255823 20653 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:14.256610 20653 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.259261 20653 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:14.260916 20653 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:14.261328 20653 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.263595 20653 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:14.321321 20650 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.501s	user 0.008s	sys 0.022s
W20260709 13:21:14.321822 20650 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.502s	user 0.008s	sys 0.022s
I20260709 13:21:14.322144 20560 raft_consensus.cc:697] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:14.323268 20560 consensus_queue.cc:237] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.345774 20651 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:14.346143 20587 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:14.346431 20651 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.346643 20587 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.348567 20651 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:14.348567 20587 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:14.366935 20622 consensus_meta.cc:287] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 1.675s	user 0.012s	sys 0.026s
W20260709 13:21:14.367903 11517 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 12659)
I20260709 13:21:14.368438 20622 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.371184 20622 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:14.371942 12749 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.591s	user 0.021s	sys 0.012s
W20260709 13:21:14.372300 12749 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.600s	user 0.021s	sys 0.012s
I20260709 13:21:12.764695 20561 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:21:14.402954 13380 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.644s	user 0.010s	sys 0.021s
W20260709 13:21:14.403745 13380 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.645s	user 0.011s	sys 0.022s
W20260709 13:21:14.406646 12672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.605s	user 0.013s	sys 0.025s
W20260709 13:21:14.407177 12672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.606s	user 0.013s	sys 0.025s
W20260709 13:21:14.407781 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 1.64375e+06us
I20260709 13:21:14.412063 20587 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.412551 20651 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.412707 20587 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.413064 20651 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.414732 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:14.415445 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:14.416411 20468 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:14.417171 20587 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:14.416956 20468 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.417698 20651 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:14.420828 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:14.421947 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:14.422595 12701 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:12.798075 20220 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.425079 20220 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.427132 20220 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:14.428632 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:14.429416 20468 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:14.429950 12701 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:21:14.430090 20587 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.430742 20587 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.431888 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:14.435074 20651 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.435526 20586 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:14.435599 20651 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.436177 20586 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.437438 13380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51768 (request call id 1983) took 1754 ms. Trace:
I20260709 13:21:14.438365 20561 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.445029 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:14.445502 20561 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:14.446518 12673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.646s	user 0.003s	sys 0.036s
W20260709 13:21:14.446847 12673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.647s	user 0.003s	sys 0.036s
I20260709 13:21:14.439090 20560 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:14.439316 20651 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:14.447978 20651 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:14.447775 20560 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.448467 20651 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.450522 20561 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:14.451373 13380 rpcz_store.cc:276] 0709 13:21:12.682579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:12.719996 (+ 37417us) service_pool.cc:224] Handling call
0709 13:21:12.720337 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:12.720532 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 13:21:14.404181 (+1683649us) log.cc:844] Serialized 14 byte log entry
0709 13:21:14.411555 (+  7374us) log.cc:844] Serialized 14 byte log entry
0709 13:21:14.433080 (+ 21525us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:14.433135 (+    55us) raft_consensus.cc:1645] Marking committed up to 19
0709 13:21:14.433178 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:14.437104 (+  3926us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:14.437356 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":196348,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":92,"raft.queue_time_us":926,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":171,"thread_start_us":1652280,"threads_started":2}
I20260709 13:21:14.452864 20467 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:14.453418 20467 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.452877 20587 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:14.455381 20467 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:14.458786 20561 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:14.459334 20561 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.445739 12701 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 15.
I20260709 13:21:14.440814 12708 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 23.
I20260709 13:21:14.441147 20635 consensus_queue.cc:1059] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:14.444167 12672 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:14.464550 20604 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:14.467526 20651 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:14.476656 20651 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:14.477391 20651 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.480823 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:14.483623 20651 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:14.485271 20651 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:14.488003 20651 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:14.491812 12774 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36478 (request call id 4343) took 2007 ms (client timeout 2432 ms). Trace:
W20260709 13:21:14.492250 12774 rpcz_store.cc:269] 0709 13:21:12.484571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:12.484766 (+   195us) service_pool.cc:224] Handling call
0709 13:21:14.491612 (+2006846us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b54001fe854. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:21:14.491721 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":817618816}
W20260709 13:21:14.479620 13351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:14.501974 20651 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:14.514163 20651 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:14.514731 20651 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.516930 20651 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:14.517447 20651 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:14.518271 20651 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:14.518934 12804 debug-util.cc:398] Leaking SignalData structure 0x7b08003d7c00 after lost signal to thread 11520
W20260709 13:21:14.520154 12804 debug-util.cc:398] Leaking SignalData structure 0x7b08003ecd00 after lost signal to thread 12566
W20260709 13:21:14.520798 12804 debug-util.cc:398] Leaking SignalData structure 0x7b080038ed00 after lost signal to thread 12658
W20260709 13:21:14.521139 12804 debug-util.cc:398] Leaking SignalData structure 0x7b08003c7c00 after lost signal to thread 12659
W20260709 13:21:14.522730 12804 debug-util.cc:398] Leaking SignalData structure 0x7b0800380f20 after lost signal to thread 12733
W20260709 13:21:14.523077 12804 debug-util.cc:398] Leaking SignalData structure 0x7b080030d720 after lost signal to thread 12734
W20260709 13:21:14.524500 12804 debug-util.cc:398] Leaking SignalData structure 0x7b080035c220 after lost signal to thread 12807
W20260709 13:21:14.526074 12804 debug-util.cc:398] Leaking SignalData structure 0x7b080021b660 after lost signal to thread 13410
W20260709 13:21:14.526417 12804 debug-util.cc:398] Leaking SignalData structure 0x7b080040d6c0 after lost signal to thread 13411
I20260709 13:21:14.529462 20651 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:14.517012 20587 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:14.532291 20587 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.510667 20467 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.534147 20558 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:14.552948 20662 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:14.554939 20662 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:21:14.564435 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 120347us
I20260709 13:21:14.548206 20651 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:14.552186 20467 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.551898 20558 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:14.571406 20651 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.579421 20651 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:14.587369 20587 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:12.756835 20581 consensus_queue.cc:1048] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.123s
W20260709 13:21:14.599191 12747 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.801s	user 0.013s	sys 0.008s
W20260709 13:21:14.599753 12747 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.801s	user 0.013s	sys 0.009s
W20260709 13:21:14.600080 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 1.80764e+06us
I20260709 13:21:14.590819 20467 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:14.569008 20662 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:14.617703 19829 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:21:14.616360 20298 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.810s	user 0.009s	sys 0.024s
W20260709 13:21:14.619627 20298 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.816s	user 0.009s	sys 0.024s
I20260709 13:21:14.618388 19829 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.622716 20581 consensus_queue.cc:1048] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.013s
I20260709 13:21:12.764279 20482 consensus_queue.cc:1048] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 1.869s
I20260709 13:21:12.802807 20589 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:12.754459 20564 consensus_queue.cc:1048] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.048s
I20260709 13:21:14.639147 12700 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:14.636333 19829 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:14.617743 20635 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:14.649538 12700 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:21:14.658522 20645 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.659173 20645 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.652935 20662 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:14.658417 19829 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:14.661175 19829 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.658635 20564 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:14.665488 20564 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.667609 20564 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:14.668056 20493 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:14.673738 12703 raft_consensus.cc:3055] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:14.674170 12703 raft_consensus.cc:740] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:14.674962 12703 consensus_queue.cc:260] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:14.676307 12703 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:14.656760 20635 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:14.661931 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 93429us
I20260709 13:21:14.687348 12672 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:14.689051 19829 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:14.688153 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:14.704391 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:14.705524 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:14.706352 12776 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:21:14.707897 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:14.692044 12703 raft_consensus.cc:1275] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:21:14.693395 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:14.694607 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:14.737643 12777 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 18.
I20260709 13:21:14.695853 12778 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 23. Current term is 24. Ops: []
I20260709 13:21:14.696534 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:14.738444 12775 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 20.
I20260709 13:21:14.697817 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:14.700342 20669 raft_consensus.cc:2804] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:14.701781 20645 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:14.702675 12779 raft_consensus.cc:1275] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:21:14.709071 20662 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:14.714962 20635 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:14.719596 20560 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:14.723194 20652 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.724117 20563 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:14.724606 20482 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:14.725057 20581 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:14.748201 20581 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:14.729624 20586 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:14.732227 20561 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:12.806515 20584 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:14.754132 13348 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:14.754679 20584 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:14.746861 20660 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.764918 20660 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:14.747228 20662 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.750727 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:14.771819 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:14.772132 12778 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 14.
I20260709 13:21:14.746978 20652 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:14.759660 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:14.757692 20669 raft_consensus.cc:697] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 21 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:14.756232 20560 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:14.789492 20560 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:14.791682 20560 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:14.751515 20482 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:14.792055 20659 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
W20260709 13:21:14.793339 12563 debug-util.cc:398] Leaking SignalData structure 0x7b0800349460 after lost signal to thread 12566
I20260709 13:21:14.793622 20482 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:14.750272 20563 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:14.795610 20563 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:14.796888 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:14.736516 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:14.771127 12701 raft_consensus.cc:1275] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 27 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 18. (term mismatch)
I20260709 13:21:14.755153 20561 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.814653 12701 pending_rounds.cc:85] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 17
I20260709 13:21:14.816210 12701 pending_rounds.cc:107] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 26.18
I20260709 13:21:14.816457 12701 raft_consensus.cc:2889] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:21:14.772925 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 66333.5us
I20260709 13:21:14.773847 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:14.775960 12703 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:21:14.776741 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:14.777942 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:14.779532 20662 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:14.779897 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:14.781916 20652 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:14.788378 20669 consensus_queue.cc:237] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:14.799635 20425 consensus_meta.cc:287] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 2.109s	user 0.012s	sys 0.024s
I20260709 13:21:14.827128 20425 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:14.801820 20579 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.766492 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:14.758633 20586 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:14.829780 20579 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.830713 20586 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:14.831382 20586 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:14.832861 20579 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:14.818832 12778 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 21.
I20260709 13:21:14.817914 12704 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 14.
I20260709 13:21:14.829339 20425 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:14.834102 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:14.821158 20561 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:14.851794 20561 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:14.853305 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:14.853880 20660 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:14.838016 12563 debug-util.cc:398] Leaking SignalData structure 0x7b0800349880 after lost signal to thread 12597
I20260709 13:21:14.851783 13348 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:14.812520 20254 raft_consensus.cc:3060] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:14.874403 12672 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:14.881084 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:14.884843 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:14.881951 12783 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 20.
I20260709 13:21:14.887696 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:14.888957 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:14.877936 20665 consensus_queue.cc:1048] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.014s
I20260709 13:21:14.880934 20554 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:14.898919 12709 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:21:14.896255 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:14.906836 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:14.919463 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:14.888126 20660 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:14.883230 20670 consensus_queue.cc:1059] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.040s
I20260709 13:21:14.925369 19829 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:21:14.927150 13349 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.123s	user 0.009s	sys 0.026s
W20260709 13:21:14.927474 13349 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.123s	user 0.009s	sys 0.026s
W20260709 13:21:14.928591 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 2.12607e+06us
W20260709 13:21:14.929252 13349 leader_election.cc:336] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:14.927456 19829 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:14.919037 20561 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:14.916137 20554 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:14.921535 20660 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:14.936771 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:14.937520 12701 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:21:14.943908 20645 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.944432 20645 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:14.945406 20564 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.946121 20660 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:14.947844 20645 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:14.945910 20564 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:14.950564 13349 leader_election.cc:336] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:14.951839 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:14.950124 20670 consensus_queue.cc:1243] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer 3d539f8d95d343d8b1696803f18b30d2 log is divergent from this leader: its last log entry 24.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 13:21:14.931232 20561 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:14.978489 19829 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:14.932224 20563 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:14.934715 20554 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:14.954419 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:14.976797 12702 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 24.
I20260709 13:21:14.983796 19829 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:14.984220 19829 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:21:14.979952 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 45143us
I20260709 13:21:14.984864 19829 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:14.984223 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:14.951934 20652 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:14.986523 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:21:14.986594 13349 leader_election.cc:336] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:14.987505 12701 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 23.
I20260709 13:21:14.987481 13349 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:14.989874 20468 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:14.990315 20468 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:14.990670 20468 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:14.994418 20652 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:14.985775 20563 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:14.998091 20586 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.003013 20586 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:14.976511 20649 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 13:21:15.007476 20558 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:15.009792 20482 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:21:14.979190 20561 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:15.011318 19829 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:15.028312 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 44146.3us
I20260709 13:21:15.038619 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:15.045181 19829 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:15.046171 12783 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 24.
I20260709 13:21:14.976059 20645 consensus_queue.cc:1048] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.018s
I20260709 13:21:15.047732 12672 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:15.058311 20652 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:15.059201 20468 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:15.059839 20669 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:21:15.060220 20669 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:15.060596 20669 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:15.061306 20468 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:15.047546 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 61085.8us
I20260709 13:21:15.037809 20651 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:21:15.068548 20554 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:12.880066 20571 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:15.069924 20560 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:15.070322 20650 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:15.070461 20560 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.070849 20650 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:15.072837 20560 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:15.073021 20650 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:15.073727 12780 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 17.
I20260709 13:21:15.075357 20563 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:15.075572 20584 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.079627 20660 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:15.082573 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:15.083216 12775 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:21:15.084558 12781 raft_consensus.cc:1240] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:15.086823 20581 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:15.091856 20564 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:15.097920 12709 raft_consensus.cc:3055] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:21:15.098392 12709 raft_consensus.cc:740] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:15.104504 20561 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.106004 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:15.108239 20660 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:15.111352 12563 debug-util.cc:398] Leaking SignalData structure 0x7b08003496c0 after lost signal to thread 12658
W20260709 13:21:15.111994 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 2.3068e+06us
W20260709 13:21:15.123356 12563 debug-util.cc:398] Leaking SignalData structure 0x7b0800349cc0 after lost signal to thread 12659
I20260709 13:21:15.099200 12709 consensus_queue.cc:260] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.124490 12709 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:15.118848 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:15.085062 12672 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:21:15.124557 12563 debug-util.cc:398] Leaking SignalData structure 0x7b080036a220 after lost signal to thread 12733
W20260709 13:21:15.126423 12563 debug-util.cc:398] Leaking SignalData structure 0x7b080036a260 after lost signal to thread 12734
I20260709 13:21:15.117592 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:15.127439 12563 debug-util.cc:398] Leaking SignalData structure 0x7b080028ce80 after lost signal to thread 12807
I20260709 13:21:15.127605 13380 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
W20260709 13:21:15.127887 12563 debug-util.cc:398] Leaking SignalData structure 0x7b0800314280 after lost signal to thread 12808
I20260709 13:21:15.127478 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:15.129595 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:15.131783 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:15.131891 12783 raft_consensus.cc:2393] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 15: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:21:15.131960 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:15.134225 12707 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:21:15.137754 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:15.138285 12708 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:21:15.139273 12563 debug-util.cc:398] Leaking SignalData structure 0x7b08000d8ce0 after lost signal to thread 13410
W20260709 13:21:15.139706 12563 debug-util.cc:398] Leaking SignalData structure 0x7b080022a3c0 after lost signal to thread 13411
W20260709 13:21:15.162088 12563 debug-util.cc:398] Leaking SignalData structure 0x7b080035eaa0 after lost signal to thread 20607
I20260709 13:21:15.147420 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:15.122447 12523 catalog_manager.cc:5697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 20 to 21, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 21 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:15.151654 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:15.154065 12775 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 13:21:15.150417 12704 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
I20260709 13:21:15.158844 20584 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:15.160590 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:15.146898 12672 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:15.164857 12563 debug-util.cc:398] Leaking SignalData structure 0x7b0800370b00 after lost signal to thread 20629
I20260709 13:21:15.122193 20554 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:15.122819 20622 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:15.166414 12706 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:21:15.172259 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:15.125054 20560 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:15.166576 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:15.177320 13380 raft_consensus.cc:2393] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 21: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:21:15.176561 12523 catalog_manager.cc:5697] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 19 to 20, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 20 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:15.167675 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:15.168121 13382 raft_consensus.cc:1275] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:21:15.178725 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:15.168373 12707 raft_consensus.cc:2393] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 15: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:15.169694 12700 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:15.169826 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:15.179324 13381 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
I20260709 13:21:15.131625 20659 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:15.133428 20640 raft_consensus.cc:3060] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:15.150442 20564 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:21:15.167202 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:15.176482 20352 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.166606 20604 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:15.180372 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:15.221937 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:15.150508 20571 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:21:15.186547 13384 raft_consensus.cc:2393] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 21: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:21:15.194797 13379 raft_consensus.cc:1275] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:21:15.224625 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:15.199786 12523 catalog_manager.cc:5697] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:15.202855 12528 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54800 (request call id 323) took 8942 ms. Trace:
I20260709 13:21:15.206034 20375 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.206501 20493 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:15.206741 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49dad260ffe242d2b4e0976f119a19d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:21:15.207262 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 83197.3us
I20260709 13:21:15.207598 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:15.211421 12773 tablet_service.cc:1558] Processing DeleteTablet for tablet f2f0a3339d324fd4902b6ff970ff0272 with delete_type TABLET_DATA_TOMBSTONED (TS 3d539f8d95d343d8b1696803f18b30d2 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:50664
I20260709 13:21:15.168494 20623 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.216492 12708 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 22.
I20260709 13:21:15.215932 12700 raft_consensus.cc:2468] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
I20260709 13:21:15.170428 20239 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.181512 20659 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:15.227741 20571 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:15.232465 12709 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 17. (term mismatch)
I20260709 13:21:15.246735 12709 pending_rounds.cc:85] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 16
I20260709 13:21:15.247125 12709 pending_rounds.cc:107] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 23.17
I20260709 13:21:15.247378 12709 raft_consensus.cc:2889] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:15.234464 12523 catalog_manager.cc:5697] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 14 to 17, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 17 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:15.212255 20564 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:15.252681 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e0d0566acd7465f87e20053c75e28fa" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:15.255028 12523 catalog_manager.cc:5697] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:15.238456 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18205d974a7c4592aec0153ef3e44c97" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:15.222079 13377 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
W20260709 13:21:15.242813 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 54628.2us
I20260709 13:21:15.246236 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:15.257382 12701 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 23.
W20260709 13:21:15.259768 12598 connection.cc:256] RPC call timeout handler was delayed by -1.3672642679998717s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
W20260709 13:21:15.264354 12598 connection.cc:256] RPC call timeout handler was delayed by -1.2105890209998051s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.225s for a response
I20260709 13:21:15.228297 20600 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:15.232983 20621 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.269712 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:15.270272 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:15.270828 13349 connection.cc:256] RPC call timeout handler was delayed by -1.4874171239998759s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19530000000000003s for a response
I20260709 13:21:15.271037 13380 raft_consensus.cc:2410] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 21 because replica has last-logged OpId of term: 20 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 16.
W20260709 13:21:15.271226 13349 connection.cc:256] RPC call timeout handler was delayed by -1.4086686590001136s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
I20260709 13:21:15.262048 13378 raft_consensus.cc:1240] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:21:15.262724 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:15.246141 20659 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:15.261693 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:15.266562 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:15.226564 12528 rpcz_store.cc:276] 0709 13:21:06.260144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:06.260355 (+   211us) service_pool.cc:224] Handling call
0709 13:21:15.202795 (+8942440us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:11.918732 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:11.919716 (+   984us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:11.919767 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:21:11.919785 (+    18us) tablet.cc:662] Decoding operations
0709 13:21:11.922606 (+  2821us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:11.922706 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:11.922734 (+    28us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:21:11.923855 (+  1121us) tablet.cc:701] Row locks acquired
0709 13:21:11.923911 (+    56us) write_op.cc:260] PREPARE: finished
0709 13:21:11.924073 (+   162us) write_op.cc:270] Start()
0709 13:21:11.924221 (+   148us) write_op.cc:276] Timestamp: P: 1783603271924010 usec, L: 0
0709 13:21:11.924238 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:21:11.930262 (+  6024us) log.cc:844] Serialized 4858 byte log entry
0709 13:21:12.185317 (+255055us) op_driver.cc:464] REPLICATION: finished
0709 13:21:12.293641 (+108324us) write_op.cc:301] APPLY: starting
0709 13:21:12.300494 (+  6853us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:12.403689 (+103195us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:12.403783 (+    94us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:12.436742 (+ 32959us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:12.437012 (+   270us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=24,mrs_lookups=0
0709 13:21:12.437065 (+    53us) write_op.cc:312] APPLY: finished
0709 13:21:12.437444 (+   379us) log.cc:844] Serialized 113 byte log entry
0709 13:21:12.438314 (+   870us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:12.438451 (+   137us) write_op.cc:454] Released schema lock
0709 13:21:12.438505 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:12.450333 (+ 11828us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:12.450370 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":895943,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":108041,"apply.run_cpu_time_us":20584,"apply.run_wall_time_us":156699,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54335,"num_ops":12,"prepare.queue_time_us":257316,"prepare.run_cpu_time_us":8411,"prepare.run_wall_time_us":220045,"raft.queue_time_us":34124,"raft.run_cpu_time_us":77319,"raft.run_wall_time_us":103224,"replication_time_us":260915,"thread_start_us":355585,"threads_started":4,"wal-append.queue_time_us":206727,"wal-append.run_cpu_time_us":25486,"wal-append.run_wall_time_us":2264249}]]}
I20260709 13:21:15.266611 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:15.250108 20641 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.239650 20425 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:15.273538 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:15.274516 13379 raft_consensus.cc:1240] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 27 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 26. Current term is 27. Ops: []
I20260709 13:21:15.274989 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5ddded6fd34f278dade99f2086c4e6" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:15.187007 20560 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:15.257826 20298 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 24 current term: 24 is higher.
I20260709 13:21:15.187892 20622 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.249197 20623 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.278371 13376 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 21. Current term is 22. Ops: []
I20260709 13:21:15.233740 20604 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.280285 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:15.329304 12708 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 24.
I20260709 13:21:15.250735 20571 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:15.267362 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:15.330552 12701 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:15.281307 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:15.258316 20564 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:15.281793 12707 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:21:15.242259 20493 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.229750 20352 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:15.282608 13382 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:15.285303 13377 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:21:15.239176 20375 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.278896 12523 catalog_manager.cc:5697] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 13 to 14, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 14 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:15.315132 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:15.338896 13376 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 22.
I20260709 13:21:15.339020 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:15.339727 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:15.309072 20689 log.cc:927] Time spent T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:21:15.325552 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:15.344102 12523 catalog_manager.cc:5697] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:15.344751 12700 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 18.
I20260709 13:21:15.327162 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:15.309420 20647 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:15.250028 20239 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.341264 13376 raft_consensus.cc:2393] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 20: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:21:15.285032 20650 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
W20260709 13:21:15.265162 12598 connection.cc:256] RPC call timeout handler was delayed by -1.088437372000044s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1501s for a response
I20260709 13:21:15.348472 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:15.349974 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:15.343637 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:15.309764 20561 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:15.345553 12523 catalog_manager.cc:5697] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:15.351411 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:15.357656 13380 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:21:15.351754 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:15.358613 12774 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:15.311676 20560 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:15.360224 13376 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:21:15.315172 13381 raft_consensus.cc:2410] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for term 27 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 18.
I20260709 13:21:15.286336 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:15.362560 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:15.362371 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:15.363255 12703 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 24.
I20260709 13:21:15.363332 13380 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 21.
I20260709 13:21:15.275475 20600 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.345584 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:15.367075 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:21:15.368854 11517 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 12659)
I20260709 13:21:15.278800 20621 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.378844 12774 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 21.
I20260709 13:21:15.367082 13349 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:15.360874 12523 catalog_manager.cc:5697] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:15.369751 13383 raft_consensus.cc:2376] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 15. Re-sending same reply.
I20260709 13:21:15.358644 20650 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:15.373703 13380 raft_consensus.cc:2393] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 20: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
W20260709 13:21:15.377128 12673 leader_election.cc:336] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:15.326625 20425 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:15.389883 13349 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:15.390172 12523 catalog_manager.cc:5697] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 12 to 14, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 14 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:15.389097 12673 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:15.396543 12523 catalog_manager.cc:5697] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 18 to 20, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 20 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:15.402176 13349 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:15.404824 12701 raft_consensus.cc:2468] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
I20260709 13:21:15.406852 12523 catalog_manager.cc:5697] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:15.313023 20641 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.394301 20554 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:15.418157 20621 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:15.418387 20554 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:21:15.421805 12702 raft_consensus.cc:1240] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 21. Current term is 22. Ops: []
I20260709 13:21:15.424594 12702 consensus_queue.cc:237] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.426955 20641 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:15.428121 12523 catalog_manager.cc:5697] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e reported cstate change: config changed from index 3 to 11, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New cstate: current_term: 15 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:15.430336 12523 catalog_manager.cc:5697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 18 to 21, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 21 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:15.366250 20649 consensus_queue.cc:1048] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.048s
I20260709 13:21:15.442621 20622 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:15.444375 20650 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:15.409502 20589 raft_consensus.cc:2804] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:15.446700 20589 raft_consensus.cc:697] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:15.436084 12701 raft_consensus.cc:1240] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:15.441082 12523 catalog_manager.cc:5697] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 20 to 22, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 22 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:15.450040 12523 catalog_manager.cc:5697] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e reported cstate change: term changed from 16 to 18, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 18 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:15.456347 12598 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:15.468241 20425 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:15.468466 20493 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:15.466605 20589 consensus_queue.cc:237] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.470714 12598 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:15.472781 12598 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:15.473237 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:15.407408 20669 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:15.473917 12774 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:15.488656 20352 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:15.489761 20604 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:15.490664 20623 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:15.493763 12774 raft_consensus.cc:2468] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 22.
I20260709 13:21:15.498152 20669 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:15.499015 20640 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for earlier term 15. Current term is 16.
I20260709 13:21:15.478345 20425 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:21:15.481315 20493 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:21:15.524370 12783 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:21:15.527184 13348 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:15.531669 12747 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:15.475469 20622 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:21:15.541649 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 65360.5us
I20260709 13:21:15.543272 12598 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:15.504427 20493 raft_consensus.cc:697] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 23 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:15.551836 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:21:15.570640 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 290764us
I20260709 13:21:15.570878 12598 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:15.540884 20701 tablet_replica.cc:333] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:15.571338 12746 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:15.581907 12783 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 20.
W20260709 13:21:15.585008 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 182895us
I20260709 13:21:15.538059 20467 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:15.585808 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:15.585377 12598 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:15.587656 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:15.483981 20254 consensus_queue.cc:1059] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.086s
I20260709 13:21:15.588361 12783 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:21:15.591254 13348 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:15.570261 20640 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:21:15.600801 19829 consensus_queue.cc:1048] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:21:15.570123 20622 raft_consensus.cc:697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:15.600332 12746 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:15.586412 20641 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:15.601965 13348 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:15.595363 20623 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:21:15.549882 20425 consensus_queue.cc:1048] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:21:15.589318 20467 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:15.600370 20554 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:15.611848 13349 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:15.586633 20352 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.020s
W20260709 13:21:15.613958 12747 consensus_peers.cc:597] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:15.617036 12747 consensus_peers.cc:597] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:15.617801 12747 leader_election.cc:336] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:15.618511 20715 raft_consensus.cc:2804] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:21:15.618887 12747 leader_election.cc:336] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:15.619174 20715 raft_consensus.cc:697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:15.619472 12747 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:15.620201 20715 consensus_queue.cc:237] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.622273 12775 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:21:15.623697 20645 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:21:15.625069 12747 consensus_peers.cc:597] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:15.625249 12779 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:21:15.626355 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:21:15.625661 12747 consensus_peers.cc:597] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:15.643028 20719 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:15.649991 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:15.608690 20586 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:15.613150 20554 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:15.611516 20640 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.674011 20640 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:15.675261 20711 log.cc:927] Time spent T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:21:15.676249 20640 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:15.676417 20696 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:15.677023 20696 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.679857 20696 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:15.614576 20600 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:15.611312 20604 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:15.613348 20641 raft_consensus.cc:697] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:15.612089 20467 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:15.613346 20623 raft_consensus.cc:697] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 22 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:15.670284 20685 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:15.684962 20564 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:15.665702 20493 consensus_queue.cc:237] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:15.686081 20702 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.103s
I20260709 13:21:15.686731 20690 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.098s
I20260709 13:21:15.687812 20697 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:15.681840 20586 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.682078 20554 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:15.688294 20710 raft_consensus.cc:2804] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:15.704054 12781 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 24. Current term is 25. Ops: []
I20260709 13:21:15.696902 20685 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:15.706940 20564 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:15.709807 20622 consensus_queue.cc:237] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.676306 20698 raft_consensus.cc:2804] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:15.762508 20703 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.763432 20600 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:15.763854 20621 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.764441 20425 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.764621 12597 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:15.762760 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 150999us
W20260709 13:21:15.770537 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 56320.3us
I20260709 13:21:15.748335 20697 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:15.811096 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 46567.4us
I20260709 13:21:15.767822 20685 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:15.694521 20604 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.772181 20710 raft_consensus.cc:697] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:21:15.819250 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:15.816926 20696 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:21:15.864435 12597 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:15.817368 13349 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:21:15.934958 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 117667us
W20260709 13:21:15.936707 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 72376.4us
W20260709 13:21:15.816804 20659 consensus_meta.cc:287] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.531s	user 0.005s	sys 0.004s
I20260709 13:21:15.938527 13349 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:15.942704 20694 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:15.809661 20698 raft_consensus.cc:697] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:15.944514 20604 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:15.946388 20571 consensus_meta.cc:287] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.602s	user 0.010s	sys 0.000s
I20260709 13:21:15.946683 20665 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:21:15.947152 20663 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:15.953727 20722 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.816442 20697 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:16.003504 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:15.960685 13349 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:15.817144 20425 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:16.004767 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 44181.4us
I20260709 13:21:16.006022 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:16.006106 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:15.966014 20665 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:15.988010 20723 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:16.014019 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:15.988747 20698 consensus_queue.cc:237] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.021546 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:15.991940 13379 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
W20260709 13:21:16.022769 12747 consensus_peers.cc:597] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:15.992887 12780 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:15.993487 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:16.023627 12747 leader_election.cc:336] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:15.994246 20663 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:21:15.994270 20640 consensus_queue.cc:1059] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:21:15.808878 20703 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.880736 20623 consensus_queue.cc:237] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.954907 20696 raft_consensus.cc:697] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 27 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:15.997470 20710 consensus_queue.cc:237] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.965576 20659 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:15.811709 20600 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:15.958535 20694 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } attrs { replace: true } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:15.967914 20571 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:16.006274 20425 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:16.008669 13377 raft_consensus.cc:2468] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 24.
I20260709 13:21:15.880951 20641 consensus_queue.cc:237] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.085292 20571 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:16.009783 13349 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:16.020228 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:16.021072 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:16.024209 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:16.025977 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:16.030777 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:16.031998 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:16.020256 20722 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.032296 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:16.035068 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:16.020771 20665 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:16.045781 12780 raft_consensus.cc:1240] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:15.859773 20621 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:16.033246 20719 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:21:16.082134 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:16.082544 20696 consensus_queue.cc:237] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.083367 20600 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:16.009097 20220 consensus_queue.cc:1059] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.038s
I20260709 13:21:16.094396 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:16.085505 20467 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.096626 12779 raft_consensus.cc:2393] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 21: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:16.102079 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:16.116909 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:16.116988 12706 raft_consensus.cc:2393] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 19: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
W20260709 13:21:16.118201 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 108542us
I20260709 13:21:16.134212 13349 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:16.135121 12780 raft_consensus.cc:1240] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 21. Current term is 22. Ops: []
W20260709 13:21:16.130395 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 106812us
I20260709 13:21:16.130671 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:16.136889 12747 leader_election.cc:336] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:16.138167 12747 leader_election.cc:336] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.138473 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:16.138543 12747 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:16.139422 20703 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:16.044838 20604 consensus_queue.cc:1059] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 13:21:16.140122 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:16.135839 13349 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:16.143649 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:16.146358 20727 log.cc:927] Time spent T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:21:16.145377 20685 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.046253 20493 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.084s
I20260709 13:21:16.149933 12774 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:16.154340 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:16.156903 12747 leader_election.cc:336] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:16.130954 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 91708.6us
I20260709 13:21:16.157581 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:16.127543 20715 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:21:16.158066 12597 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:16.171216 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:16.140132 20375 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:16.176995 12747 leader_election.cc:336] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.154945 20690 raft_consensus.cc:3055] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:21:16.185439 12747 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:16.177687 12597 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:16.149771 20698 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:16.192417 20467 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:16.192240 20722 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:16.193405 20721 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:16.155591 20685 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:16.204397 12747 leader_election.cc:336] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.209209 12747 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:16.173537 20571 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:16.211982 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:16.212657 13377 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:21:16.213833 12747 leader_election.cc:336] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.214391 12747 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:16.183398 20352 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:21:16.184331 20584 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:16.188877 20690 raft_consensus.cc:740] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:21:16.210206 20652 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.855s	user 0.020s	sys 0.031s
I20260709 13:21:16.221832 20640 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:16.226712 20690 consensus_queue.cc:260] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.228364 20737 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:16.229066 20713 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:16.229534 20713 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:16.229919 20713 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:16.230841 20729 log.cc:927] Time spent T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.289s	user 0.001s	sys 0.001s
I20260709 13:21:16.210408 20721 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:16.231300 20721 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:16.231987 20733 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:16.238081 20220 consensus_queue.cc:1059] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:21:16.239665 20659 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:16.240623 20623 consensus_queue.cc:1059] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:16.241354 12700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 3377) took 1081 ms. Trace:
I20260709 13:21:16.241588 20375 consensus_queue.cc:1059] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:21:16.241760 20623 raft_consensus.cc:3055] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:16.242379 20623 raft_consensus.cc:740] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:21:16.243376 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 874333us
I20260709 13:21:16.244513 20239 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:16.243505 12706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42490 (request call id 3376) took 1087 ms. Trace:
I20260709 13:21:16.245167 12706 rpcz_store.cc:276] 0709 13:21:15.156458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:15.266318 (+109860us) service_pool.cc:224] Handling call
0709 13:21:16.243327 (+977009us) spinlock_profiling.cc:243] Waited 839 ms on lock 0x7b54001090d4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b62fa 00007f4c72377b17 00007f4c6b8f26d4 00007f4c6b8f2662 00007f4c6b8f2591 00007f4c6b8f250c 00007f4c6b8f17c2 00007f4c6b6ed48c 00007f4c6b6ec8c6 00007f4c6b6ec26f
0709 13:21:16.243450 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1930569088}
I20260709 13:21:16.245781 20670 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:16.246842 12709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42490 (request call id 3378) took 1077 ms. Trace:
I20260709 13:21:16.247155 12709 rpcz_store.cc:276] 0709 13:21:15.169053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:15.388205 (+219152us) service_pool.cc:224] Handling call
0709 13:21:15.388514 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:15.388717 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:21:15.405856 (+ 17139us) log.cc:844] Serialized 14 byte log entry
0709 13:21:16.246522 (+840666us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:16.246572 (+    50us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:16.246624 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:16.246689 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:16.246791 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":840444,"prepare.queue_time_us":17353,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":14200}
W20260709 13:21:16.248107 12675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:16.248267 20707 log.cc:927] Time spent T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.703s	user 0.009s	sys 0.030s
I20260709 13:21:16.215919 20685 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:16.248921 20623 consensus_queue.cc:260] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:16.249840 20650 raft_consensus.cc:493] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.250139 20623 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:16.250365 20650 raft_consensus.cc:515] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.251601 20586 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:16.252550 20650 leader_election.cc:290] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:16.253271 20554 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.253777 20717 raft_consensus.cc:3055] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:21:16.254225 20717 raft_consensus.cc:740] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:16.255501 20554 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:16.258206 20730 log.cc:927] Time spent T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 13:21:16.259543 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:16.260332 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:16.260723 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:16.262076 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:16.262825 12707 raft_consensus.cc:2393] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 25: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:16.241809 12700 rpcz_store.cc:276] 0709 13:21:15.159722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:15.347752 (+188030us) service_pool.cc:224] Handling call
0709 13:21:15.353162 (+  5410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:15.353392 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:21:15.366364 (+ 12972us) log.cc:844] Serialized 14 byte log entry
0709 13:21:16.240779 (+874415us) spinlock_profiling.cc:243] Waited 8.39 ms on lock 0x7b1800249580. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a23fc0a 00007f4c7397f219 00007f4c6a2c7a40 00007f4c6a2c24e4 00007f4c6a2c2455 00007f4c6d3aa060 00007f4c6d3bc716 00007f4c6d3cd287 00007f4c6d3cd208
0709 13:21:16.240973 (+   194us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:16.241048 (+    75us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:16.241101 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:16.241179 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:16.241300 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2570,"prepare.queue_time_us":14955,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":875540,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":179,"spinlock_wait_cycles":19307392}
I20260709 13:21:16.255036 20717 consensus_queue.cc:260] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.271185 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:16.273707 12597 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:16.273938 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 48541.3us
W20260709 13:21:16.274459 12673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:16.275950 12597 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:16.276966 13376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:16.277216 12597 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:16.262176 20621 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:16.237102 20665 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.278923 20738 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:16.279323 13379 raft_consensus.cc:1240] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 23. Current term is 24. Ops: []
I20260709 13:21:16.279078 20621 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:21:16.277897 12747 leader_election.cc:336] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.282780 12747 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:16.281646 12673 leader_election.cc:336] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.283955 12597 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:16.284274 12673 leader_election.cc:336] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.259914 20694 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:16.288022 20665 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 24 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:16.291280 12597 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:16.296474 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:16.297245 12700 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:16.302950 13377 raft_consensus.cc:2468] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 21.
W20260709 13:21:16.225354 20652 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.879s	user 0.020s	sys 0.031s
I20260709 13:21:16.307932 13378 raft_consensus.cc:1240] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:16.281553 20641 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:16.308856 12597 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:16.290439 20621 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:16.312265 12746 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:16.302706 20663 raft_consensus.cc:2804] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:21:16.318109 12597 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:16.319175 20685 raft_consensus.cc:2804] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:16.303025 20694 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:16.302800 20737 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:16.306953 20717 raft_consensus.cc:3060] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:16.325440 12597 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:16.308095 20738 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:21:16.324760 12673 leader_election.cc:336] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:16.334803 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 178922us
I20260709 13:21:16.334098 12597 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:16.349622 12673 leader_election.cc:336] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.350255 13382 raft_consensus.cc:1240] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:16.317623 20493 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:21:16.351794 12700 raft_consensus.cc:2468] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 24.
I20260709 13:21:16.323537 20663 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:16.362051 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:16.360703 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:16.366200 12776 raft_consensus.cc:1240] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 23. Current term is 24. Ops: []
I20260709 13:21:16.367303 12774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:16.369336 11517 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 12659)
I20260709 13:21:16.370301 20713 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.372161 13385 raft_consensus.cc:1240] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:21:16.364382 20425 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:16.333284 20641 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:16.340375 20733 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:16.333470 20685 raft_consensus.cc:697] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:16.383024 12747 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:16.384323 20581 raft_consensus.cc:2804] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:16.384863 20581 raft_consensus.cc:697] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:16.385788 20581 consensus_queue.cc:237] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:16.387637 20713 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:16.358012 20621 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:16.364542 20663 raft_consensus.cc:3060] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:16.326503 20737 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:16.362191 20697 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.411211 20493 consensus_queue.cc:1059] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.012s
I20260709 13:21:16.424135 20493 raft_consensus.cc:3055] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:21:16.424628 20493 raft_consensus.cc:740] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:16.426002 20685 consensus_queue.cc:237] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.430859 20737 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:16.413307 20690 raft_consensus.cc:3060] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:16.441704 12774 raft_consensus.cc:2410] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 27 because replica has last-logged OpId of term: 26 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 12.
I20260709 13:21:16.401726 20641 consensus_queue.cc:1059] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.018s
I20260709 13:21:16.445495 20621 raft_consensus.cc:3055] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:21:16.445957 20621 raft_consensus.cc:740] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:16.446030 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:16.446753 13377 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:16.446938 20621 consensus_queue.cc:260] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:16.448566 20662 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:16.417518 12747 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:16.413872 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:16.449095 20662 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.454236 20621 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:16.454264 20721 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:16.460091 12673 leader_election.cc:336] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.467372 12777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:16.501493 12673 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:21:16.499776 13349 leader_election.cc:336] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.502589 13349 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:16.467774 12776 raft_consensus.cc:1275] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
W20260709 13:21:16.469810 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:16.472429 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:21:16.474291 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 60529.4us
I20260709 13:21:16.506024 12709 raft_consensus.cc:1240] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:16.506224 12707 raft_consensus.cc:1240] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:16.493191 20702 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:21:16.474709 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 57266.8us
W20260709 13:21:16.508906 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 48869.3us
W20260709 13:21:16.509933 12673 leader_election.cc:336] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.510432 12673 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:16.492549 20662 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:16.509212 12747 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:16.442982 20493 consensus_queue.cc:260] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.516521 12709 raft_consensus.cc:1275] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:21:16.507426 13377 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 21.
I20260709 13:21:16.508512 12777 raft_consensus.cc:2393] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 25: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:21:16.515036 12672 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:16.518525 20493 raft_consensus.cc:3060] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:16.518368 20298 consensus_queue.cc:1059] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 13:21:16.519506 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:16.519562 20298 raft_consensus.cc:3055] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:21:16.520164 20298 raft_consensus.cc:740] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:16.521445 20738 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.521663 20686 raft_consensus.cc:2804] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:21:16.522297 20686 raft_consensus.cc:697] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 24 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:16.523188 20239 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:16.523281 20686 consensus_queue.cc:237] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:16.524652 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:16.523908 13381 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:16.521979 20738 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.513680 12775 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:21:16.498873 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:16.533416 12673 leader_election.cc:336] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.534125 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:16.534365 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:16.534894 12783 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:16.533897 12673 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:21:16.536209 12673 consensus_peers.cc:597] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:16.537186 12672 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:16.516219 20670 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:16.498059 20623 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:16.537909 20669 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:21:16.537948 20641 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:16.538110 20650 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:21:16.543542 12780 raft_consensus.cc:1240] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:16.543691 20662 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:16.543187 20647 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:16.523515 20721 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:16.516695 20714 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:16.523842 20239 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:16.532384 12746 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:16.521111 20298 consensus_queue.cc:260] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.551335 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:16.551151 12672 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:16.557034 13349 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:16.559504 12672 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:16.544057 20663 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.562110 20719 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:16.562628 20719 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:16.562747 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:16.563081 20733 raft_consensus.cc:2804] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:16.563568 20733 raft_consensus.cc:697] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:16.564467 20733 consensus_queue.cc:237] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:16.566972 20720 log.cc:927] Time spent T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.625s	user 0.002s	sys 0.000s
I20260709 13:21:16.516415 20690 consensus_queue.cc:1059] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 13:21:16.569722 20690 raft_consensus.cc:3055] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:21:16.570117 20690 raft_consensus.cc:740] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:16.570943 20690 consensus_queue.cc:260] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:16.571540 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:16.573012 12778 raft_consensus.cc:1275] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:21:16.573228 20662 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:16.549907 20647 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:16.574252 20647 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:16.574349 20589 raft_consensus.cc:3055] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:16.574867 20589 raft_consensus.cc:740] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:16.575804 20589 consensus_queue.cc:260] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.576894 20589 raft_consensus.cc:3060] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:21:16.577023 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:16.578985 20239 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:16.562832 20663 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 25 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:16.585770 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:16.586907 20703 raft_consensus.cc:3055] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:16.587347 20703 raft_consensus.cc:740] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:16.587771 12778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:16.588179 20703 consensus_queue.cc:260] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:16.589277 20703 raft_consensus.cc:3060] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:21:16.592442 12702 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.145s	user 0.012s	sys 0.015s
W20260709 13:21:16.611256 12702 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.164s	user 0.012s	sys 0.015s
I20260709 13:21:16.601754 13377 raft_consensus.cc:3055] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:16.604477 12783 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 18.
W20260709 13:21:16.605712 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 48786.5us
W20260709 13:21:16.607378 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 47939.8us
I20260709 13:21:16.612406 13377 raft_consensus.cc:740] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:16.615537 12672 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:16.622910 20713 consensus_queue.cc:1059] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:21:16.624396 20721 raft_consensus.cc:3055] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:21:16.624747 20721 raft_consensus.cc:740] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:16.625422 20670 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:16.626020 20670 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:16.625581 20721 consensus_queue.cc:260] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.626408 20670 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:16.611944 20714 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:21:16.627060 20714 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:16.627491 20714 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:16.629415 12672 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:16.638484 12672 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:16.639688 13349 consensus_peers.cc:597] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:16.641286 12672 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
W20260709 13:21:16.642145 13349 leader_election.cc:336] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.642535 13349 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:16.644474 20701 raft_consensus.cc:2243] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:16.645622 20701 raft_consensus.cc:2272] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:16.645938 20665 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:16.617758 20659 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:21:16.646205 13349 leader_election.cc:336] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:16.648363 13349 leader_election.cc:336] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:16.649155 13349 leader_election.cc:336] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.649508 13349 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:16.639689 13377 consensus_queue.cc:260] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.630780 20662 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
W20260709 13:21:16.650609 13349 leader_election.cc:336] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:16.695344 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:16.639923 20660 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:16.653189 12702 raft_consensus.cc:1240] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:16.704382 12597 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:16.667440 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 124913us
I20260709 13:21:16.652598 12709 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:16.682366 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "866523bf2d2f442f971d504267ed200f" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:16.715874 12708 raft_consensus.cc:3060] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:16.681891 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:16.710744 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:16.711127 12701 raft_consensus.cc:1240] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 24. Current term is 25. Ops: []
I20260709 13:21:16.639115 20710 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.717290 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:21:16.720643 12673 leader_election.cc:336] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:16.721604 12673 leader_election.cc:336] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:16.722420 12673 leader_election.cc:336] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.722790 12673 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:21:16.717960 12747 leader_election.cc:336] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.725715 12747 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:16.718616 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:16.716924 12779 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 21.
I20260709 13:21:16.720140 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:16.708863 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:16.720778 12703 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:21:16.728430 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:21:16.729526 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 78933.2us
W20260709 13:21:16.730448 13349 leader_election.cc:336] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.731086 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:16.731815 13349 leader_election.cc:336] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.733824 13349 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:21:16.739317 12673 consensus_peers.cc:597] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:16.740577 12673 leader_election.cc:336] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:16.727301 12747 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 144 similar messages]
W20260709 13:21:16.757385 12747 leader_election.cc:336] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.757972 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:16.759332 12747 leader_election.cc:336] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:16.760334 12747 leader_election.cc:336] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.760859 12747 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:16.763254 13349 leader_election.cc:336] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:16.764529 13349 leader_election.cc:336] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.647733 20650 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:21:16.767160 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9119721f3ee04810b1f277c06c966a45" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:16.782759 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:16.731767 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:16.731717 13384 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 21.
I20260709 13:21:16.767619 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:16.784344 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb247fe4862541ccadc22bb79417d66c" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:16.785477 12701 raft_consensus.cc:2410] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:21:16.699429 20298 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:21:16.654445 20659 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:16.654608 20665 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:21:16.772019 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:16.769109 12597 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:21:16.656554 20721 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:16.782945 13348 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:16.732383 20664 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:16.689695 20738 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:16.702240 20690 raft_consensus.cc:3060] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:16.689283 20697 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:16.715798 20660 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:16.767122 20467 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:16.806303 12703 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:21:16.754485 20622 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:21:16.766083 13349 leader_election.cc:336] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.761780 20651 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:16.653331 20723 consensus_queue.cc:1059] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.142s
I20260709 13:21:16.792815 20686 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:16.808573 20717 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:16.828636 20717 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:16.829002 20717 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:16.812798 20660 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:16.792599 20696 consensus_queue.cc:1059] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:21:16.807818 20659 raft_consensus.cc:697] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:16.831126 20696 raft_consensus.cc:3055] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:16.831740 20696 raft_consensus.cc:740] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:16.832135 20659 consensus_queue.cc:237] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.794909 20554 consensus_queue.cc:1048] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.023s
I20260709 13:21:16.769423 20563 raft_consensus.cc:2749] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:16.769279 20640 consensus_queue.cc:1059] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 13:21:16.836790 20640 raft_consensus.cc:3055] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:16.817973 13378 raft_consensus.cc:3055] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:16.837594 13378 raft_consensus.cc:740] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:16.818847 12704 raft_consensus.cc:1240] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:21:16.821693 13349 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:16.842407 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 76835.6us
I20260709 13:21:16.818814 20647 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.719425 20692 consensus_queue.cc:1059] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.092s
I20260709 13:21:16.825994 12703 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:21:16.808468 20739 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:16.832770 20696 consensus_queue.cc:260] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:16.771060 12747 leader_election.cc:336] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:16.888093 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 117068us
I20260709 13:21:16.837258 20640 raft_consensus.cc:740] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:16.768288 19829 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:21:16.856608 13349 leader_election.cc:336] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.755986 20710 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.896278 13378 consensus_queue.cc:260] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:16.889343 12747 leader_election.cc:336] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:16.899267 12747 leader_election.cc:336] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:16.900358 12747 leader_election.cc:336] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.900717 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:16.876027 12708 raft_consensus.cc:2468] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 21.
I20260709 13:21:16.894817 13349 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:16.910101 12709 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 22. Current term is 23. Ops: []
I20260709 13:21:16.847039 20554 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:16.914109 13378 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:16.914522 13377 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:16.924662 13377 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 13:21:16.943903 12747 leader_election.cc:336] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.956525 12523 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54792 (request call id 346) took 6248 ms. Trace:
W20260709 13:21:16.951455 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:16.952893 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:16.914812 20649 raft_consensus.cc:3055] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:21:16.963754 20714 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.964355 20649 raft_consensus.cc:740] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:21:16.928917 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 72353.6us
I20260709 13:21:16.967422 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:16.915284 20692 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:16.912940 20739 raft_consensus.cc:697] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:21:16.966784 13349 leader_election.cc:336] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.928352 20467 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:16.965233 20649 consensus_queue.cc:260] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.972867 12700 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:21:16.957257 12523 rpcz_store.cc:276] 0709 13:21:10.690038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.690340 (+   302us) service_pool.cc:224] Handling call
0709 13:21:16.938214 (+6247874us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:15.652887 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:15.653539 (+   652us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:15.653610 (+    71us) write_op.cc:435] Acquired schema lock
0709 13:21:15.653640 (+    30us) tablet.cc:662] Decoding operations
0709 13:21:15.679425 (+ 25785us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:15.679568 (+   143us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:15.679623 (+    55us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:21:15.995558 (+315935us) tablet.cc:701] Row locks acquired
0709 13:21:15.995583 (+    25us) write_op.cc:260] PREPARE: finished
0709 13:21:15.995741 (+   158us) write_op.cc:270] Start()
0709 13:21:15.995871 (+   130us) write_op.cc:276] Timestamp: P: 1783603275995682 usec, L: 0
0709 13:21:15.995895 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:21:15.996709 (+   814us) log.cc:844] Serialized 6260 byte log entry
0709 13:21:16.380096 (+383387us) op_driver.cc:464] REPLICATION: finished
0709 13:21:16.561292 (+181196us) write_op.cc:301] APPLY: starting
0709 13:21:16.561372 (+    80us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:16.572484 (+ 11112us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:16.572583 (+    99us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:16.655815 (+ 83232us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:16.656109 (+   294us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=32,mrs_lookups=0
0709 13:21:16.656150 (+    41us) write_op.cc:312] APPLY: finished
0709 13:21:16.656504 (+   354us) log.cc:844] Serialized 148 byte log entry
0709 13:21:16.669753 (+ 13249us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:16.670000 (+   247us) write_op.cc:454] Released schema lock
0709 13:21:16.670079 (+    79us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:16.670582 (+   503us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:16.670611 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":41242,"reported_tablets":34,"child_traces":[["op",{"apply.queue_time_us":177152,"apply.run_cpu_time_us":23733,"apply.run_wall_time_us":109278,"cfile_cache_hit":66,"cfile_cache_hit_bytes":71071,"num_ops":16,"prepare.queue_time_us":43111,"prepare.run_cpu_time_us":71941,"prepare.run_wall_time_us":474489,"raft.queue_time_us":179723,"raft.run_cpu_time_us":72188,"raft.run_wall_time_us":154322,"replication_time_us":384036,"thread_start_us":433533,"threads_started":4,"wal-append.queue_time_us":180361}]]}
W20260709 13:21:16.980680 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 202307us
I20260709 13:21:16.957150 20701 ts_tablet_manager.cc:1916] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:21:16.971848 12747 leader_election.cc:336] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.969434 20696 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:16.986930 12747 leader_election.cc:336] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.972548 20581 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:17.026057 20649 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:21:16.991438 20760 log.cc:927] Time spent T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 13:21:16.992388 20763 log.cc:927] Time spent T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:21:16.993705 12525 catalog_manager.cc:5697] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 17 to 20, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 20 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:16.995303 20647 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:16.997900 12695 tablet_service.cc:1558] Processing DeleteTablet for tablet 981fb2f435e24d06a81fc02cf4591cc5 with delete_type TABLET_DATA_TOMBSTONED (TS 639ab2db265f4747bb7afb93e833f530 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:49378
I20260709 13:21:16.999735 13380 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:21:17.000125 12749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:17.002666 20710 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:17.043829 19829 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:17.016415 20670 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.971143 20640 consensus_queue.cc:260] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:16.985448 20664 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.033s
W20260709 13:21:16.985836 13349 leader_election.cc:336] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:17.053676 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 67894.5us
I20260709 13:21:16.990046 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:17.031281 20714 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:17.044569 19829 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:17.041420 13378 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 13:21:17.049978 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 308614us
W20260709 13:21:17.077208 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 90308.2us
I20260709 13:21:17.077056 12525 catalog_manager.cc:5697] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 12 to 14, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 14 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:21:17.078290 12747 leader_election.cc:336] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:17.071614 13349 consensus_peers.cc:597] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:17.078838 12747 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:17.079458 12705 raft_consensus.cc:1240] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 16. Current term is 17. Ops: []
W20260709 13:21:17.080449 13349 leader_election.cc:336] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.090085 19829 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:17.089329 13349 leader_election.cc:336] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.094579 19829 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:21:17.094906 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 111474us
I20260709 13:21:17.095592 12672 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:17.099098 13382 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 27 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 13:21:17.099766 13384 raft_consensus.cc:1240] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 24. Current term is 25. Ops: []
W20260709 13:21:17.093036 13349 leader_election.cc:336] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:17.105494 13349 leader_election.cc:336] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.105988 13349 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:21:17.096215 12747 leader_election.cc:336] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.109190 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:17.088002 20697 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:17.107013 12525 catalog_manager.cc:5697] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:17.101070 20640 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:21:17.136238 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:17.131633 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:17.108590 20686 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:17.143754 12705 raft_consensus.cc:1275] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:21:17.116294 20651 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:21:17.115311 20571 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:21:17.078860 20719 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:17.101338 20622 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 13:21:17.106739 20690 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 13:21:17.138149 20662 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.069765 20739 consensus_queue.cc:237] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:17.129319 20663 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.046s
I20260709 13:21:17.187100 13378 raft_consensus.cc:1240] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 24. Current term is 25. Ops: []
I20260709 13:21:17.186606 12525 catalog_manager.cc:5697] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 15 to 16, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 16 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:17.158922 20664 raft_consensus.cc:2764] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader election decision vote started in defunct term 20: won
I20260709 13:21:17.189518 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:21:17.195048 13349 leader_election.cc:336] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:17.195854 13349 consensus_peers.cc:597] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:17.164532 20775 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.205956 20775 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:17.208863 20764 log.cc:927] Time spent T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:21:17.210832 13382 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:21:17.179836 20621 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.205857 13381 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:21:17.237121 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:17.282487 12747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:17.291448 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:21:17.287652 13349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:17.309695 12782 raft_consensus.cc:1275] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:21:17.185428 20662 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:17.317312 13349 leader_election.cc:336] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.282369 20650 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.335994 13382 raft_consensus.cc:1275] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:21:17.289453 20686 consensus_queue.cc:1059] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.048s
I20260709 13:21:17.299775 20732 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
W20260709 13:21:17.382269 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:17.283221 20621 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:17.328048 20697 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.099s
I20260709 13:21:17.338193 20650 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:17.441184 20781 tablet_replica.cc:333] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:17.433925 20733 raft_consensus.cc:3055] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 13:21:17.465039 20765 log.cc:927] Time spent T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
W20260709 13:21:17.446753 20717 consensus_meta.cc:287] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.617s	user 0.013s	sys 0.004s
I20260709 13:21:17.465934 20717 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:17.468222 20717 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:17.476743 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 381222us
I20260709 13:21:17.478837 12672 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:17.437377 20686 consensus_queue.cc:1048] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 13:21:17.448393 20719 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:17.479785 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:17.456734 20733 raft_consensus.cc:740] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:17.494699 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:17.494917 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:17.495180 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:17.495582 12709 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 17.
I20260709 13:21:17.496685 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:17.438645 20622 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.499186 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:17.500358 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:17.501302 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:17.500592 12782 raft_consensus.cc:1240] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:17.501302 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:17.505357 13378 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:17.501995 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:17.501672 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:17.503305 12706 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 27 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 13:21:17.509388 12703 raft_consensus.cc:2393] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 17: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:17.482220 20717 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:17.516603 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:17.511844 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:17.517541 12781 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:21:17.503010 20660 consensus_meta.cc:287] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.673s	user 0.007s	sys 0.004s
I20260709 13:21:17.501312 20767 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:17.503912 20696 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.511193 20622 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:17.530730 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 134036us
W20260709 13:21:17.533066 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 177353us
I20260709 13:21:17.534940 13378 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 22.
I20260709 13:21:17.501654 20771 consensus_queue.cc:1048] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:21:17.537968 11517 tablet_replica.cc:333] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:17.523847 20767 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:17.519874 20717 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:17.526840 20660 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:17.541433 20703 raft_consensus.cc:493] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.542042 20662 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:17.548900 20767 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:17.546027 20785 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:17.561774 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:17.563313 20717 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:17.533677 20696 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:17.575290 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:17.576016 12707 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 21. Current term is 22. Ops: []
W20260709 13:21:17.556270 20780 log.cc:927] Time spent T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:21:17.546193 20482 consensus_queue.cc:1048] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 13:21:17.589953 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:17.592005 20622 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:17.594074 20784 log.cc:927] Time spent T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:21:17.594174 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:17.595376 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:17.598577 20783 log.cc:927] Time spent T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:21:17.599712 20622 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.600278 20622 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:17.602357 20622 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:17.586797 20776 log.cc:927] Time spent T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:21:17.604522 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:17.606137 12598 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:17.583770 20785 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:17.609009 20785 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:17.611231 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:17.611845 12782 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:21:17.610653 20622 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:17.613855 12598 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:17.615955 20622 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:17.618083 12598 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:17.618077 20650 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:17.619416 11517 raft_consensus.cc:2243] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:17.547768 20738 consensus_queue.cc:1059] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
W20260709 13:21:17.622776 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 71142.2us
I20260709 13:21:17.531658 20733 consensus_queue.cc:260] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:17.630363 12598 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:17.631232 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 186448us
I20260709 13:21:17.634086 11517 raft_consensus.cc:2272] T 37e99a6b09d744a9b056b83be1033820 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:17.636183 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:17.568982 20703 raft_consensus.cc:515] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:17.640228 12598 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:17.643204 13349 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:17.643393 12598 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:17.628418 20785 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:21:17.636263 20649 consensus_meta.cc:287] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.595s	user 0.011s	sys 0.000s
I20260709 13:21:17.645617 12705 raft_consensus.cc:1275] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:21:17.646099 12703 raft_consensus.cc:1275] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
W20260709 13:21:17.638811 20640 consensus_meta.cc:287] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.002s
I20260709 13:21:17.648303 12598 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:17.647063 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:17.653623 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:17.654433 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:17.654603 12782 raft_consensus.cc:1240] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: [15.13-17.14]
I20260709 13:21:17.656993 20721 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:17.654363 20651 consensus_queue.cc:1048] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 13:21:17.654224 20493 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:21:17.659888 11517 tablet_replica.cc:333] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:17.660187 20660 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:17.660852 11517 raft_consensus.cc:2243] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:17.662143 13349 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:17.663363 20660 consensus_queue.cc:1059] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:17.664335 20771 raft_consensus.cc:3055] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:17.664739 20771 raft_consensus.cc:740] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:17.665256 20651 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:21:17.652554 20467 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:17.666051 20467 raft_consensus.cc:493] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:17.665611 20771 consensus_queue.cc:260] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:17.666409 20467 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:17.667001 20771 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:17.667302 13349 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:17.668030 20651 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:17.668308 20775 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:17.669176 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:17.669889 20651 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:21:17.671571 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "199931c4ffc44fee980d8bb49cf37f00" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:17.672634 20493 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:17.673907 20703 leader_election.cc:290] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:17.673089 20640 consensus_queue.cc:1059] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:17.675022 20640 raft_consensus.cc:3055] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:21:17.675371 20640 raft_consensus.cc:740] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:17.676152 20640 consensus_queue.cc:260] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:17.677048 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:21:17.648432 20778 log.cc:927] Time spent T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.164s	user 0.001s	sys 0.001s
I20260709 13:21:17.641530 20738 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:17.661384 11517 raft_consensus.cc:2272] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:17.680365 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:17.681945 12597 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:17.682293 20650 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.685899 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb2fc32fdf0a4cd981b02de0a47a7f96" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:17.686496 12779 raft_consensus.cc:2468] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 21.
I20260709 13:21:17.648702 20693 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:17.652577 20717 raft_consensus.cc:3060] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:17.677356 20640 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:17.678700 20665 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:17.690835 20738 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 13:21:17.690670 20785 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:21:17.647965 20723 consensus_queue.cc:1048] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 13:21:17.703522 20785 raft_consensus.cc:697] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 22 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:17.704058 20696 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:17.705894 12781 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 22.
I20260709 13:21:17.706818 13378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:17.708029 13349 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:17.692212 20650 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:17.710776 12707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42500 (request call id 3666) took 1058 ms. Trace:
I20260709 13:21:17.711369 20650 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:17.704464 20785 consensus_queue.cc:237] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:17.715441 13378 raft_consensus.cc:2393] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 16: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:21:17.722711 20622 raft_consensus.cc:3055] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:17.723206 20622 raft_consensus.cc:740] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:17.724288 20701 ts_tablet_manager.cc:1929] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 17.12
I20260709 13:21:17.724080 20622 consensus_queue.cc:260] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:17.727480 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:17.728421 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:17.728973 13380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:17.729954 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:17.687744 20703 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 13:21:17.730747 13383 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 27.
I20260709 13:21:17.730139 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:17.743674 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 57951.2us
I20260709 13:21:17.724896 20701 log.cc:1199] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-2-root/wals/f2f0a3339d324fd4902b6ff970ff0272
I20260709 13:21:17.711166 12707 rpcz_store.cc:276] 0709 13:21:16.652292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:16.718775 (+ 66483us) service_pool.cc:224] Handling call
0709 13:21:16.719015 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:17.587055 (+868040us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:17.710551 (+123496us) spinlock_profiling.cc:243] Waited 857 ms on lock 0x7b5400249ad4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3b2e01 00007f4c6d3b183e 00007f4c72376419 00007f4c6b8edbd4 00007f4c6b8edb62 00007f4c6b8eda91 00007f4c6b8eda0c 00007f4c6b8eccc2 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:21:17.710715 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1970475776}
I20260709 13:21:17.745832 12597 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:17.751098 12597 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:21:17.756354 12597 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:17.732827 20665 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:21:17.770655 12597 leader_election.cc:304] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:17.777866 12672 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:17.775170 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:17.752648 20690 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:21:17.802875 12597 raft_consensus.cc:2686] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 19cd864c053744bebd96093a7982537e: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:21:17.763245 20693 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:21:17.810927 13349 consensus_peers.cc:597] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:17.803094 12672 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:21:17.812372 12673 leader_election.cc:336] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.815295 12673 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:17.762702 20696 consensus_queue.cc:1059] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.038s
W20260709 13:21:17.817991 12673 leader_election.cc:336] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.820188 12673 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:17.818691 13385 raft_consensus.cc:1240] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 LEADER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:21:17.783149 20493 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:21:17.812829 13349 leader_election.cc:336] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.822834 13349 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:17.826514 12673 leader_election.cc:336] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.826926 12673 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:17.793586 20622 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:17.828260 12673 leader_election.cc:336] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.833036 12673 leader_election.cc:304] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:17.836158 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:17.823766 20739 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:21:17.811774 20660 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:17.810989 20693 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:17.830423 20769 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:21:17.855353 11517 tablet_replica.cc:333] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:17.863372 11517 raft_consensus.cc:2243] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:21:17.824498 20650 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:21:17.820238 20785 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.857298 20767 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:17.858886 12673 consensus_peers.cc:597] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:17.832249 20771 raft_consensus.cc:2749] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:21:17.856120 20701 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50664 (request call id 15) took 2645 ms. Trace:
I20260709 13:21:17.869422 11517 pending_rounds.cc:70] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e: Trying to abort 1 pending ops.
W20260709 13:21:17.868808 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 68643us
I20260709 13:21:17.819252 20298 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:17.862175 20693 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:17.878255 12673 consensus_peers.cc:597] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:17.825803 20663 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.046s
W20260709 13:21:17.880733 12747 leader_election.cc:336] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:17.884747 12673 leader_election.cc:336] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.885229 12673 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:17.886458 12673 consensus_peers.cc:597] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:17.877497 11517 pending_rounds.cc:77] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e: Aborting op as it isn't in flight: id { term: 22 index: 17 } timestamp: 7305639025518256128 op_type: NO_OP noop_request { }
W20260709 13:21:17.887089 12673 consensus_peers.cc:597] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:17.887166 11517 raft_consensus.cc:2889] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 22 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:21:17.887737 11517 raft_consensus.cc:2272] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:17.872216 20650 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:17.889104 12700 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:17.881285 20662 raft_consensus.cc:2749] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:17.868780 20670 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:17.831054 20797 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 13:21:17.880017 20781 raft_consensus.cc:2243] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Raft consensus shutting down.
W20260709 13:21:17.897133 12747 leader_election.cc:336] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.891364 20771 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:21:17.904251 12747 leader_election.cc:336] T f2f0a3339d324fd4902b6ff970ff0272 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:17.905462 12673 leader_election.cc:336] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.892675 20650 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:21:17.906339 12673 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:17.877905 20785 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:17.909632 12673 leader_election.cc:336] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:17.913501 12673 consensus_peers.cc:597] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:17.915203 12747 leader_election.cc:336] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.915419 20690 raft_consensus.cc:3055] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:21:17.915696 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:17.915931 20690 raft_consensus.cc:740] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:17.916834 20690 consensus_queue.cc:260] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:17.917867 20581 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:21:17.925884 20298 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:17.896793 20696 consensus_queue.cc:1059] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 21.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.015s
I20260709 13:21:17.907573 20717 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:17.934789 12705 raft_consensus.cc:1275] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:21:17.926287 20467 raft_consensus.cc:515] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:17.937772 20795 log.cc:927] Time spent T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:21:17.938424 13377 raft_consensus.cc:2393] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 18: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:21:17.927578 20781 raft_consensus.cc:2272] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:17.943825 12673 leader_election.cc:336] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.968441 20581 consensus_queue.cc:1048] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 13:21:17.973070 20781 ts_tablet_manager.cc:1916] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:21:17.978000 19829 consensus_queue.cc:1048] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:21:17.981246 20804 log.cc:927] Time spent T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 13:21:17.978505 12673 leader_election.cc:336] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.987427 20767 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:17.955317 20663 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:17.968189 20805 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.995266 12673 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:17.877357 20701 rpcz_store.cc:276] 0709 13:21:15.210930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:15.211131 (+   201us) service_pool.cc:224] Handling call
0709 13:21:15.540414 (+329283us) ts_tablet_manager.cc:1151] Deleting tablet f2f0a3339d324fd4902b6ff970ff0272
0709 13:21:15.540524 (+   110us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:21:17.207827 (+1667303us) tablet_metadata.cc:665] Metadata flushed
0709 13:21:17.724081 (+516254us) tablet_metadata.cc:665] Metadata flushed
0709 13:21:17.856037 (+131956us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1050827,"tablet-delete.queue_time_us":290147,"thread_start_us":263723,"threads_started":1}
W20260709 13:21:18.014816 12673 leader_election.cc:336] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:18.018028 12673 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:18.022650 20467 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:18.008333 20482 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:18.023675 20650 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:18.015786 20670 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:18.034339 20298 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:21:18.034044 20686 consensus_queue.cc:1048] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 13:21:18.015631 20663 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:18.023818 20482 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:18.026041 20805 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:18.054600 20738 raft_consensus.cc:493] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:18.078209 13349 leader_election.cc:336] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:18.061964 20650 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 28 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:18.108304 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:18.110251 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 68927.3us
I20260709 13:21:18.091015 20703 raft_consensus.cc:3055] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:21:18.153800 20703 raft_consensus.cc:740] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:18.154697 20703 consensus_queue.cc:260] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:18.155470 20798 log.cc:927] Time spent T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.299s	user 0.000s	sys 0.001s
I20260709 13:21:18.155778 20703 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:18.092458 20785 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:18.147428 20790 log.cc:927] Time spent T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.481s	user 0.003s	sys 0.000s
I20260709 13:21:18.147753 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:18.149646 20663 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:18.150179 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:18.151767 20482 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:18.107327 20710 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:18.080497 20693 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:18.146551 20621 leader_election.cc:290] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:18.156213 20798 consensus_queue.cc:1175] T e2a417d650a44658bfd5e2f5305185ac P 19cd864c053744bebd96093a7982537e [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "19cd864c053744bebd96093a7982537e" status { last_received { term: 22 index: 17 } last_committed_idx: 16 last_received_current_leader { term: 22 index: 17 } }
I20260709 13:21:18.162067 20733 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:18.165280 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:18.166555 20791 log.cc:927] Time spent T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.500s	user 0.004s	sys 0.000s
I20260709 13:21:18.185490 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:18.194696 13383 raft_consensus.cc:2393] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 18: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:21:18.082152 20738 raft_consensus.cc:515] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:18.251175 20622 raft_consensus.cc:493] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:18.251844 20622 raft_consensus.cc:515] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:18.253994 20622 leader_election.cc:290] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:18.255934 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:18.257656 20785 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.258358 20785 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:18.259140 20693 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:18.261111 20785 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:18.205219 20710 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:18.216594 20621 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.264935 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:18.277956 12775 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:18.265722 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:18.267359 12673 leader_election.cc:336] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:18.275416 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "981fb2f435e24d06a81fc02cf4591cc5" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:18.283699 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9155962ca7f34a8c83a1780ce0018a59" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:18.305362 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 99133.3us
I20260709 13:21:18.274606 20693 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.282152 20622 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:18.328383 12672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:18.333636 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:18.323459 20721 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:18.326431 20693 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:18.346391 20721 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:18.348490 20721 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:18.349730 20693 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:18.349647 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:18.350571 12779 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:21:18.350884 13383 raft_consensus.cc:1240] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Rejecting Update request from peer 639ab2db265f4747bb7afb93e833f530 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:18.326076 20622 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:18.353554 20622 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:18.360721 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:18.361150 12672 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:18.360721 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:18.362260 13381 raft_consensus.cc:2393] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 17: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:21:18.268280 20663 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.374941 12775 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
I20260709 13:21:18.376461 20767 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:18.435174 12775 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:21:18.367451 20662 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:18.046514 12516 catalog_manager.cc:5028] TS 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): tablet f2f0a3339d324fd4902b6ff970ff0272 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:21:18.386523 20738 leader_election.cc:290] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:18.389611 20622 raft_consensus.cc:493] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:18.390475 13349 leader_election.cc:336] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:18.376302 20663 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:18.394522 20664 raft_consensus.cc:493] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:18.387840 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:18.400353 12673 leader_election.cc:336] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:18.453311 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:18.453790 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 53465.7us
I20260709 13:21:18.400743 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:18.401064 11517 tablet_replica.cc:333] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:18.401430 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:18.454834 13385 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:21:18.402407 20710 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:18.406185 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:18.409271 12597 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:21:18.458343 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 49159.8us
W20260709 13:21:18.458858 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:18.286100 20621 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:18.432272 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:18.463078 11517 raft_consensus.cc:2243] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:18.457800 13381 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
W20260709 13:21:18.440954 12516 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 394444us
W20260709 13:21:18.450253 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 59813us
I20260709 13:21:18.435201 20493 raft_consensus.cc:493] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.435765 20771 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:18.409909 12672 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:18.455494 12747 consensus_peers.cc:597] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:18.469825 11517 raft_consensus.cc:2272] T eb7fca96394f4ba9b4fe3d7a6e278878 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:18.460027 12597 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:18.492662 12597 leader_election.cc:304] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:18.462308 20796 log.cc:927] Time spent T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.605s	user 0.004s	sys 0.000s
I20260709 13:21:18.463629 20664 raft_consensus.cc:515] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:18.457017 20622 raft_consensus.cc:515] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:18.482507 20493 raft_consensus.cc:515] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:18.441730 20769 consensus_queue.cc:1059] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:21:18.462343 20805 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:18.485710 20663 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:18.488384 20710 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:21:18.491675 12747 consensus_peers.cc:597] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:18.523859 12747 leader_election.cc:336] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:18.524308 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:18.495942 12597 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:18.503827 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 94022us
I20260709 13:21:18.462879 20785 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.528309 12701 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 13:21:18.528064 20785 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:18.528692 12701 pending_rounds.cc:85] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530: Aborting all ops after (but not including) 13
I20260709 13:21:18.528879 20493 leader_election.cc:290] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:18.530205 20785 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:18.467047 20738 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:18.454074 20797 consensus_queue.cc:1059] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:21:18.495666 20812 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:21:18.532384 12747 consensus_peers.cc:597] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:18.534004 20830 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:18.534865 20664 leader_election.cc:290] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:18.536650 11517 tablet_replica.cc:333] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:18.539562 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:18.528930 12701 pending_rounds.cc:107] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:21:18.543041 12701 raft_consensus.cc:2889] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:18.544309 20622 leader_election.cc:290] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:18.505187 20640 raft_consensus.cc:2749] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:18.560142 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:18.560837 12776 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
W20260709 13:21:18.563694 12747 leader_election.cc:336] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:18.564318 12747 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:18.565248 20793 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:18.565778 20690 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:18.505448 20650 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:18.537516 11517 raft_consensus.cc:2243] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:18.451874 20662 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:18.527070 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:21:18.509477 20809 log.cc:927] Time spent T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:21:18.599231 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 103349us
I20260709 13:21:18.506850 20771 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:18.628011 13349 leader_election.cc:336] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:18.637828 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:18.638722 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:18.648277 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:18.604638 20785 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:18.604858 20738 raft_consensus.cc:697] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:18.586061 20710 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:18.658591 11517 raft_consensus.cc:2272] T 9119721f3ee04810b1f277c06c966a45 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:18.651567 20769 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.648404 20669 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.648741 20493 raft_consensus.cc:493] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:18.699968 20493 raft_consensus.cc:515] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:18.702647 20493 leader_election.cc:290] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:18.690781 20769 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:18.693089 11517 tablet_replica.cc:333] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:18.698885 20669 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:18.757891 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:18.762434 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 173219us
W20260709 13:21:18.763872 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 46164.7us
I20260709 13:21:18.774138 13348 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:18.782491 12597 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:18.784041 20621 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:18.785064 20669 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:18.757469 20806 raft_consensus.cc:493] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.786069 20806 raft_consensus.cc:515] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:18.788407 20806 leader_election.cc:290] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:18.788810 20662 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:18.789896 20771 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:18.790733 20769 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:18.735348 20781 ts_tablet_manager.cc:1929] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.10
I20260709 13:21:18.792079 20710 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:18.792527 20781 log.cc:1199] T 981fb2f435e24d06a81fc02cf4591cc5 P 639ab2db265f4747bb7afb93e833f530: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-1-root/wals/981fb2f435e24d06a81fc02cf4591cc5
I20260709 13:21:18.794337 20554 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:21:18.800555 20687 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:18.801194 20687 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:18.801978 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:18.803514 20664 raft_consensus.cc:3055] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:18.803787 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94e3333498524f14a17661cdb38ef144" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:18.803983 20664 raft_consensus.cc:740] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:18.805850 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d46747dede234934bea0f0c324106c28" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:18.806473 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:18.807097 13382 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:21:18.808382 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "775128ed7b8d4283bf303c8d40247615" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:18.810092 13349 consensus_peers.cc:597] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:18.804919 20664 consensus_queue.cc:260] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:18.764828 12672 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:18.731384 20812 raft_consensus.cc:493] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.818419 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:18.822670 12783 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 25.
I20260709 13:21:18.818799 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:18.820462 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 55760.1us
I20260709 13:21:18.821091 12597 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:18.825170 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:18.827818 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:18.828346 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:18.831544 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:18.828522 12707 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 15.
I20260709 13:21:18.837528 12704 raft_consensus.cc:1240] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 28 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 27. Current term is 28. Ops: [27.19-27.19]
I20260709 13:21:18.803802 20687 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:18.820284 20792 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.844069 12597 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:21:18.848064 12707 raft_consensus.cc:1240] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 21. Current term is 22. Ops: []
I20260709 13:21:18.848194 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b98bb10e3e04334b7fcf7f781593682" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:18.822605 20817 raft_consensus.cc:2749] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:21:18.843595 20651 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:18.849921 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:18.850543 20651 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:18.842022 20844 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:21:18.853020 20651 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:18.853703 20493 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:21:18.854158 20493 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:18.854513 20493 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:18.778728 20719 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 13:21:18.833251 20717 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:18.867697 20717 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:18.868085 20717 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:18.859970 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:18.868985 12707 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:18.865729 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:18.863989 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa34731f37084c608660f0088a2dba09" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:18.872859 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:18.866701 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:18.875218 12709 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 17.
I20260709 13:21:18.876085 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:18.877118 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:18.832793 20812 raft_consensus.cc:515] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:18.878957 20835 log.cc:927] Time spent T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.253s	user 0.000s	sys 0.002s
I20260709 13:21:18.879323 12709 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:21:18.880304 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4679cef42d5540e9a5d3a66e52cdcaf9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:18.880921 12704 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 25.
I20260709 13:21:18.847411 20664 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:18.883832 20733 consensus_meta.cc:287] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2: Time spent flushing consensus metadata: real 0.672s	user 0.019s	sys 0.011s
I20260709 13:21:18.884306 20563 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.884825 20563 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:18.820158 20738 consensus_queue.cc:237] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:18.883409 12709 raft_consensus.cc:1240] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 22. Current term is 23. Ops: []
I20260709 13:21:18.892211 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f91059867c444d89718e83475c089cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:18.865696 12597 leader_election.cc:304] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:18.895476 12597 leader_election.cc:304] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:18.898801 12598 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:18.901685 12598 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:18.905997 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:18.907202 12709 raft_consensus.cc:2393] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 18: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:18.908283 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:18.912308 12703 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:21:18.908456 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:21:18.920411 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 147858us
I20260709 13:21:18.858068 20781 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49378 (request call id 36) took 1913 ms. Trace:
I20260709 13:21:18.927393 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:18.927699 12776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:18.928292 12775 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 25.
I20260709 13:21:18.911331 20703 consensus_queue.cc:1059] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.002s
I20260709 13:21:18.907732 20663 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:21:18.910391 20621 raft_consensus.cc:2749] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:21:18.911492 20640 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:18.929796 12672 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:18.911515 20782 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:18.926739 12673 leader_election.cc:336] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:18.942444 12673 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:18.863298 20792 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:18.943888 12673 consensus_peers.cc:597] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:18.957261 12707 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 16.
W20260709 13:21:18.961608 13349 consensus_peers.cc:597] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:18.962110 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 146577us
I20260709 13:21:18.963971 11517 raft_consensus.cc:2243] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:18.957777 20640 raft_consensus.cc:2749] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:18.975355 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:18.976488 12707 raft_consensus.cc:2393] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 23: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:18.974450 12776 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 24.
I20260709 13:21:18.979598 13349 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:18.986433 20467 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:21:18.991595 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 96194.4us
I20260709 13:21:18.995744 12597 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:19.008546 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:18.952900 20838 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 13:21:18.952530 20843 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:21:18.994776 20739 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:18.999338 20792 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:19.003789 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af96e52b004d4c8c942cc0ec33c95c99" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:19.005595 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:18.950845 20782 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:19.010679 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:19.009359 12597 leader_election.cc:304] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:19.013403 13383 raft_consensus.cc:2393] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 25: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
W20260709 13:21:19.014643 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 111628us
I20260709 13:21:19.015854 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:19.019869 13384 raft_consensus.cc:2393] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 24: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
W20260709 13:21:19.019848 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 100401us
I20260709 13:21:19.015163 20690 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:19.022396 20690 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:19.022759 20690 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:19.014475 20843 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:19.029925 20843 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:21:19.033160 20767 consensus_meta.cc:287] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530: Time spent flushing consensus metadata: real 0.583s	user 0.007s	sys 0.000s
I20260709 13:21:18.925705 20781 rpcz_store.cc:276] 0709 13:21:16.972617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:16.988741 (+ 16124us) service_pool.cc:224] Handling call
0709 13:21:17.343991 (+355250us) ts_tablet_manager.cc:1151] Deleting tablet 981fb2f435e24d06a81fc02cf4591cc5
0709 13:21:17.350434 (+  6443us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:21:18.160427 (+809993us) tablet_metadata.cc:665] Metadata flushed
0709 13:21:18.697452 (+537025us) tablet_metadata.cc:665] Metadata flushed
0709 13:21:18.841461 (+144009us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":237181,"tablet-delete.queue_time_us":278696,"thread_start_us":74228,"threads_started":1}
I20260709 13:21:19.040515 20717 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:19.044061 11517 raft_consensus.cc:2272] T c4e4ae9d38ba4fd0a700998e4f4750be P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.046052 12701 raft_consensus.cc:2410] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
W20260709 13:21:19.059342 13349 leader_election.cc:336] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:19.065764 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:19.069653 11517 tablet_replica.cc:333] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:19.070756 11517 raft_consensus.cc:2243] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:19.048362 20767 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:19.076772 12672 leader_election.cc:304] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:19.100620 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:19.101955 12598 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:19.111009 20690 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:19.073321 20739 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:19.062253 20797 raft_consensus.cc:2749] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:19.112260 20664 raft_consensus.cc:2749] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:19.069708 20493 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:19.125442 20493 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:19.137308 20664 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:19.138151 20664 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:19.141633 20664 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:19.143435 20717 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:19.148396 12705 raft_consensus.cc:2468] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 18.
I20260709 13:21:19.196597 20621 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:21:19.197743 20861 raft_consensus.cc:493] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.198405 20861 raft_consensus.cc:515] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:19.201176 20861 leader_election.cc:290] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:19.202291 20860 log.cc:927] Time spent T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:21:19.116855 20690 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:19.204751 11517 raft_consensus.cc:2272] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.224186 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:19.225741 13384 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 25: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:21:19.235582 20843 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:19.238229 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:19.239176 13384 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:19.240944 11517 tablet_replica.cc:333] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:19.242437 11517 raft_consensus.cc:2243] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:19.253021 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
W20260709 13:21:19.254359 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 152473us
I20260709 13:21:19.254464 13383 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:19.255692 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:19.256376 13384 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:21:19.259187 11517 raft_consensus.cc:2272] T 1fa805ee07a7429abe6b520bbfb94cd9 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.259754 12598 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:19.262269 12598 leader_election.cc:304] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:19.259833 12597 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:19.261394 20861 raft_consensus.cc:3060] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:19.266885 13383 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 20.
I20260709 13:21:19.267720 20817 raft_consensus.cc:2749] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:21:19.277644 12598 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:19.279691 20855 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:19.282318 20663 raft_consensus.cc:3055] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 27 LEADER]: Stepping down as leader of term 27
I20260709 13:21:19.282945 20663 raft_consensus.cc:740] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:19.284099 11517 tablet_replica.cc:333] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:19.283792 20663 consensus_queue.cc:260] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:19.285120 20663 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:21:19.285327 11517 raft_consensus.cc:2243] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:19.286042 11517 raft_consensus.cc:2272] T 9155962ca7f34a8c83a1780ce0018a59 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.288751 11517 tablet_replica.cc:333] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:19.289600 11517 raft_consensus.cc:2243] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 22 LEADER]: Raft consensus shutting down.
W20260709 13:21:19.290086 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:19.290987 11517 raft_consensus.cc:2272] T 6146f826f1904f6f8b10ad518d454c22 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.267292 20703 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:19.293881 20703 raft_consensus.cc:697] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
W20260709 13:21:19.293092 13349 leader_election.cc:336] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:19.296232 12598 leader_election.cc:304] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:19.297842 13349 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:19.300556 13349 consensus_peers.cc:597] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:19.301153 13349 consensus_peers.cc:597] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:19.306005 12598 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:19.298382 20861 raft_consensus.cc:2749] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:19.301911 20589 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:21:19.319998 12598 raft_consensus.cc:2686] T 8f91059867c444d89718e83475c089cd P 19cd864c053744bebd96093a7982537e: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:21:19.302106 20785 raft_consensus.cc:2749] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:21:19.325423 13349 consensus_peers.cc:597] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:19.329638 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:19.330358 12702 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:19.332990 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:19.333137 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:19.333600 12700 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:21:19.333784 12708 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:19.334762 12707 raft_consensus.cc:1240] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Rejecting Update request from peer 19cd864c053744bebd96093a7982537e for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:19.346948 12700 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 20.
I20260709 13:21:19.352739 20863 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:19.354327 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 333308us
I20260709 13:21:19.359495 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:19.360338 13384 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 24.
I20260709 13:21:19.365424 12708 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 18.
I20260709 13:21:19.366194 12702 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 26.
I20260709 13:21:19.306924 20703 consensus_queue.cc:237] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:19.386950 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a0f7a1130449548d1659cd21760fc9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:19.390120 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:19.367646 20863 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:19.374400 20665 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.381891 20854 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:19.390977 12775 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:19.400825 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:19.401829 12782 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:21:19.392026 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:19.403558 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:19.403741 12672 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:19.404359 12781 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:21:19.387977 12746 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:19.368202 20622 raft_consensus.cc:3055] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:19.410218 20806 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.417415 12775 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 26.
I20260709 13:21:19.393489 20830 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.417482 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:19.420745 12781 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 20.
I20260709 13:21:19.412712 20622 raft_consensus.cc:740] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:19.453150 12597 leader_election.cc:304] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:19.417038 20467 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.418674 20645 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:21:19.458714 12597 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
I20260709 13:21:19.416723 20771 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:21:19.452540 20855 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.463655 12597 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:19.458128 11517 tablet_replica.cc:333] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:19.444621 20670 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:19.406276 20665 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:19.443534 20690 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.427614 20830 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:19.463351 20851 raft_consensus.cc:2749] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:19.462647 20645 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:19.463985 20771 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:19.468482 20805 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:21:19.470880 20717 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:19.490803 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:19.496111 13379 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 25.
I20260709 13:21:19.472690 20670 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:19.466307 20467 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:19.420490 20806 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:19.499884 20805 raft_consensus.cc:697] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 26 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:19.486896 20645 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:19.493126 20563 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:19.469256 20855 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:19.488359 20771 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:19.413156 20854 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:19.518782 20863 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:19.519315 20467 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:19.519552 20855 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:19.520941 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:19.562667 13384 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 25.
I20260709 13:21:19.521878 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:19.563999 12701 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 24.
I20260709 13:21:19.522354 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:19.524976 12781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:19.529690 20771 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:19.502846 20670 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:19.502178 20717 raft_consensus.cc:697] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:19.550935 20665 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:19.485122 20690 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:19.558537 11517 raft_consensus.cc:2243] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:19.484493 20622 consensus_queue.cc:260] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:19.575896 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 248760us
I20260709 13:21:19.583475 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:19.584343 12701 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 24.
I20260709 13:21:19.590341 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:19.591183 12707 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 25.
I20260709 13:21:19.590408 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:21:19.600368 13349 consensus_peers.cc:597] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:19.609622 11517 raft_consensus.cc:2272] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.612874 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:19.599570 20622 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:19.611235 20854 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:19.599179 20782 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:19.622668 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:19.623308 12700 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 24.
I20260709 13:21:19.620565 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:19.624755 12705 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 24.
W20260709 13:21:19.598809 20864 consensus_queue.cc:1175] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3d539f8d95d343d8b1696803f18b30d2" responder_term: 17 status { last_received { term: 17 index: 12 } last_committed_idx: 12 last_received_current_leader { term: 17 index: 12 } }
I20260709 13:21:19.617396 20830 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:19.582731 20805 consensus_queue.cc:237] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:19.651263 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:19.652189 12700 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:21:19.665711 20868 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.666278 20868 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:19.668346 20868 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:19.669178 12747 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:19.682185 20806 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:19.682917 20792 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:21:19.683337 20792 raft_consensus.cc:493] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:19.683672 20792 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:19.626359 20717 consensus_queue.cc:237] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:19.688411 20690 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:19.662767 20771 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:19.691831 20812 leader_election.cc:290] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:19.692557 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 747131us
I20260709 13:21:19.681133 20589 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:21:19.701069 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:19.702374 12747 leader_election.cc:336] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:19.702839 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:19.703178 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d53f974eee5475b9cd2701948f533ac" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:19.692476 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:19.706652 13349 consensus_peers.cc:597] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:19.717665 20828 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:19.716863 20665 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:19.692015 20792 raft_consensus.cc:515] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:19.748881 20792 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 26 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:19.734452 20665 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:19.750976 20665 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:19.733489 20645 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:19.753152 20645 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:19.753846 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:19.754531 12775 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:21:19.754935 20843 leader_election.cc:290] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:19.736276 13349 consensus_peers.cc:597] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:19.752803 20670 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:19.761602 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:19.762364 13384 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 22.
W20260709 13:21:19.759913 13349 consensus_peers.cc:597] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:19.763376 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9132272ce4c847a99e395a53c1015230" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:19.765141 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 59857.9us
W20260709 13:21:19.766925 13349 leader_election.cc:336] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:19.769181 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:19.783869 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:19.784829 13377 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:19.791272 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:19.792603 13379 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:19.792882 13377 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 26.
I20260709 13:21:19.796384 20828 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:19.799952 20670 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:19.806907 13379 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 25.
I20260709 13:21:19.851864 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:21:19.859727 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 157407us
I20260709 13:21:19.866088 12709 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:21:19.867884 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:19.868968 12707 raft_consensus.cc:3060] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:19.034909 12517 catalog_manager.cc:5028] TS 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): tablet 981fb2f435e24d06a81fc02cf4591cc5 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
I20260709 13:21:19.870453 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:21:19.871070 12517 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 835953us
I20260709 13:21:19.871646 12708 raft_consensus.cc:3060] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:21:19.888660 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 167394us
W20260709 13:21:19.894984 13349 leader_election.cc:336] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:19.896566 13349 consensus_peers.cc:597] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:19.898522 20844 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.899835 20844 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:19.900683 11517 tablet_replica.cc:333] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:19.903589 11517 raft_consensus.cc:2243] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:21:19.905318 12597 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:19.907817 13349 leader_election.cc:304] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:19.912009 20844 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:19.912387 20589 raft_consensus.cc:2804] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:19.926591 20589 raft_consensus.cc:697] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:19.927100 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:19.928221 12779 raft_consensus.cc:2393] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 25: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:21:19.928777 12707 raft_consensus.cc:2468] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 26.
I20260709 13:21:19.928241 20589 consensus_queue.cc:237] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:19.934151 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:19.937217 12779 raft_consensus.cc:2368] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 25. Current term is 26.
I20260709 13:21:19.944758 12708 raft_consensus.cc:2468] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 25.
I20260709 13:21:19.915681 20838 raft_consensus.cc:2955] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Committing config change with OpId 16.13: config changed from index 11 to 13, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:19.950168 11517 raft_consensus.cc:2272] T 1b13b87a5cea4263937041d123c5889d P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:19.949990 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 92865.4us
I20260709 13:21:19.951478 20645 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:21:19.952275 12599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:19.952347 20645 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:19.953586 20693 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 13:21:19.954031 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 62295.5us
I20260709 13:21:19.954315 20693 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:19.954033 20717 raft_consensus.cc:493] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.956545 20717 raft_consensus.cc:515] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:19.959443 20717 leader_election.cc:290] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:19.959997 20693 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:19.953310 12597 leader_election.cc:304] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:19.968281 12597 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2; no voters: 
W20260709 13:21:19.969139 12597 raft_consensus.cc:2686] T ea604dddf4c244a181921c4dbdd0e720 P 19cd864c053744bebd96093a7982537e: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:21:19.973497 20571 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:19.973863 20817 raft_consensus.cc:2749] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:19.974475 20571 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:19.975098 20723 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.975780 20723 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:19.965297 11517 tablet_replica.cc:333] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:19.980721 20723 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:19.982110 20645 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:19.997532 20571 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:20.002522 12597 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:20.003993 20717 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:20.006245 12597 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:20.022394 13349 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:20.024215 20739 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:21:20.024919 20739 raft_consensus.cc:493] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:20.025451 20739 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:20.025279 13349 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:20.028712 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:20.028795 20467 raft_consensus.cc:2764] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 13:21:20.028091 20817 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:20.038400 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:21:20.049793 13349 leader_election.cc:336] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:20.053216 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:20.052309 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4d7de736149496b86a37ba2ca92670d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:20.061971 12779 raft_consensus.cc:2368] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 24. Current term is 25.
I20260709 13:21:20.067423 13379 raft_consensus.cc:2368] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 24. Current term is 25.
I20260709 13:21:20.053200 20563 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:20.065826 12747 leader_election.cc:304] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:20.069136 20563 raft_consensus.cc:493] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:20.075378 20563 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:20.086515 20792 raft_consensus.cc:2804] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:21:20.089082 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37e99a6b09d744a9b056b83be1033820" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:21:20.088807 12673 leader_election.cc:336] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:20.109030 12673 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:20.097374 11517 raft_consensus.cc:2243] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:20.091811 20792 raft_consensus.cc:697] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 26 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:20.110896 20792 consensus_queue.cc:237] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:20.090952 12747 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:20.113525 20645 raft_consensus.cc:2804] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:21:20.114235 20645 raft_consensus.cc:697] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 25 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:20.115588 20645 consensus_queue.cc:237] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:20.124713 12746 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:20.126099 20853 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:20.126559 20853 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:20.126940 20853 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:20.132180 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:20.133109 12708 raft_consensus.cc:2393] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 22: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:20.133680 20883 raft_consensus.cc:2749] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:20.134704 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad2866c2079c4c5b9df5ad6a072e192d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:21:20.150501 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 98322.8us
W20260709 13:21:20.165659 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 86122.2us
I20260709 13:21:20.168560 20817 raft_consensus.cc:493] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:20.170063 11517 raft_consensus.cc:2272] T 9132272ce4c847a99e395a53c1015230 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:20.169958 20817 raft_consensus.cc:515] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:20.172514 20817 leader_election.cc:290] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:20.170468 12525 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40528 (request call id 231) took 12548 ms (client timeout 15000 ms). Trace:
I20260709 13:21:20.169060 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
W20260709 13:21:20.173738 12673 leader_election.cc:336] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:20.183233 12673 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:20.184309 20854 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:20.156024 12531 catalog_manager.cc:5697] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 14 to 16, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 16 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:21:20.174170 12525 rpcz_store.cc:269] 0709 13:21:07.622408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:07.627842 (+  5434us) service_pool.cc:224] Handling call
0709 13:21:20.170252 (+12542410us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:17.549687 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:17.614127 (+ 64440us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:17.614167 (+    40us) write_op.cc:435] Acquired schema lock
0709 13:21:17.614186 (+    19us) tablet.cc:662] Decoding operations
0709 13:21:17.614849 (+   663us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:17.614940 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:17.614971 (+    31us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:21:17.615387 (+   416us) tablet.cc:701] Row locks acquired
0709 13:21:17.615407 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:21:17.615568 (+   161us) write_op.cc:270] Start()
0709 13:21:17.615705 (+   137us) write_op.cc:276] Timestamp: P: 1783603277615519 usec, L: 0
0709 13:21:17.615731 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:21:17.616428 (+   697us) log.cc:844] Serialized 1674 byte log entry
0709 13:21:17.792663 (+176235us) op_driver.cc:464] REPLICATION: finished
0709 13:21:17.913013 (+120350us) write_op.cc:301] APPLY: starting
0709 13:21:17.913092 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:17.916004 (+  2912us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:17.916104 (+   100us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:17.918647 (+  2543us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:17.918865 (+   218us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 13:21:17.918915 (+    50us) write_op.cc:312] APPLY: finished
0709 13:21:17.919165 (+   250us) log.cc:844] Serialized 49 byte log entry
0709 13:21:17.919326 (+   161us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:17.919430 (+   104us) write_op.cc:454] Released schema lock
0709 13:21:17.919475 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:17.919887 (+   412us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:17.919923 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":161671,"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":105435,"apply.run_cpu_time_us":6855,"apply.run_wall_time_us":6856,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20863,"num_ops":4,"prepare.queue_time_us":336564,"prepare.run_cpu_time_us":3519,"prepare.run_wall_time_us":76872,"replication_time_us":146930,"thread_start_us":399157,"threads_started":2}]]}
I20260709 13:21:20.187228 20854 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:20.198791 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:20.207156 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "059429c0c4024dfb941d1ff325fe8f98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:20.212440 13377 raft_consensus.cc:2368] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 25. Current term is 26.
I20260709 13:21:20.215425 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:20.199261 20883 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:20.220580 13384 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:21:20.221517 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:20.225190 11517 tablet_replica.cc:333] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:20.208911 12531 catalog_manager.cc:5697] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 24 to 25, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 25 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:20.230360 12700 raft_consensus.cc:2410] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
W20260709 13:21:20.235276 12597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:20.234440 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:20.232345 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:20.240859 20785 raft_consensus.cc:493] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
W20260709 13:21:20.248623 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 80570.6us
I20260709 13:21:20.243255 20785 raft_consensus.cc:515] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } attrs { replace: true } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:20.261289 13381 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 23.
I20260709 13:21:20.261803 12709 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:21:20.267535 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:20.271661 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fd9f4e830ed4a178665ad4e4ecc4452" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:20.278750 12700 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:21:20.230803 20717 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 13:21:20.293974 20885 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:20.244864 20739 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:20.304464 13379 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:21:20.254899 20693 raft_consensus.cc:515] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:20.275496 12531 catalog_manager.cc:5697] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:20.318770 20805 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 13:21:20.328025 12781 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:21:20.328660 12775 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 13:21:20.287598 12779 raft_consensus.cc:2955] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index 11 to 13, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:21:20.350853 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 102623us
W20260709 13:21:20.357488 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 100402us
I20260709 13:21:20.306923 20853 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:20.360422 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 76851.4us
I20260709 13:21:20.375034 12672 leader_election.cc:400] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 24. Current term is 25.
I20260709 13:21:20.314123 13383 raft_consensus.cc:2955] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index 11 to 13, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:21:20.386093 20886 log.cc:927] Time spent T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:21:20.356401 12531 catalog_manager.cc:5697] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 20 to 21, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 21 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:20.376153 20785 leader_election.cc:290] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:20.382365 20854 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:20.382792 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 106799us
I20260709 13:21:20.387526 12672 leader_election.cc:403] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
W20260709 13:21:20.394400 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 124902us
W20260709 13:21:20.401870 12598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:20.402834 12597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 91116.6us
W20260709 13:21:20.404999 12673 leader_election.cc:336] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:20.408780 12673 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:20.413520 11517 raft_consensus.cc:2243] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:20.417985 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:20.419083 13385 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 26: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:21:20.422246 20739 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 26 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:20.422724 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:20.417351 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1054d44ad14423fb3d51a5243499bb9" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:20.430570 20828 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:20.431372 20828 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:20.434193 20828 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:20.444991 20838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:49378 (request call id 35) took 5238 ms. Trace:
I20260709 13:21:20.453548 11517 raft_consensus.cc:2272] T 98a43c5d78c945bb8c5541bc892f4865 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:20.457090 11517 tablet_replica.cc:333] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:20.449450 20838 rpcz_store.cc:276] 0709 13:21:15.206264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:15.422637 (+216373us) service_pool.cc:224] Handling call
0709 13:21:15.446868 (+ 24231us) log.cc:844] Serialized 494 byte log entry
0709 13:21:16.230012 (+783144us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b4400022b4c. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6a14ddb1 00007f4c6a1ebc85 00007f4c6a1ec6a2 00007f4c6cf5e1db 00007f4c6cf59aaf 00007f4c6cf4d86e 00007f4c6cf4cbb6 00007f4c6cf4c7d8 00007f4c6cf56221 00007f4c6cf561d9 00007f4c6cf56169
0709 13:21:20.444235 (+4214223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42893184,"thread_start_us":1164436,"threads_started":1,"wal-append.queue_time_us":423897,"wal-append.run_cpu_time_us":3719,"wal-append.run_wall_time_us":1361331}
I20260709 13:21:20.458361 12531 catalog_manager.cc:5697] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 23 to 24, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 24 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:20.461084 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 98a43c5d78c945bb8c5541bc892f4865 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:21:20.460420 12531 catalog_manager.cc:5697] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 22 to 23, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 23 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:20.462736 12531 catalog_manager.cc:5697] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 22 to 26, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 26 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:20.464486 12531 catalog_manager.cc:5697] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 24 to 26, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 26 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:21:20.425504 20589 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:20.466174 20589 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:20.467989 13349 leader_election.cc:336] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:20.468433 20589 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:20.475035 20853 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:20.476679 13349 leader_election.cc:336] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:20.497514 20782 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:21:20.495254 12747 leader_election.cc:336] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:20.504652 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 87133us
I20260709 13:21:20.495330 11517 raft_consensus.cc:2243] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:20.516870 12672 leader_election.cc:400] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 election: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 25. Current term is 26.
I20260709 13:21:20.517447 12672 leader_election.cc:403] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 13:21:20.519625 12531 catalog_manager.cc:5697] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:20.529654 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 112649us
I20260709 13:21:20.536991 20844 raft_consensus.cc:2749] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 25. Current term is 26.
I20260709 13:21:20.534492 12672 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:20.547248 12531 catalog_manager.cc:5697] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:21:20.551146 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 149916us
I20260709 13:21:20.510749 20563 raft_consensus.cc:515] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:20.551908 12672 leader_election.cc:400] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 25 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 25. Current term is 26.
I20260709 13:21:20.557248 20771 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:20.558243 20771 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:20.560981 20771 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:20.564344 20563 leader_election.cc:290] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:20.565392 11517 raft_consensus.cc:2272] T 94e3333498524f14a17661cdb38ef144 P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:20.551335 20693 leader_election.cc:290] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:20.569384 20891 log.cc:927] Time spent T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:21:20.549329 20885 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:21:20.569962 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 85837.1us
I20260709 13:21:20.427840 20883 raft_consensus.cc:2749] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021) with higher term. Message: Invalid argument: T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 24. Current term is 25.
I20260709 13:21:20.582453 12597 leader_election.cc:304] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:20.581025 20844 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:21:20.600670 12597 leader_election.cc:304] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:20.609678 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 57919.4us
I20260709 13:21:20.611318 12672 leader_election.cc:400] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 election: Vote denied by peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181) with higher term. Message: Invalid argument: T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 for earlier term 24. Current term is 25.
I20260709 13:21:20.612843 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:20.614485 20792 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:20.615154 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2a417d650a44658bfd5e2f5305185ac" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:20.615284 20792 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:20.616014 13381 raft_consensus.cc:3060] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:20.605626 20785 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.011s
I20260709 13:21:20.618047 20792 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:20.618767 11517 tablet_replica.cc:333] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:20.599786 20844 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:20.620607 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:20.622110 20889 raft_consensus.cc:2749] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:21:20.624928 13349 consensus_peers.cc:597] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.626024 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 75981.3us
I20260709 13:21:20.620857 20844 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:20.621372 12531 catalog_manager.cc:5697] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 21 to 22, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 22 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:20.629845 12597 leader_election.cc:304] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:20.633908 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:20.635030 20889 raft_consensus.cc:2749] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:20.600643 20805 raft_consensus.cc:2749] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:20.639592 13377 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:21:20.641791 20838 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:21:20.635182 12779 raft_consensus.cc:2410] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 22 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:21:20.602336 20854 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:21:20.620666 11517 raft_consensus.cc:2243] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:20.649739 11517 raft_consensus.cc:2272] T 3b98bb10e3e04334b7fcf7f781593682 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:20.655045 13349 consensus_peers.cc:597] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:20.661788 12673 leader_election.cc:336] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:20.686071 12598 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:20.687988 20805 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:21:20.692191 11517 tablet_replica.cc:333] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:20.693015 11517 raft_consensus.cc:2243] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:20.693884 11517 raft_consensus.cc:2272] T 20db0f202ea546699789c2325d241b35 P 19cd864c053744bebd96093a7982537e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:20.696367 20589 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:20.697520 20792 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:21:20.698283 11517 tablet_replica.cc:333] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:20.699524 11517 raft_consensus.cc:2243] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:20.697075 20589 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:20.700330 11517 raft_consensus.cc:2272] T fb247fe4862541ccadc22bb79417d66c P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:20.669725 13381 raft_consensus.cc:2468] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 23.
I20260709 13:21:20.688971 12531 catalog_manager.cc:5697] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
W20260709 13:21:20.713739 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 101021us
I20260709 13:21:20.735148 20589 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:20.735469 12531 catalog_manager.cc:5697] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 22 to 25. New cstate: current_term: 25 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:20.736948 20805 raft_consensus.cc:493] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:20.729647 12673 leader_election.cc:336] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:20.737524 12531 catalog_manager.cc:5697] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:21:20.763005 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 54015.3us
W20260709 13:21:20.763505 13349 leader_election.cc:336] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:20.764640 12746 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:21:20.773716 20895 log.cc:927] Time spent T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:21:20.772426 12531 catalog_manager.cc:5697] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:20.776067 12531 catalog_manager.cc:5697] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:20.777750 20693 raft_consensus.cc:493] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:20.778439 20693 raft_consensus.cc:515] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:20.780750 20693 leader_election.cc:290] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:20.784578 12531 catalog_manager.cc:5697] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 19 to 22. New cstate: current_term: 22 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:20.786881 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:20.786540 12531 catalog_manager.cc:5697] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:20.797770 20589 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:20.798466 20589 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:20.739115 20805 raft_consensus.cc:515] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:20.802940 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 47904.9us
I20260709 13:21:20.806838 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:20.812848 11517 tablet_replica.cc:333] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:20.814112 11517 raft_consensus.cc:2243] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:21:20.815156 20782 raft_consensus.cc:2749] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:21:20.815838 11517 pending_rounds.cc:70] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e: Trying to abort 3 pending ops.
I20260709 13:21:20.816179 11517 pending_rounds.cc:77] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e: Aborting op as it isn't in flight: id { term: 20 index: 15 } timestamp: 7305639033603989504 op_type: NO_OP noop_request { }
I20260709 13:21:20.816636 11517 raft_consensus.cc:2889] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:21:20.816948 11517 pending_rounds.cc:77] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e: Aborting op as it isn't in flight: id { term: 18 index: 14 } timestamp: 7305639002852593664 op_type: NO_OP noop_request { }
I20260709 13:21:20.817350 11517 raft_consensus.cc:2889] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:21:20.819909 11517 pending_rounds.cc:77] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e: Aborting op as it isn't in flight: id { term: 16 index: 13 } timestamp: 7305638959386808320 op_type: NO_OP noop_request { }
I20260709 13:21:20.820271 11517 raft_consensus.cc:2889] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 20 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:21:20.793515 20792 raft_consensus.cc:2804] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:21:20.831151 13385 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:21:20.837743 12709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:20.839136 11517 raft_consensus.cc:2272] T 612a4639a7a749be8856bbc60766e312 P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:20.839895 20589 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:20.840901 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:20.841874 20782 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:20.822736 20792 raft_consensus.cc:697] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 23 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:20.844339 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:20.847275 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:20.852761 20782 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:20.854884 12700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:20.856997 12708 raft_consensus.cc:2393] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 522fd2ba0e9f402691b5ef29af8579cd in current term 28: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:20.866542 12706 raft_consensus.cc:1240] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Rejecting Update request from peer 522fd2ba0e9f402691b5ef29af8579cd for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:20.869002 12702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:20.869499 20782 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:20.869779 12702 raft_consensus.cc:3060] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:20.874060 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79ffe63999346abaccd90ab600091d7" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:20.874342 12707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd1bc5f9b25e46cc866068bd346e11c6" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 22 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:20.875005 20792 consensus_queue.cc:237] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:20.876570 20665 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:20.877867 11517 tablet_replica.cc:333] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:20.878925 20805 leader_election.cc:290] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:20.879336 11517 raft_consensus.cc:2243] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:20.881319 12702 raft_consensus.cc:2468] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 26.
I20260709 13:21:20.854269 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20db0f202ea546699789c2325d241b35" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:20.884827 20771 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:20.887895 13385 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:21:20.894438 20771 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:20.895227 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea604dddf4c244a181921c4dbdd0e720" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:20.895905 12704 raft_consensus.cc:3060] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:20.851666 20883 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:20.898077 20883 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:20.899407 12705 raft_consensus.cc:1275] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:21:20.900373 20883 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:20.856033 20785 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 13:21:20.901933 11517 raft_consensus.cc:2272] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:20.889063 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:20.904870 12708 raft_consensus.cc:2410] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:21:20.906509 12701 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
W20260709 13:21:20.908316 13349 leader_election.cc:336] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:20.908872 12597 leader_election.cc:304] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:20.897316 20771 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:20.909966 20693 raft_consensus.cc:2749] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:20.905426 11517 tablet_replica.cc:333] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:20.848176 20717 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:20.911375 20717 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:20.911960 11517 raft_consensus.cc:2243] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:20.916062 20844 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:20.918004 13349 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:20.913349 11517 raft_consensus.cc:2272] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:20.921244 12673 leader_election.cc:336] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:20.923038 12598 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:20.923138 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:20.893605 12700 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 22.
I20260709 13:21:20.925320 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:20.928073 12703 raft_consensus.cc:2468] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 19.
I20260709 13:21:20.935622 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:20.938313 20467 raft_consensus.cc:2804] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:21:20.942631 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:20.938796 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18b2f67fae7f4963a4b4c562003df2fe" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
W20260709 13:21:20.945016 12598 raft_consensus.cc:2686] T bd1bc5f9b25e46cc866068bd346e11c6 P 19cd864c053744bebd96093a7982537e: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:21:20.948674 13349 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:20.938570 20894 consensus_queue.cc:1048] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 13:21:20.943375 20723 consensus_queue.cc:1059] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.008s
I20260709 13:21:20.942993 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:20.954423 13384 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 22.
I20260709 13:21:20.938817 20885 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:21:20.959450 12701 raft_consensus.cc:2393] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 19cd864c053744bebd96093a7982537e in current term 26: Already voted for candidate 522fd2ba0e9f402691b5ef29af8579cd in this term.
I20260709 13:21:20.960886 11517 tablet_replica.cc:333] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:20.961388 20717 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:20.962438 11517 raft_consensus.cc:2243] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:20.963136 11517 raft_consensus.cc:2272] T ccf3f471aa044537a9492a86bb48b0aa P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:20.966481 11517 tablet_replica.cc:333] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:20.967511 11517 raft_consensus.cc:2243] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:20.968099 11517 raft_consensus.cc:2272] T ad2866c2079c4c5b9df5ad6a072e192d P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:20.971077 20844 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:20.971587 12704 raft_consensus.cc:2468] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
I20260709 13:21:20.949059 12672 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:20.981803 12598 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:20.982769 20717 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:20.985280 12673 leader_election.cc:336] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:20.986091 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fa805ee07a7429abe6b520bbfb94cd9" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:20.986722 12673 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:21:20.993258 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 44675.5us
I20260709 13:21:20.994993 20723 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:20.995402 20723 raft_consensus.cc:493] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:20.995776 20723 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:21.002912 12747 leader_election.cc:304] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:21.004868 20828 raft_consensus.cc:2804] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:20.985778 20467 raft_consensus.cc:697] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 26 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:21.005409 20828 raft_consensus.cc:697] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:20.990002 20883 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:20.990649 20771 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:21.007431 20771 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:21.007664 12704 raft_consensus.cc:1275] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:21:21.007781 20771 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:21.008396 20828 consensus_queue.cc:237] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:20.988276 20838 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:21.010298 12747 leader_election.cc:336] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:21.005446 20905 log.cc:927] Time spent T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:21:21.010295 20467 consensus_queue.cc:237] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:21.015585 20838 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:21.023325 20838 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:21.029691 12704 raft_consensus.cc:1275] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:21:20.992396 20894 raft_consensus.cc:3055] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:21.039316 20894 raft_consensus.cc:740] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:21.048281 20805 raft_consensus.cc:493] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:21.048996 20805 raft_consensus.cc:515] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:21.050616 12747 consensus_peers.cc:597] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:21.051278 12747 leader_election.cc:336] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:21.051668 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:21.052611 20571 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:21.053000 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:21.053375 20805 leader_election.cc:290] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:21.040752 20894 consensus_queue.cc:260] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:21.059211 20894 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:21.059765 11517 tablet_replica.cc:333] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:21.061462 20571 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 13:21:21.062608 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 114335us
I20260709 13:21:21.063601 20645 consensus_queue.cc:1048] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:21:21.065048 20805 raft_consensus.cc:493] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:21.072091 20805 raft_consensus.cc:515] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:21.074059 20717 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:21.074450 20805 leader_election.cc:290] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:21.074745 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 96240.6us
I20260709 13:21:21.075515 11517 raft_consensus.cc:2243] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:21:21.075543 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:21.077390 11517 raft_consensus.cc:2272] T e390d5b6aea04f09ab5dbf11b20a7b98 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.078378 20645 consensus_queue.cc:1048] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:21:21.074667 20693 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.006s
W20260709 13:21:21.090062 12672 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 141060us
W20260709 13:21:21.097932 12747 leader_election.cc:336] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:21.098443 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:21.039647 20885 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:21:21.102339 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3938a96da1794e32b484ac729226a8ce" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:21.080178 20771 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:21.104343 12747 leader_election.cc:336] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:21.106947 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:21.112388 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:21.083384 20693 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:21.113304 20892 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:21:21.113615 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db63389cf0d45de92e4b01621db07cd" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:21.113337 20693 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:21.116864 20693 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:21.112553 12706 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 28.
I20260709 13:21:21.121148 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6cd31a55742463db9f0f0060bb60a77" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:21.128118 12598 leader_election.cc:304] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:21.129037 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:21.083534 20723 raft_consensus.cc:515] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:21.132155 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:21.130043 12598 raft_consensus.cc:2686] T 18b2f67fae7f4963a4b4c562003df2fe P 19cd864c053744bebd96093a7982537e: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:21:21.134047 12782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:21.134799 12782 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:21:21.135251 12598 leader_election.cc:304] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:21.130805 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 153826us
W20260709 13:21:21.139407 12673 leader_election.cc:336] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:21.140489 12673 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:21.141386 20883 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:21.137207 12598 leader_election.cc:304] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:21.130769 12747 leader_election.cc:336] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:21.139362 12672 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:21.143561 12598 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:21.155479 13379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4e4ae9d38ba4fd0a700998e4f4750be" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:21.157960 13379 raft_consensus.cc:3060] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:21:21.152755 12747 leader_election.cc:336] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:21.159492 12747 leader_election.cc:336] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:21.145399 20805 raft_consensus.cc:2749] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:21:21.153489 20867 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:21:21.171516 12517 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 612a4639a7a749be8856bbc60766e312 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER failed with leader 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 639ab2db265f4747bb7afb93e833f530 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:21.177717 20844 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:21.178951 12672 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:21.160418 12747 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:21.166950 20771 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:21.153056 20889 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:21:21.196484 20889 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:21.197002 20889 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:21:21.198699 20838 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:21.199195 20838 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:21.199577 20838 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:21.153877 20785 raft_consensus.cc:2749] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:21:21.145850 20885 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:21.178269 20844 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:21.180106 12673 leader_election.cc:336] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:21.185712 12703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:21.186880 12598 leader_election.cc:304] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:21.236482 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 77018.8us
W20260709 13:21:21.215248 12673 leader_election.cc:336] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:21.240896 12598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 54176.7us
I20260709 13:21:21.251480 12598 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:21.246515 13379 raft_consensus.cc:2468] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 24.
I20260709 13:21:21.248395 20867 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:21.255713 20907 log.cc:927] Time spent T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 13:21:21.263474 13384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:21.265224 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6609d5f93ab048558ea46e511ef3b2ec" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:21.266055 13381 raft_consensus.cc:3060] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:21.264443 13384 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 28.
I20260709 13:21:21.241043 20844 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:21.258368 20913 log.cc:927] Time spent T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb: Append to log took a long time: real 0.163s	user 0.000s	sys 0.002s
I20260709 13:21:21.258256 20693 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:21.252740 20883 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:21.267225 20805 raft_consensus.cc:2749] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:21.305099 11517 tablet_replica.cc:333] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:21.263027 20838 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:21.324658 20883 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:21.337941 20723 leader_election.cc:290] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:21.300334 20894 raft_consensus.cc:493] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:21.344054 11517 raft_consensus.cc:2243] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Raft consensus shutting down.
W20260709 13:21:21.349961 13349 consensus_peers.cc:597] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:21.360608 13349 leader_election.cc:336] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:21.363248 13381 raft_consensus.cc:2468] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 17.
I20260709 13:21:21.389434 11517 raft_consensus.cc:2272] T b1054d44ad14423fb3d51a5243499bb9 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.376991 20467 raft_consensus.cc:493] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:21.372937 20894 raft_consensus.cc:515] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:21.423362 12708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6146f826f1904f6f8b10ad518d454c22" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:21.424165 12708 raft_consensus.cc:3060] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:21.422861 12672 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:21.428678 12672 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:21.440909 20912 raft_consensus.cc:2804] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:21.446007 20894 leader_election.cc:290] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:21.450364 20915 log.cc:927] Time spent T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
W20260709 13:21:21.451761 20896 log.cc:927] Time spent T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:21:21.399580 20889 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:21.454393 12673 leader_election.cc:336] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:21.454677 20889 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 29 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:21.432416 20911 raft_consensus.cc:2804] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:21:21.455819 20911 raft_consensus.cc:697] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 24 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:21.456904 20911 consensus_queue.cc:237] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:21.411072 20467 raft_consensus.cc:515] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:21.460283 12673 consensus_peers.cc:597] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:21.460357 20467 leader_election.cc:290] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:21.460872 12673 consensus_peers.cc:597] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:21.467828 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:21.473855 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:21.474186 20883 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:21.493114 11517 tablet_replica.cc:333] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:21.494061 11517 raft_consensus.cc:2243] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:21.495007 11517 raft_consensus.cc:2272] T 981fb2f435e24d06a81fc02cf4591cc5 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.451812 20912 raft_consensus.cc:697] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:21.499019 20912 consensus_queue.cc:237] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:21.505820 20933 raft_consensus.cc:493] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:21.506464 20933 raft_consensus.cc:515] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:21.510212 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:21.510407 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "522bcf0a155f4da196cb33f1aae90a82" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:21.509146 20933 leader_election.cc:290] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:21.515262 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c13fa2aff324f65ad0bf6191ea68fab" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:21.524911 12705 raft_consensus.cc:1275] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 13:21:21.526607 20838 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:21.528713 11517 tablet_replica.cc:333] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:21.529876 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:21.530573 12701 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 28 FOLLOWER]: Advancing to term 29
W20260709 13:21:21.521544 13349 consensus_peers.cc:597] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:21.538586 13385 raft_consensus.cc:1275] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:21:21.529346 20844 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:21.547457 20844 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:21.554801 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccf3f471aa044537a9492a86bb48b0aa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:21.555848 12779 raft_consensus.cc:3060] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:21.557108 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 68905.4us
I20260709 13:21:21.554276 12708 raft_consensus.cc:2468] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 23.
I20260709 13:21:21.578841 20828 consensus_queue.cc:1048] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:21:21.568221 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:21.582762 11517 raft_consensus.cc:2243] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:21:21.586833 12673 leader_election.cc:336] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:21.590816 11517 raft_consensus.cc:2272] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 19cd864c053744bebd96093a7982537e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.591861 12701 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 29.
I20260709 13:21:21.592972 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7827272b424bad88349b6b7437844b" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:21.595373 12779 raft_consensus.cc:2468] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
I20260709 13:21:21.593145 12598 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530; no voters: 
W20260709 13:21:21.595314 12673 leader_election.cc:336] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:21.601133 20933 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 29 FOLLOWER]: Leader election won for term 29
I20260709 13:21:21.606422 20933 raft_consensus.cc:697] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 29 LEADER]: Becoming Leader. State: Replica: 19cd864c053744bebd96093a7982537e, State: Running, Role: LEADER
I20260709 13:21:21.607492 20933 consensus_queue.cc:237] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:21.598004 12672 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:21.616616 12598 leader_election.cc:304] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:21.618429 20844 raft_consensus.cc:2804] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:21.619145 12701 raft_consensus.cc:1275] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:21:21.620437 20785 raft_consensus.cc:2749] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:21.619858 20844 raft_consensus.cc:697] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:21.623361 20906 consensus_queue.cc:1048] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.030s
I20260709 13:21:21.624094 20844 consensus_queue.cc:237] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:21.626770 12673 leader_election.cc:336] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:21.627210 11517 tablet_replica.cc:333] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:21.628315 11517 raft_consensus.cc:2243] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:21.629101 11517 pending_rounds.cc:70] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e: Trying to abort 1 pending ops.
I20260709 13:21:21.648844 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb7fca96394f4ba9b4fe3d7a6e278878" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:21.649638 13385 raft_consensus.cc:3060] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:21.651379 13349 leader_election.cc:304] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
W20260709 13:21:21.641156 12673 consensus_peers.cc:597] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:21.636976 13348 leader_election.cc:304] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:21.653906 12672 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:21.629383 11517 pending_rounds.cc:77] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e: Aborting op as it isn't in flight: id { term: 23 index: 15 } timestamp: 7305639017307308032 op_type: NO_OP noop_request { }
I20260709 13:21:21.656834 11517 raft_consensus.cc:2889] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:21:21.657610 11517 raft_consensus.cc:2272] T 0db63389cf0d45de92e4b01621db07cd P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.658521 13348 leader_election.cc:304] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:21.657397 20894 raft_consensus.cc:2749] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:21:21.660846 20723 raft_consensus.cc:2804] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:21:21.661741 20883 raft_consensus.cc:2749] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:21.661854 20867 consensus_queue.cc:1048] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:21.664551 20739 raft_consensus.cc:2749] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:21:21.657879 12673 consensus_peers.cc:597] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:21.679296 13349 consensus_peers.cc:597] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:21.683871 13385 raft_consensus.cc:2468] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 23.
W20260709 13:21:21.684613 12747 consensus_peers.cc:597] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:21.685745 12747 consensus_peers.cc:597] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:21.675169 20723 raft_consensus.cc:697] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:21.687028 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:21.687714 13382 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:21:21.687484 20723 consensus_queue.cc:237] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:21.703199 13382 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19cd864c053744bebd96093a7982537e in term 29.
W20260709 13:21:21.685547 13349 consensus_peers.cc:597] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:21.711505 12672 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
W20260709 13:21:21.695911 12747 consensus_peers.cc:597] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:21.715590 20844 raft_consensus.cc:2804] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:21:21.721240 13349 consensus_peers.cc:597] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:21.721271 12747 consensus_peers.cc:597] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:21.727993 20589 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:21.729369 20589 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:21.735219 12747 consensus_peers.cc:597] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:21.737843 11517 tablet_replica.cc:333] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
W20260709 13:21:21.727630 13349 consensus_peers.cc:597] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:21.738914 11517 raft_consensus.cc:2243] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:21.727754 20844 raft_consensus.cc:697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 23 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:21.739631 11517 raft_consensus.cc:2272] T e1a0f7a1130449548d1659cd21760fc9 P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.741955 20589 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:21.741863 20844 consensus_queue.cc:237] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:21.744413 11517 tablet_replica.cc:333] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:21.745363 11517 raft_consensus.cc:2243] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Raft consensus shutting down.
W20260709 13:21:21.736778 12747 consensus_peers.cc:597] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:21.746831 12673 consensus_peers.cc:597] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:21.745872 13349 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 237 similar messages]
W20260709 13:21:21.748189 12747 consensus_peers.cc:597] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:21.755875 20883 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:21.756471 20883 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:21.759841 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
W20260709 13:21:21.760442 13349 consensus_peers.cc:597] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:21.761060 13349 leader_election.cc:336] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:21.762274 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:21.762923 12704 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 18.
W20260709 13:21:21.765834 12673 leader_election.cc:336] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:21.766139 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:21.766314 12673 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:21.767365 20883 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:21.765232 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:21.772395 20589 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:21.772416 20931 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:21.773021 20589 raft_consensus.cc:493] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:21.773602 20589 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:21.786230 11517 pending_rounds.cc:70] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e: Trying to abort 1 pending ops.
I20260709 13:21:21.786669 11517 pending_rounds.cc:77] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e: Aborting op as it isn't in flight: id { term: 16 index: 15 } timestamp: 7305638979006701568 op_type: NO_OP noop_request { }
I20260709 13:21:21.787149 11517 raft_consensus.cc:2889] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:21:21.789759 11517 raft_consensus.cc:2272] T 4d53f974eee5475b9cd2701948f533ac P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.795637 20589 raft_consensus.cc:515] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:21.799031 20589 leader_election.cc:290] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:21.809461 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b13b87a5cea4263937041d123c5889d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:21.810308 12705 raft_consensus.cc:3060] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:21.816288 13349 leader_election.cc:336] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:21.835198 11517 tablet_replica.cc:333] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:21.836522 11517 raft_consensus.cc:2243] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:21.837366 11517 raft_consensus.cc:2272] T c6cd31a55742463db9f0f0060bb60a77 P 19cd864c053744bebd96093a7982537e [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:21.868907 12673 consensus_peers.cc:597] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:21.882843 12705 raft_consensus.cc:2468] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 19.
I20260709 13:21:21.898751 11517 tablet_replica.cc:333] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:21.899791 11517 raft_consensus.cc:2243] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:21.900388 11517 pending_rounds.cc:70] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e: Trying to abort 1 pending ops.
I20260709 13:21:21.900648 11517 pending_rounds.cc:77] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e: Aborting op as it isn't in flight: id { term: 19 index: 12 } timestamp: 7305639001194004480 op_type: NO_OP noop_request { }
I20260709 13:21:21.901014 11517 raft_consensus.cc:2889] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:21:21.901355 11517 raft_consensus.cc:2272] T bc7827272b424bad88349b6b7437844b P 19cd864c053744bebd96093a7982537e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:21.912196 13349 leader_election.cc:304] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:21.913606 20885 raft_consensus.cc:2804] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:21.914187 20885 raft_consensus.cc:697] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 19 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:21.916473 20885 consensus_queue.cc:237] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:21.958238 12525 catalog_manager.cc:5697] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 16 to 17, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 17 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:21.960217 12525 catalog_manager.cc:5697] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:21:21.961750 12525 catalog_manager.cc:5697] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 16 to 18, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 18 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:21.963297 12525 catalog_manager.cc:5697] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 18 to 20, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 20 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:21:21.967818 12531 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54814 (request call id 313) took 9421 ms. Trace:
I20260709 13:21:21.967944 13381 raft_consensus.cc:1275] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
W20260709 13:21:21.968093 12673 consensus_peers.cc:597] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:21.979012 20844 consensus_queue.cc:1048] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:21:21.978645 12525 catalog_manager.cc:5697] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 18 to 21, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 21 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:21.981093 12525 catalog_manager.cc:5697] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 22 to 24, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 24 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:21.982769 12525 catalog_manager.cc:5697] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 22 to 24, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 24 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:21:21.984193 12525 catalog_manager.cc:5697] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 19 to 20, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 20 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:21.994777 12525 catalog_manager.cc:5697] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 17 to 20, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 20 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:21.997117 12525 catalog_manager.cc:5697] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 12 to 15, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 15 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:21.999310 12525 catalog_manager.cc:5697] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 17 to 21, leader changed from 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 21 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:21.978338 12531 rpcz_store.cc:276] 0709 13:21:12.545959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:12.546404 (+   445us) service_pool.cc:224] Handling call
0709 13:21:21.967758 (+9421354us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:21.017487 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:21.018226 (+   739us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:21.018274 (+    48us) write_op.cc:435] Acquired schema lock
0709 13:21:21.018291 (+    17us) tablet.cc:662] Decoding operations
0709 13:21:21.020105 (+  1814us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:21.020233 (+   128us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:21.020260 (+    27us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:21:21.060557 (+ 40297us) tablet.cc:701] Row locks acquired
0709 13:21:21.060607 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:21:21.060784 (+   177us) write_op.cc:270] Start()
0709 13:21:21.060920 (+   136us) write_op.cc:276] Timestamp: P: 1783603281060720 usec, L: 0
0709 13:21:21.060960 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 13:21:21.061736 (+   776us) log.cc:844] Serialized 7019 byte log entry
0709 13:21:21.715474 (+653738us) op_driver.cc:464] REPLICATION: finished
0709 13:21:21.801028 (+ 85554us) spinlock_profiling.cc:243] Waited 6.05 ms on lock 0x7b18002432e0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2932a6 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c53f3 00007f4c6a2c77c1
0709 13:21:21.801272 (+   244us) write_op.cc:301] APPLY: starting
0709 13:21:21.801401 (+   129us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:21.831193 (+ 29792us) spinlock_profiling.cc:243] Waited 1.91 ms on lock 0x7b18002432e0. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c7318dde1 00007f4c7319a5b1 00007f4c6a2c5557 00007f4c6a2c77c1 00007f4c6a2c7779 00007f4c6a2c7709 00007f4c6a2c76d1 00007f4c6a2c69cd 000000000038f354 000000000038f199 00007f4c6a2a883a
0709 13:21:21.848702 (+ 17509us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:21.848736 (+    34us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:21.907007 (+ 58271us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:21.907673 (+   666us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=36,mrs_lookups=0
0709 13:21:21.907715 (+    42us) write_op.cc:312] APPLY: finished
0709 13:21:21.908314 (+   599us) log.cc:844] Serialized 164 byte log entry
0709 13:21:21.908491 (+   177us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:21.908831 (+   340us) write_op.cc:454] Released schema lock
0709 13:21:21.908911 (+    80us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:21.920581 (+ 11670us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:21.920617 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1747243,"reported_tablets":40,"spinlock_wait_cycles":17408,"child_traces":[["op",{"apply.queue_time_us":73416,"apply.run_cpu_time_us":25979,"apply.run_wall_time_us":119255,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79439,"num_ops":18,"prepare.queue_time_us":162980,"prepare.run_cpu_time_us":11510,"prepare.run_wall_time_us":76773,"raft.queue_time_us":356261,"raft.run_cpu_time_us":27902,"raft.run_wall_time_us":73334,"replication_time_us":654346,"spinlock_wait_cycles":19825536,"thread_start_us":499577,"threads_started":4,"wal-append.queue_time_us":32334}]]}
I20260709 13:21:22.011602 11517 tablet_replica.cc:333] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:22.034628 20723 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:22.036379 20723 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:22.041401 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:22.042110 12705 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 20.
I20260709 13:21:22.042888 20723 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:22.048537 11517 raft_consensus.cc:2243] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:22.049427 11517 pending_rounds.cc:70] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: Trying to abort 1 pending ops.
I20260709 13:21:22.054876 11517 pending_rounds.cc:77] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e: Aborting op as it isn't in flight: id { term: 22 index: 13 } timestamp: 7305638979685715968 op_type: NO_OP noop_request { }
I20260709 13:21:22.055423 11517 raft_consensus.cc:2889] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
W20260709 13:21:22.061517 13349 leader_election.cc:336] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:22.069402 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:22.069667 13385 raft_consensus.cc:1275] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:21:22.069705 11517 raft_consensus.cc:2272] T fa34731f37084c608660f0088a2dba09 P 19cd864c053744bebd96093a7982537e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:22.071075 20844 consensus_queue.cc:1048] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:22.075106 13348 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:22.075938 20894 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:21:22.077806 12747 consensus_peers.cc:597] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:22.093719 13349 consensus_peers.cc:597] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:22.098235 11517 tablet_replica.cc:333] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:22.105427 20933 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:22.105767 11517 raft_consensus.cc:2243] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:22.106034 20933 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:22.108228 20933 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:22.106664 11517 raft_consensus.cc:2272] T 5067d5ea86fa4774a6be5b882e5b289d P 19cd864c053744bebd96093a7982537e [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:22.109999 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:22.114425 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f0a3339d324fd4902b6ff970ff0272" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:22.118705 12673 consensus_peers.cc:597] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:22.124509 12597 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:22.126286 20785 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:22.129088 11517 tablet_replica.cc:333] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:22.130014 11517 raft_consensus.cc:2243] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:22.134465 11517 raft_consensus.cc:2272] T d46747dede234934bea0f0c324106c28 P 19cd864c053744bebd96093a7982537e [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:22.130800 13382 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 29 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
W20260709 13:21:22.135792 12673 consensus_peers.cc:597] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:22.188952 20873 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:22.230640 20873 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:22.233273 20873 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:22.234187 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 98440.7us
I20260709 13:21:22.234540 13385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:22.235169 13385 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:21:22.236307 12746 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:22.185447 11517 tablet_replica.cc:333] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:22.237944 11517 raft_consensus.cc:2243] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:22.238529 11517 raft_consensus.cc:2272] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:22.222335 12705 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 29 FOLLOWER]: Refusing update from remote peer 19cd864c053744bebd96093a7982537e: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 13:21:22.222375 12706 raft_consensus.cc:1275] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:21:22.243491 12779 raft_consensus.cc:1275] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:21:22.244207 20873 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:22.244709 20873 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 522fd2ba0e9f402691b5ef29af8579cd)
I20260709 13:21:22.245134 20873 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:22.246549 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:22.247275 12706 raft_consensus.cc:2393] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 19: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:22.247912 20933 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:21:22.252501 20844 consensus_queue.cc:1048] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:21:22.253813 12673 consensus_peers.cc:597] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:22.255185 12673 consensus_peers.cc:597] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:22.228464 20949 log.cc:927] Time spent T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:21:22.200230 20563 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:22.256953 20563 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:22.259121 20894 consensus_queue.cc:1048] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:22.259833 20563 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:22.262545 13381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:22.263221 13381 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
I20260709 13:21:22.264906 20785 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 13:21:22.267433 11517 tablet_replica.cc:333] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:22.268424 11517 raft_consensus.cc:2243] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:21:22.269498 11517 raft_consensus.cc:2272] T 414b06f3e5f8472ea4071ebdd0161b9d P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:22.277549 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:22.278190 12704 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 20.
I20260709 13:21:22.278663 20933 raft_consensus.cc:493] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:22.279294 20933 raft_consensus.cc:515] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:22.286087 12747 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:22.296427 20867 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:22.299347 20867 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:22.299760 20867 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:21:22.312152 13349 consensus_peers.cc:597] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:22.307331 12747 leader_election.cc:336] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:22.312098 11517 tablet_replica.cc:333] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:22.326103 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:22.342073 11517 raft_consensus.cc:2243] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:21:22.343371 11517 raft_consensus.cc:2272] T 5febc2df9d3c41359ceccc1bef8a54d8 P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:22.353088 12705 raft_consensus.cc:1275] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:21:22.356242 20906 consensus_queue.cc:1048] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:21:22.364253 20933 leader_election.cc:290] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:22.364691 13349 consensus_peers.cc:597] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:22.367172 12525 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54800 (request call id 324) took 1945 ms. Trace:
I20260709 13:21:22.360764 12523 catalog_manager.cc:5697] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 21 to 22, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 22 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:22.369210 12523 catalog_manager.cc:5697] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:22.371198 12523 catalog_manager.cc:5697] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 15 to 18, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 18 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:22.387557 12523 catalog_manager.cc:5697] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:22.397979 12523 catalog_manager.cc:5697] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:22.402526 13382 raft_consensus.cc:1275] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 23 index: 19. (index mismatch)
I20260709 13:21:22.367527 12525 rpcz_store.cc:276] 0709 13:21:20.422114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:20.429890 (+  7776us) service_pool.cc:224] Handling call
0709 13:21:22.367075 (+1937185us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:22.044520 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:22.045059 (+   539us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:22.045094 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:21:22.045111 (+    17us) tablet.cc:662] Decoding operations
0709 13:21:22.046429 (+  1318us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:22.046505 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:22.046532 (+    27us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:21:22.047512 (+   980us) tablet.cc:701] Row locks acquired
0709 13:21:22.047530 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:21:22.047683 (+   153us) write_op.cc:270] Start()
0709 13:21:22.047791 (+   108us) write_op.cc:276] Timestamp: P: 1783603282047637 usec, L: 0
0709 13:21:22.047819 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:21:22.048514 (+   695us) log.cc:844] Serialized 4354 byte log entry
0709 13:21:22.051524 (+  3010us) op_driver.cc:464] REPLICATION: finished
0709 13:21:22.051864 (+   340us) write_op.cc:301] APPLY: starting
0709 13:21:22.051936 (+    72us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:22.062359 (+ 10423us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:22.062451 (+    92us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:22.182550 (+120099us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:22.182875 (+   325us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=22,mrs_lookups=0
0709 13:21:22.182920 (+    45us) write_op.cc:312] APPLY: finished
0709 13:21:22.183247 (+   327us) log.cc:844] Serialized 105 byte log entry
0709 13:21:22.184019 (+   772us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:22.184188 (+   169us) write_op.cc:454] Released schema lock
0709 13:21:22.184243 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:22.290280 (+106037us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:22.291765 (+  1485us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":542,"reported_tablets":40,"spinlock_wait_cycles":12544,"child_traces":[["op",{"apply.queue_time_us":112,"apply.run_cpu_time_us":15869,"apply.run_wall_time_us":132709,"cfile_cache_hit":44,"cfile_cache_hit_bytes":46024,"num_ops":11,"prepare.queue_time_us":37046,"prepare.run_cpu_time_us":4244,"prepare.run_wall_time_us":4314,"replication_time_us":3533,"thread_start_us":12857,"threads_started":1}]]}
I20260709 13:21:22.407099 12523 catalog_manager.cc:5697] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 11 to 13, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New cstate: current_term: 16 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:22.411459 12523 catalog_manager.cc:5697] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 20 to 21, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 21 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:22.412746 12523 catalog_manager.cc:5697] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 17 to 18, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 18 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:22.424329 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1362166bcbb48e88ac7b5862a23517e" candidate_uuid: "19cd864c053744bebd96093a7982537e" candidate_term: 26 candidate_status { last_received { term: 22 index: 20 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:22.426539 11517 tablet_replica.cc:333] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:22.427471 11517 raft_consensus.cc:2243] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:22.428108 11517 raft_consensus.cc:2272] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:22.434295 20911 consensus_queue.cc:1048] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 13:21:22.436085 12673 consensus_peers.cc:597] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:22.447245 12597 leader_election.cc:304] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19cd864c053744bebd96093a7982537e; no voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:22.452072 12597 raft_consensus.cc:2686] T d1362166bcbb48e88ac7b5862a23517e P 19cd864c053744bebd96093a7982537e: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:21:22.459836 11517 tablet_replica.cc:333] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:22.460177 20873 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:22.461249 11517 raft_consensus.cc:2243] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:21:22.450516 12523 catalog_manager.cc:5697] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 21 to 22, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 22 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:22.473820 20873 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:22.490979 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:22.493301 11517 raft_consensus.cc:2272] T 3ca762dedae144be828acb659b8d9e4b P 19cd864c053744bebd96093a7982537e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:22.494300 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:22.502873 12705 raft_consensus.cc:2393] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 19: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:22.506880 12523 catalog_manager.cc:5697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 17 to 18, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 18 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:22.518204 13383 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 19.
I20260709 13:21:22.515127 12523 catalog_manager.cc:5697] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 20 to 21, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 21 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:22.530085 12523 catalog_manager.cc:5697] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:22.531661 12523 catalog_manager.cc:5697] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: config changed from index 14 to 15, VOTER 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) evicted. New cstate: current_term: 19 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:22.533283 12523 catalog_manager.cc:5697] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 26 to 27, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 27 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:22.537797 12523 catalog_manager.cc:5697] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 16 to 20, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 19cd864c053744bebd96093a7982537e (127.11.63.65). New cstate: current_term: 20 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:22.545011 12746 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:22.554806 20929 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:22.555377 20929 raft_consensus.cc:697] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:22.556322 20929 consensus_queue.cc:237] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:22.560017 13349 consensus_peers.cc:597] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:22.560779 13349 consensus_peers.cc:597] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:22.527714 20867 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:22.587296 11517 tablet_replica.cc:333] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:22.592286 11517 raft_consensus.cc:2243] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 29 LEADER]: Raft consensus shutting down.
I20260709 13:21:22.593767 11517 raft_consensus.cc:2272] T 15d86ce6de4b43719dc715ab6f4de742 P 19cd864c053744bebd96093a7982537e [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:22.588781 20961 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:22.602841 20961 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:22.632442 20883 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:22.632788 20932 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:22.633490 20932 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:22.633317 20883 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:22.635926 20932 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:22.636497 20867 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:22.637861 20883 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:22.648250 11517 tablet_replica.cc:333] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e: stopping tablet replica
W20260709 13:21:22.648415 12673 leader_election.cc:336] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:22.649744 11517 raft_consensus.cc:2243] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:22.650671 11517 raft_consensus.cc:2272] T 3938a96da1794e32b484ac729226a8ce P 19cd864c053744bebd96093a7982537e [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:22.655920 11517 tablet_replica.cc:333] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:22.667908 11517 raft_consensus.cc:2243] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:22.687032 11517 raft_consensus.cc:2272] T 775128ed7b8d4283bf303c8d40247615 P 19cd864c053744bebd96093a7982537e [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:22.686893 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 80264us
I20260709 13:21:22.648376 20961 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:22.704615 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:22.705289 13377 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
I20260709 13:21:22.707603 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:22.708230 12704 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:22.709197 12705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:22.709887 12705 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:21:22.711268 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612a4639a7a749be8856bbc60766e312" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:22.711925 13383 raft_consensus.cc:3060] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:22.713640 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:22.716952 20932 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:22.717622 20932 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:22.718024 20932 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:22.721479 12704 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 21.
I20260709 13:21:22.726646 20932 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:22.731652 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:22.732959 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:22.734174 12775 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 16.
I20260709 13:21:22.738631 11517 tablet_replica.cc:333] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
W20260709 13:21:22.744845 13349 consensus_peers.cc:597] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:22.746300 20932 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 election: Requested vote from peers 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:22.757716 11517 raft_consensus.cc:2243] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:22.758854 11517 raft_consensus.cc:2272] T f2f0a3339d324fd4902b6ff970ff0272 P 19cd864c053744bebd96093a7982537e [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:22.761348 13377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:22.761981 13377 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:22.768770 13383 raft_consensus.cc:2468] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 21.
I20260709 13:21:22.769950 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:22.772969 12672 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:22.767652 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:22.782946 20844 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:22.783457 20844 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:22.783871 20844 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:22.786330 20929 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:22.786828 20929 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:22.787251 20929 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:22.791167 13377 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 20.
I20260709 13:21:22.821919 12746 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:22.825505 20965 raft_consensus.cc:2804] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:21:22.824110 12747 consensus_peers.cc:597] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:22.826246 20965 raft_consensus.cc:697] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
W20260709 13:21:22.826725 12747 leader_election.cc:336] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:22.827723 12747 leader_election.cc:336] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:22.827368 20965 consensus_queue.cc:237] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:22.829741 12747 consensus_peers.cc:597] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:22.851018 11517 tablet_replica.cc:333] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e: stopping tablet replica
W20260709 13:21:22.863695 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 86685.8us
I20260709 13:21:22.864221 11517 raft_consensus.cc:2243] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:22.868140 12672 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:22.889607 11517 raft_consensus.cc:2272] T 6609d5f93ab048558ea46e511ef3b2ec P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:22.897552 20911 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:22.919464 20911 raft_consensus.cc:697] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 20 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:22.920112 20844 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:22.920555 20911 consensus_queue.cc:237] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:22.917085 20929 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:22.947623 12780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:22.948632 12780 raft_consensus.cc:2393] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 639ab2db265f4747bb7afb93e833f530 in current term 17: Already voted for candidate 3d539f8d95d343d8b1696803f18b30d2 in this term.
I20260709 13:21:22.952953 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:22.954602 12706 raft_consensus.cc:2393] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d539f8d95d343d8b1696803f18b30d2 in current term 17: Already voted for candidate 639ab2db265f4747bb7afb93e833f530 in this term.
I20260709 13:21:22.941448 20844 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:22.959197 11517 tablet_replica.cc:333] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:22.964357 11517 raft_consensus.cc:2243] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:21:22.966840 11517 raft_consensus.cc:2272] T febe63321d6c417cbfc5ccbca7caeefa P 19cd864c053744bebd96093a7982537e [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:22.948649 20929 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:22.967507 12747 leader_election.cc:336] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:22.968789 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:22.969822 12673 leader_election.cc:336] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:22.974646 12673 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:22.979925 20931 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:22.974263 20929 raft_consensus.cc:2749] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:23.009318 11517 tablet_replica.cc:333] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e: stopping tablet replica
I20260709 13:21:23.010185 11517 raft_consensus.cc:2243] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.011042 11517 raft_consensus.cc:2272] T e79ffe63999346abaccd90ab600091d7 P 19cd864c053744bebd96093a7982537e [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:23.012060 12747 consensus_peers.cc:597] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:23.057154 12704 raft_consensus.cc:1240] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 20 LEADER]: Rejecting Update request from peer 3d539f8d95d343d8b1696803f18b30d2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:23.059768 20929 consensus_queue.cc:1059] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:23.060607 20929 consensus_queue.cc:1243] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Peer 639ab2db265f4747bb7afb93e833f530 log is divergent from this leader: its last log entry 20.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:21:23.061159 20929 raft_consensus.cc:3055] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:23.061509 20929 raft_consensus.cc:740] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:23.062567 20929 consensus_queue.cc:260] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:23.068663 20929 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:21:23.158114 12579 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:23.158614 12579 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:23.158937 12579 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:23.201879 12531 catalog_manager.cc:5697] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 20 to 23, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 23 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:21:23.203094 12517 catalog_manager.cc:4755] TS 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 98a43c5d78c945bb8c5541bc892f4865: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20260709 13:21:23.214357 12531 catalog_manager.cc:5697] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } }
I20260709 13:21:23.225322 12695 tablet_service.cc:1558] Processing DeleteTablet for tablet 20db0f202ea546699789c2325d241b35 with delete_type TABLET_DATA_TOMBSTONED (TS 639ab2db265f4747bb7afb93e833f530 not found in new config with opid_index 15) from {username='slave'} at 127.0.0.1:49378
I20260709 13:21:23.216741 12531 catalog_manager.cc:5697] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 21 to 26, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 26 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:23.240118 12531 catalog_manager.cc:5697] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 18 to 20, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 20 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:23.245597 12523 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40528 (request call id 232) took 2247 ms. Trace:
I20260709 13:21:23.246721 12523 rpcz_store.cc:276] 0709 13:21:20.996254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:20.996564 (+   310us) service_pool.cc:224] Handling call
0709 13:21:23.243498 (+2246934us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:22.736339 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:22.736937 (+   598us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:22.736988 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:21:22.737002 (+    14us) tablet.cc:662] Decoding operations
0709 13:21:22.738608 (+  1606us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:22.738665 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:22.738700 (+    35us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:21:22.739924 (+  1224us) tablet.cc:701] Row locks acquired
0709 13:21:22.739946 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:21:22.740118 (+   172us) write_op.cc:270] Start()
0709 13:21:22.740243 (+   125us) write_op.cc:276] Timestamp: P: 1783603282740063 usec, L: 0
0709 13:21:22.740270 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:21:22.757831 (+ 17561us) log.cc:844] Serialized 5912 byte log entry
0709 13:21:22.869347 (+111516us) op_driver.cc:464] REPLICATION: finished
0709 13:21:22.958354 (+ 89007us) write_op.cc:301] APPLY: starting
0709 13:21:22.958447 (+    93us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:23.042327 (+ 83880us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:23.042438 (+   111us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:23.121347 (+ 78909us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:23.121727 (+   380us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=30,mrs_lookups=0
0709 13:21:23.121786 (+    59us) write_op.cc:312] APPLY: finished
0709 13:21:23.122280 (+   494us) log.cc:844] Serialized 139 byte log entry
0709 13:21:23.136059 (+ 13779us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:23.136243 (+   184us) write_op.cc:454] Released schema lock
0709 13:21:23.136304 (+    61us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:23.169706 (+ 33402us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:23.169738 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":66928,"reported_tablets":39,"child_traces":[["op",{"apply.queue_time_us":88733,"apply.run_cpu_time_us":21490,"apply.run_wall_time_us":178389,"cfile_cache_hit":62,"cfile_cache_hit_bytes":66887,"num_ops":15,"prepare.queue_time_us":134849,"prepare.run_cpu_time_us":5400,"prepare.run_wall_time_us":53417,"replication_time_us":128889,"spinlock_wait_cycles":90496,"thread_start_us":131535,"threads_started":2}]]}
W20260709 13:21:23.285907 12747 consensus_peers.cc:597] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:23.314594 12780 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 13:21:23.315029 12780 pending_rounds.cc:85] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2: Aborting all ops after (but not including) 14
I20260709 13:21:23.315270 12780 pending_rounds.cc:107] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:21:23.315527 12780 raft_consensus.cc:2889] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:23.317291 20844 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:21:23.324067 12704 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:21:23.331662 20965 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:21:23.339191 13383 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:21:23.346146 20980 tablet_replica.cc:333] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:23.349005 20980 raft_consensus.cc:2243] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:23.360479 20844 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:21:23.366432 20844 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:23.366918 20980 raft_consensus.cc:2272] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:23.367048 20844 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:23.372452 12673 leader_election.cc:336] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:23.381634 13349 consensus_peers.cc:597] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:23.383332 20980 ts_tablet_manager.cc:1916] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:21:23.385428 20844 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:23.386962 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:23.398197 12775 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:21:23.401346 12672 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:23.402558 20883 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:23.402973 20883 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:23.403349 20883 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:23.406625 12525 catalog_manager.cc:5697] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 22 to 23, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 23 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:23.409325 13377 raft_consensus.cc:1275] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:21:23.411128 20965 consensus_queue.cc:1048] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:21:23.415673 20883 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:23.419855 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "414b06f3e5f8472ea4071ebdd0161b9d" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:23.420400 12775 raft_consensus.cc:3060] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:23.409341 12525 catalog_manager.cc:5697] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 24 to 26, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 26 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:23.428925 12531 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54814 (request call id 314) took 1304 ms. Trace:
I20260709 13:21:23.429576 12775 raft_consensus.cc:2468] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
I20260709 13:21:23.430370 20883 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:23.429526 12531 rpcz_store.cc:276] 0709 13:21:22.124053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:22.124202 (+   149us) service_pool.cc:224] Handling call
0709 13:21:22.137901 (+ 13699us) spinlock_profiling.cc:243] Waited 9.41 ms on lock 0x7b540011eed4. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31c090 00007f4c6d3baf3a 00007f4c731711cc 00007f4c7326f37e 00007f4c717302a4 00007f4c71730232 00007f4c71730161 00007f4c717300dc 00007f4c7172f392 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:21:23.428852 (+1290951us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:23.279832 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:23.280365 (+   533us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:23.280397 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:21:23.280415 (+    18us) tablet.cc:662] Decoding operations
0709 13:21:23.281011 (+   596us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:23.281067 (+    56us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:23.281100 (+    33us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:21:23.281485 (+   385us) tablet.cc:701] Row locks acquired
0709 13:21:23.281499 (+    14us) write_op.cc:260] PREPARE: finished
0709 13:21:23.281673 (+   174us) write_op.cc:270] Start()
0709 13:21:23.281798 (+   125us) write_op.cc:276] Timestamp: P: 1783603283281620 usec, L: 0
0709 13:21:23.281821 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:21:23.282457 (+   636us) log.cc:844] Serialized 1684 byte log entry
0709 13:21:23.284556 (+  2099us) op_driver.cc:464] REPLICATION: finished
0709 13:21:23.291000 (+  6444us) write_op.cc:301] APPLY: starting
0709 13:21:23.292470 (+  1470us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:23.302696 (+ 10226us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:23.302817 (+   121us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:23.320763 (+ 17946us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:23.323547 (+  2784us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 13:21:23.323605 (+    58us) write_op.cc:312] APPLY: finished
0709 13:21:23.329610 (+  6005us) log.cc:844] Serialized 49 byte log entry
0709 13:21:23.330405 (+   795us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:23.331183 (+   778us) write_op.cc:454] Released schema lock
0709 13:21:23.332672 (+  1489us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:23.341067 (+  8395us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:23.341118 (+    51us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":29559,"reported_tablets":29,"spinlock_wait_cycles":21648640,"child_traces":[["op",{"apply.queue_time_us":785,"apply.run_cpu_time_us":7832,"apply.run_wall_time_us":51323,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16736,"num_ops":4,"prepare.queue_time_us":246,"prepare.run_cpu_time_us":2863,"prepare.run_wall_time_us":2862,"replication_time_us":2618}]]}
I20260709 13:21:23.436137 12672 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:23.437314 20883 raft_consensus.cc:2804] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:23.439199 20883 raft_consensus.cc:697] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:23.440124 20883 consensus_queue.cc:237] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:23.428925 12525 catalog_manager.cc:5697] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 21 to 23, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 23 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:23.462210 12525 catalog_manager.cc:5697] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:23.474957 20973 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:23.475536 20973 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:23.477533 20973 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:23.479446 12673 leader_election.cc:336] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:23.472501 12525 catalog_manager.cc:5697] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 18 to 19, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 19 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:23.482465 12525 catalog_manager.cc:5697] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 17 to 19, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 19 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:23.483757 12704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:23.484159 12525 catalog_manager.cc:5697] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 27 to 29. New cstate: current_term: 29 leader_uuid: "19cd864c053744bebd96093a7982537e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:23.496307 20980 ts_tablet_manager.cc:1929] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 19.14
I20260709 13:21:23.497198 20980 log.cc:1199] T 20db0f202ea546699789c2325d241b35 P 639ab2db265f4747bb7afb93e833f530: Deleting WAL directory at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0/minicluster-data/ts-1-root/wals/20db0f202ea546699789c2325d241b35
I20260709 13:21:23.502465 12525 catalog_manager.cc:5697] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 22 to 24, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 24 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:21:23.515849 13349 consensus_peers.cc:597] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:23.516415 13349 leader_election.cc:336] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:23.516925 13349 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:23.518260 20973 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:23.563117 12517 catalog_manager.cc:5028] TS 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): tablet 20db0f202ea546699789c2325d241b35 (table test-workload [id=0a30ba58723d445b9932a2283c60055a]) successfully deleted
W20260709 13:21:23.585106 12747 consensus_peers.cc:597] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:23.618835 12531 catalog_manager.cc:5697] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 20 to 21, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 21 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:23.621400 12531 catalog_manager.cc:5697] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 18 to 20, leader changed from 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 20 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:23.703086 12525 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54800 (request call id 325) took 1045 ms. Trace:
I20260709 13:21:23.706682 20995 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:23.707341 20995 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:23.703493 12525 rpcz_store.cc:276] 0709 13:21:22.657471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:22.657681 (+   210us) service_pool.cc:224] Handling call
0709 13:21:23.702995 (+1045314us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:23.559800 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:23.560347 (+   547us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:23.560382 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:21:23.560401 (+    19us) tablet.cc:662] Decoding operations
0709 13:21:23.561206 (+   805us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:23.561264 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:23.561293 (+    29us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:21:23.561991 (+   698us) tablet.cc:701] Row locks acquired
0709 13:21:23.562007 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:21:23.562170 (+   163us) write_op.cc:270] Start()
0709 13:21:23.564252 (+  2082us) write_op.cc:276] Timestamp: P: 1783603283562113 usec, L: 0
0709 13:21:23.564281 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:21:23.568869 (+  4588us) log.cc:844] Serialized 3199 byte log entry
0709 13:21:23.571199 (+  2330us) op_driver.cc:464] REPLICATION: finished
0709 13:21:23.571724 (+   525us) write_op.cc:301] APPLY: starting
0709 13:21:23.571799 (+    75us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:23.576251 (+  4452us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:23.576341 (+    90us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:23.604086 (+ 27745us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:23.604326 (+   240us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=16,mrs_lookups=0
0709 13:21:23.604379 (+    53us) write_op.cc:312] APPLY: finished
0709 13:21:23.604692 (+   313us) log.cc:844] Serialized 81 byte log entry
0709 13:21:23.604877 (+   185us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:23.605015 (+   138us) write_op.cc:454] Released schema lock
0709 13:21:23.605062 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:23.605403 (+   341us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:23.605430 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":31709,"reported_tablets":23,"child_traces":[["op",{"apply.queue_time_us":285,"apply.run_cpu_time_us":11632,"apply.run_wall_time_us":33652,"cfile_cache_hit":32,"cfile_cache_hit_bytes":33472,"num_ops":8,"prepare.queue_time_us":2502,"prepare.run_cpu_time_us":3632,"prepare.run_wall_time_us":11299,"replication_time_us":5751}]]}
I20260709 13:21:23.709944 20995 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:23.717151 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
W20260709 13:21:23.726284 13349 leader_election.cc:336] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:23.726758 13349 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:23.727550 20973 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:23.761620 12523 catalog_manager.cc:5697] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 18 to 19, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 19 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:23.780005 12673 consensus_peers.cc:597] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:23.787987 20844 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:23.788578 20844 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:23.807015 20844 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:23.808656 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:23.809403 13383 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
I20260709 13:21:23.810776 12672 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:23.824440 20844 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:23.824937 20844 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:23.846786 20844 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:23.852488 12673 leader_election.cc:336] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:23.869307 12775 raft_consensus.cc:1275] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:21:23.871321 20931 consensus_queue.cc:1048] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:21:23.969521 12673 consensus_peers.cc:597] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:23.970747 20932 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:23.971309 20932 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:23.973692 20932 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:23.975246 20883 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:23.975759 20883 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:23.977519 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:23.977880 20883 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:23.978206 13383 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 18.
W20260709 13:21:23.978967 13349 consensus_peers.cc:597] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:23.979790 12672 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:23.980545 20932 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:23.980913 20932 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:23.981280 20932 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:23.983084 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:23.991211 20995 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:23.990871 20932 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:23.991809 20995 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:23.993637 20932 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:23.993897 20995 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:23.997368 13349 consensus_peers.cc:597] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:23.997290 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:23.998076 12706 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 29.
I20260709 13:21:23.997320 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca762dedae144be828acb659b8d9e4b" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:23.999022 13382 raft_consensus.cc:3060] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:24.007228 13382 raft_consensus.cc:2468] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
I20260709 13:21:23.924440 20844 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:24.019945 20844 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:24.034312 12672 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:24.035099 20883 raft_consensus.cc:2804] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:21:24.035274 13349 leader_election.cc:336] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:24.035627 20883 raft_consensus.cc:697] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:24.036558 20883 consensus_queue.cc:237] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:24.037868 13383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5febc2df9d3c41359ceccc1bef8a54d8" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:24.038405 13383 raft_consensus.cc:3060] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:24.057371 12747 consensus_peers.cc:597] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:24.057983 12747 consensus_peers.cc:597] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:24.037240 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
W20260709 13:21:24.030560 12673 leader_election.cc:336] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:24.060943 12673 leader_election.cc:336] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:24.061331 12673 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e, 522fd2ba0e9f402691b5ef29af8579cd
I20260709 13:21:24.062053 20973 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 13:21:24.062458 20973 raft_consensus.cc:493] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 29 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:24.062814 20973 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 29 FOLLOWER]: Advancing to term 30
W20260709 13:21:24.068908 12673 leader_election.cc:336] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:24.074361 13383 raft_consensus.cc:2468] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 19.
I20260709 13:21:24.078763 20844 raft_consensus.cc:2749] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 13:21:24.113075 12672 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:24.126909 21001 raft_consensus.cc:2804] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:24.127508 21001 raft_consensus.cc:697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
W20260709 13:21:24.128082 12673 leader_election.cc:336] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:24.128428 21001 consensus_queue.cc:237] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:24.134817 12673 consensus_peers.cc:597] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:24.144021 20973 raft_consensus.cc:515] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:24.138485 12523 catalog_manager.cc:5697] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 18 to 19, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 19 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:24.152644 12673 consensus_peers.cc:597] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:24.159464 20973 leader_election.cc:290] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 30 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:24.174192 13349 leader_election.cc:336] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:24.178058 12701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15d86ce6de4b43719dc715ab6f4de742" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:24.180400 12701 raft_consensus.cc:3060] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 29 FOLLOWER]: Advancing to term 30
W20260709 13:21:24.197110 12673 consensus_peers.cc:597] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:24.199615 13349 consensus_peers.cc:597] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:24.204950 13349 consensus_peers.cc:597] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:24.205492 13349 consensus_peers.cc:597] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:24.216449 12701 raft_consensus.cc:2468] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 30.
I20260709 13:21:24.226598 13349 leader_election.cc:304] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:24.227490 20973 raft_consensus.cc:2804] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 30 FOLLOWER]: Leader election won for term 30
I20260709 13:21:24.227984 20973 raft_consensus.cc:697] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 30 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:24.228782 20973 consensus_queue.cc:237] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:24.230691 12523 catalog_manager.cc:5697] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 18 to 19, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 19 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:24.260130 12747 consensus_peers.cc:597] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:24.260716 12747 consensus_peers.cc:597] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:24.261210 12747 consensus_peers.cc:597] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:24.264143 13349 consensus_peers.cc:597] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:24.261756 12523 catalog_manager.cc:5697] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 29 to 30, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 30 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:21:24.283248 12673 consensus_peers.cc:597] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:24.288431 12747 consensus_peers.cc:597] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:24.310595 13349 consensus_peers.cc:597] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:24.377995 12673 consensus_peers.cc:597] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:24.410997 12747 consensus_peers.cc:597] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:24.413810 12706 consensus_queue.cc:237] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:21:24.424054 20969 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:24.424654 20969 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:24.426374 11517 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 12655)
I20260709 13:21:24.431584 20969 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:24.433396 12779 raft_consensus.cc:1275] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:21:24.435297 21003 consensus_queue.cc:1048] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:21:24.437245 12673 consensus_peers.cc:597] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:24.438244 12672 consensus_peers.cc:597] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 -> Peer 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021): Couldn't send request to peer 522fd2ba0e9f402691b5ef29af8579cd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5067d5ea86fa4774a6be5b882e5b289d. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:24.439608 12747 leader_election.cc:336] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:24.442399 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:24.447799 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:24.448626 20969 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:21:24.460479 13382 raft_consensus.cc:1275] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:21:24.462358 21001 consensus_queue.cc:1048] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:24.463455 20932 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:24.464043 20932 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:24.466353 20932 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:24.476251 12775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:24.476994 12775 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 26.
W20260709 13:21:24.478183 12673 leader_election.cc:336] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:24.480762 12672 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:24.481590 20932 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:21:24.482008 20932 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 19cd864c053744bebd96093a7982537e)
I20260709 13:21:24.482376 20932 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:21:24.517060 20844 raft_consensus.cc:2955] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Committing config change with OpId 15.9: config changed from index -1 to 9, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:21:24.519164 12775 raft_consensus.cc:2955] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Committing config change with OpId 15.9: config changed from index -1 to 9, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
W20260709 13:21:24.522374 12673 consensus_peers.cc:597] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:24.533123 12747 consensus_peers.cc:597] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:24.537534 20932 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:24.563946 12523 catalog_manager.cc:5697] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index -1 to 9, NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) added. New cstate: current_term: 15 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } } }
I20260709 13:21:24.568470 20932 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 27 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:24.579674 12673 leader_election.cc:336] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:24.582647 12779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "febe63321d6c417cbfc5ccbca7caeefa" candidate_uuid: "639ab2db265f4747bb7afb93e833f530" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:24.583312 12779 raft_consensus.cc:3060] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:21:24.590966 12779 raft_consensus.cc:2468] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 639ab2db265f4747bb7afb93e833f530 in term 27.
I20260709 13:21:24.592203 12672 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:24.602854 20932 raft_consensus.cc:2804] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 13:21:24.617877 12673 consensus_peers.cc:597] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:24.618150 20932 raft_consensus.cc:697] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 27 LEADER]: Becoming Leader. State: Replica: 639ab2db265f4747bb7afb93e833f530, State: Running, Role: LEADER
I20260709 13:21:24.619263 20932 consensus_queue.cc:237] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:24.641664 21004 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:24.646162 21004 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:24.648221 21004 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:24.651695 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530" is_pre_election: true
I20260709 13:21:24.652357 12706 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 17.
W20260709 13:21:24.655755 12673 consensus_peers.cc:597] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:24.654958 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 
I20260709 13:21:24.657541 20965 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:24.657958 20965 raft_consensus.cc:493] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:24.658289 20965 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:24.662005 12673 consensus_peers.cc:597] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:24.666438 20965 raft_consensus.cc:515] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:24.668628 20965 leader_election.cc:290] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Requested vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:24.671471 12747 leader_election.cc:336] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:24.684428 12706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e390d5b6aea04f09ab5dbf11b20a7b98" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "639ab2db265f4747bb7afb93e833f530"
I20260709 13:21:24.685101 12706 raft_consensus.cc:3060] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:24.694152 12673 consensus_peers.cc:597] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:24.698086 12706 raft_consensus.cc:2468] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 18.
W20260709 13:21:24.699815 12747 leader_election.cc:336] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:24.702345 12747 leader_election.cc:304] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530; no voters: 19cd864c053744bebd96093a7982537e
I20260709 13:21:24.703184 20969 raft_consensus.cc:2804] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:24.703778 20969 raft_consensus.cc:697] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:24.705061 20969 consensus_queue.cc:237] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
W20260709 13:21:24.711192 13349 consensus_peers.cc:597] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:24.730533 12517 catalog_manager.cc:4755] TS 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 98a43c5d78c945bb8c5541bc892f4865: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:24.729740 13349 consensus_peers.cc:597] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:24.734764 13382 raft_consensus.cc:1275] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:21:24.737465 20932 consensus_queue.cc:1048] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:24.737927 12523 catalog_manager.cc:5697] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 reported cstate change: term changed from 26 to 27, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 639ab2db265f4747bb7afb93e833f530 (127.11.63.66). New cstate: current_term: 27 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:24.756510 12706 raft_consensus.cc:1275] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 30 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 13:21:24.758167 20973 consensus_queue.cc:1048] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
W20260709 13:21:24.774380 12673 consensus_peers.cc:597] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:24.777755 12531 catalog_manager.cc:5697] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 16 to 18, leader changed from 19cd864c053744bebd96093a7982537e (127.11.63.65) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 18 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:24.906203 21019 ts_tablet_manager.cc:933] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd: Initiating tablet copy from peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:24.944566 13349 consensus_peers.cc:597] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:24.960719 13349 consensus_peers.cc:597] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:24.961200 13349 consensus_peers.cc:597] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:24.990881 12673 consensus_peers.cc:597] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:24.999697 21019 tablet_copy_client.cc:323] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.63.66:45173
I20260709 13:21:25.001751 12718 tablet_copy_service.cc:140] P 639ab2db265f4747bb7afb93e833f530: Received BeginTabletCopySession request for tablet 5067d5ea86fa4774a6be5b882e5b289d from peer 522fd2ba0e9f402691b5ef29af8579cd ({username='slave'} at 127.0.0.1:38496)
I20260709 13:21:25.002372 12718 tablet_copy_service.cc:161] P 639ab2db265f4747bb7afb93e833f530: Beginning new tablet copy session on tablet 5067d5ea86fa4774a6be5b882e5b289d from peer 522fd2ba0e9f402691b5ef29af8579cd at {username='slave'} at 127.0.0.1:38496: session id = 522fd2ba0e9f402691b5ef29af8579cd-5067d5ea86fa4774a6be5b882e5b289d
I20260709 13:21:25.015585 12718 tablet_copy_source_session.cc:215] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:21:25.023902 21019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5067d5ea86fa4774a6be5b882e5b289d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:25.072423 12775 raft_consensus.cc:1275] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 27 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 13:21:25.074185 20932 consensus_queue.cc:1048] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:21:25.074831 21019 tablet_copy_client.cc:806] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 0 data blocks...
I20260709 13:21:25.075446 21019 tablet_copy_client.cc:670] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Starting download of 1 WAL segments...
I20260709 13:21:25.083408 21019 tablet_copy_client.cc:538] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:21:25.086030 12673 consensus_peers.cc:597] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:25.100677 21019 tablet_bootstrap.cc:492] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap starting.
W20260709 13:21:25.181202 13349 consensus_peers.cc:597] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:25.203586 12706 raft_consensus.cc:1275] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:21:25.205092 21004 consensus_queue.cc:1048] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:21:25.226037 12747 consensus_peers.cc:597] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.260442 13349 consensus_peers.cc:597] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:25.272130 12747 consensus_peers.cc:597] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:25.326610 21019 tablet_bootstrap.cc:492] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:25.327646 21019 tablet_bootstrap.cc:492] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd: Bootstrap complete.
I20260709 13:21:25.328408 21019 ts_tablet_manager.cc:1403] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd: Time spent bootstrapping tablet: real 0.228s	user 0.040s	sys 0.009s
I20260709 13:21:25.351773 21019 raft_consensus.cc:359] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd [term 15 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:21:25.352680 21019 raft_consensus.cc:740] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd [term 15 LEARNER]: Becoming Follower/Learner. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Initialized, Role: LEARNER
I20260709 13:21:25.353363 21019 consensus_queue.cc:260] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: NON_VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: true } }
I20260709 13:21:25.391249 21019 ts_tablet_manager.cc:1434] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd: Time spent starting tablet: real 0.062s	user 0.009s	sys 0.000s
I20260709 13:21:25.407779 12718 tablet_copy_service.cc:342] P 639ab2db265f4747bb7afb93e833f530: Request end of tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-5067d5ea86fa4774a6be5b882e5b289d received from {username='slave'} at 127.0.0.1:38496
I20260709 13:21:25.408241 12718 tablet_copy_service.cc:434] P 639ab2db265f4747bb7afb93e833f530: ending tablet copy session 522fd2ba0e9f402691b5ef29af8579cd-5067d5ea86fa4774a6be5b882e5b289d on tablet 5067d5ea86fa4774a6be5b882e5b289d with peer 522fd2ba0e9f402691b5ef29af8579cd
W20260709 13:21:25.426817 11517 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 12655)
I20260709 13:21:25.446178 13383 raft_consensus.cc:1217] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd [term 15 LEARNER]: Deduplicated request from leader. Original: 15.8->[15.9-15.9]   Dedup: 15.9->[]
W20260709 13:21:25.477205 12747 consensus_peers.cc:597] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.540725 12747 consensus_peers.cc:597] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.760804 13349 consensus_peers.cc:597] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:25.763840 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 45913.2us
W20260709 13:21:25.803285 13349 consensus_peers.cc:597] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:25.809203 12747 consensus_peers.cc:597] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:25.836071 12517 catalog_manager.cc:4755] TS 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 98a43c5d78c945bb8c5541bc892f4865: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:25.918370 21001 raft_consensus.cc:1064] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530: attempting to promote NON_VOTER 522fd2ba0e9f402691b5ef29af8579cd to VOTER
I20260709 13:21:25.921020 21001 consensus_queue.cc:237] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 7, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:25.961750 13383 raft_consensus.cc:1275] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd [term 15 LEARNER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:21:25.963522 12779 raft_consensus.cc:1275] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:21:25.970374 20844 consensus_queue.cc:1048] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:21:25.980692 20844 consensus_queue.cc:1048] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:21:25.993305 12673 consensus_peers.cc:597] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:26.051241 20932 raft_consensus.cc:2955] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Committing config change with OpId 15.10: config changed from index 9 to 10, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:26.071185 13377 raft_consensus.cc:2955] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index 9 to 10, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:21:26.193739 12747 consensus_peers.cc:597] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:26.205888 12779 raft_consensus.cc:2955] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index 9 to 10, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:26.276463 12523 catalog_manager.cc:5697] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: config changed from index 9 to 10, 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68) changed from NON_VOTER to VOTER. New cstate: current_term: 15 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:21:26.350682 12673 consensus_peers.cc:597] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:26.427335 11517 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 12655)
I20260709 13:21:26.490761 12706 consensus_queue.cc:237] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:26.566759 13383 raft_consensus.cc:1275] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:21:26.568583 20932 consensus_queue.cc:1048] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:26.577590 21032 raft_consensus.cc:2955] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Committing config change with OpId 15.11: config changed from index 10 to 11, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:26.589663 13383 raft_consensus.cc:2955] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
I20260709 13:21:26.622884 12517 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5067d5ea86fa4774a6be5b882e5b289d with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:21:26.632006 12531 catalog_manager.cc:5697] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 reported cstate change: config changed from index 10 to 11, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New cstate: current_term: 15 leader_uuid: "639ab2db265f4747bb7afb93e833f530" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:21:26.646306 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 104892us
W20260709 13:21:26.657511 12673 consensus_peers.cc:597] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:26.663968 13349 consensus_peers.cc:597] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:26.669727 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 80395.6us
I20260709 13:21:26.683214 12779 raft_consensus.cc:1275] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Refusing update from remote peer 639ab2db265f4747bb7afb93e833f530: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:21:26.707712 12673 consensus_peers.cc:597] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:26.708535 12673 consensus_peers.cc:597] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:26.715204 12673 consensus_peers.cc:597] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:26.723953 20932 consensus_queue.cc:1048] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:21:26.725929 12517 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 60489.6us
W20260709 13:21:26.732395 13349 consensus_peers.cc:597] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:26.748144 12779 raft_consensus.cc:2955] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, VOTER 19cd864c053744bebd96093a7982537e (127.11.63.65) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } } }
W20260709 13:21:26.773049 13349 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111) [suppressed 441 similar messages]
W20260709 13:21:26.804659 12747 consensus_peers.cc:597] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:26.819141 13349 consensus_peers.cc:597] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:26.822337 12673 consensus_peers.cc:597] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:26.828526 12747 consensus_peers.cc:597] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:26.829247 12747 consensus_peers.cc:597] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:26.829944 12747 consensus_peers.cc:597] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:26.866840 13349 consensus_peers.cc:597] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:26.889156 12747 consensus_peers.cc:597] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:26.896791 12517 catalog_manager.cc:4755] TS 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 5067d5ea86fa4774a6be5b882e5b289d: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:26.941612 13349 consensus_peers.cc:597] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:26.949024 13349 consensus_peers.cc:597] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:26.949657 13349 consensus_peers.cc:597] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:26.959986 12673 consensus_peers.cc:597] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:26.960682 12673 consensus_peers.cc:597] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:27.022846 12747 consensus_peers.cc:597] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:27.060856 12747 consensus_peers.cc:597] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:27.080464 12747 consensus_peers.cc:597] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:27.169624 12673 consensus_peers.cc:597] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:27.170239 12673 consensus_peers.cc:597] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:27.203038 13349 consensus_peers.cc:597] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:27.304906 12673 consensus_peers.cc:597] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:27.347149 12673 consensus_peers.cc:597] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:27.367695 13349 consensus_peers.cc:597] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:27.423079 12673 consensus_peers.cc:597] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:27.428766 11517 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 12655)
W20260709 13:21:27.536494 13349 consensus_peers.cc:597] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:27.545406 13349 consensus_peers.cc:597] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:27.548856 13349 consensus_peers.cc:597] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:27.603745 13349 consensus_peers.cc:597] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:27.674508 12673 consensus_peers.cc:597] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:27.713325 12747 consensus_peers.cc:597] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:27.901749 12517 catalog_manager.cc:4755] TS 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 98a43c5d78c945bb8c5541bc892f4865: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:27.912456 12673 consensus_peers.cc:597] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:27.937366 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 130462us
W20260709 13:21:28.029183 12747 consensus_peers.cc:597] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:28.039057 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 126628us
W20260709 13:21:28.042281 13349 consensus_peers.cc:597] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:28.223899 12747 consensus_peers.cc:597] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.265702 13349 consensus_peers.cc:597] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:28.359030 12747 consensus_peers.cc:597] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.429298 11517 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 12655)
W20260709 13:21:28.575022 12747 consensus_peers.cc:597] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:28.583230 13349 consensus_peers.cc:597] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.722044 12747 consensus_peers.cc:597] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:28.847170 12673 consensus_peers.cc:597] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:28.909451 12747 consensus_peers.cc:597] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:28.921067 12673 consensus_peers.cc:597] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:28.993394 12673 consensus_peers.cc:597] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:28.995298 12747 consensus_peers.cc:597] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:29.049470 13349 consensus_peers.cc:597] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:29.066494 12747 consensus_peers.cc:597] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:29.118321 12747 consensus_peers.cc:597] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:29.128206 12517 catalog_manager.cc:4755] TS 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 5067d5ea86fa4774a6be5b882e5b289d: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:29.144655 12673 consensus_peers.cc:597] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:29.155100 12673 consensus_peers.cc:597] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:29.174774 13407 debug-util.cc:398] Leaking SignalData structure 0x7b08003d03a0 after lost signal to thread 11520
W20260709 13:21:29.176082 13407 debug-util.cc:398] Leaking SignalData structure 0x7b08001cc980 after lost signal to thread 12566
W20260709 13:21:29.177512 13407 debug-util.cc:398] Leaking SignalData structure 0x7b0800343840 after lost signal to thread 12733
W20260709 13:21:29.187963 13407 debug-util.cc:398] Leaking SignalData structure 0x7b08002e1fa0 after lost signal to thread 12807
W20260709 13:21:29.189616 13407 debug-util.cc:398] Leaking SignalData structure 0x7b0800226f60 after lost signal to thread 13410
W20260709 13:21:29.191640 13349 consensus_peers.cc:597] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:29.210932 13349 consensus_peers.cc:597] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:29.212678 12673 consensus_peers.cc:597] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:29.256418 12673 consensus_peers.cc:597] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:29.258421 12747 consensus_peers.cc:597] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:29.297412 13349 consensus_peers.cc:597] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:29.311558 12747 consensus_peers.cc:597] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:29.403964 13349 consensus_peers.cc:597] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:29.410130 13349 consensus_peers.cc:597] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:29.429807 11517 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 12655)
W20260709 13:21:29.508857 13349 consensus_peers.cc:597] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:29.511265 12673 consensus_peers.cc:597] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:29.561553 12747 consensus_peers.cc:597] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:29.635195 12673 consensus_peers.cc:597] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:29.684247 13349 consensus_peers.cc:597] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:29.693579 12673 consensus_peers.cc:597] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:29.826109 12673 consensus_peers.cc:597] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:29.854763 13349 consensus_peers.cc:597] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:29.941551 12747 consensus_peers.cc:597] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:29.970736 12673 consensus_peers.cc:597] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:30.027760 12673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 44939.6us
W20260709 13:21:30.106541 12673 consensus_peers.cc:597] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:30.120693 12673 consensus_peers.cc:597] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:30.128024 13349 consensus_peers.cc:597] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:30.129520 13349 consensus_peers.cc:597] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:30.152808 13349 consensus_peers.cc:597] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:30.189713 12746 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36021, user_credentials={real_user=slave}} blocked reactor thread for 117462us
W20260709 13:21:30.327302 13349 consensus_peers.cc:597] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:30.389923 12747 consensus_peers.cc:597] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:30.410481 12747 consensus_peers.cc:597] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:30.430519 11517 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 12655)
W20260709 13:21:30.502538 12747 consensus_peers.cc:597] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:30.626426 12673 consensus_peers.cc:597] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:30.671943 13349 consensus_peers.cc:597] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:30.765736 13349 consensus_peers.cc:597] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:30.770047 12747 consensus_peers.cc:597] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:30.815109 12747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:43603, user_credentials={real_user=slave}} blocked reactor thread for 45128.2us
W20260709 13:21:30.866928 12747 consensus_peers.cc:597] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:31.161725 12747 consensus_peers.cc:597] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:31.227223 12517 catalog_manager.cc:4755] TS 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 5067d5ea86fa4774a6be5b882e5b289d: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:31.344545 12747 consensus_peers.cc:597] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:31.401767 12747 consensus_peers.cc:597] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:31.431038 11517 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 12655)
W20260709 13:21:31.454025 13349 consensus_peers.cc:597] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:31.498461 13349 consensus_peers.cc:597] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:31.548909 13349 consensus_peers.cc:597] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:31.693697 12747 consensus_peers.cc:597] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:31.694212 12747 consensus_peers.cc:597] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:31.771473 13349 consensus_peers.cc:597] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:31.781463 12747 consensus_peers.cc:597] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:31.785090 12673 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111) [suppressed 428 similar messages]
W20260709 13:21:31.879521 12673 consensus_peers.cc:597] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:31.919499 13349 consensus_peers.cc:597] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:31.932802 13349 consensus_peers.cc:597] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:31.965541 12673 consensus_peers.cc:597] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:31.975623 12673 consensus_peers.cc:597] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:32.032004 13349 consensus_peers.cc:597] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:32.036415 12673 consensus_peers.cc:597] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:32.050678 12747 consensus_peers.cc:597] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:32.086161 13348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41181, user_credentials={real_user=slave}} blocked reactor thread for 64803.9us
W20260709 13:21:32.115136 13349 consensus_peers.cc:597] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:32.147038 13349 consensus_peers.cc:597] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:32.171888 12747 consensus_peers.cc:597] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:32.175819 12673 consensus_peers.cc:597] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:32.215602 12747 consensus_peers.cc:597] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:21:32.281922 12521 maintenance_manager.cc:419] P 480284ff372740a09dfd1f972d55d7eb: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033384
W20260709 13:21:32.338199 12673 consensus_peers.cc:597] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:32.427073 12520 maintenance_manager.cc:643] P 480284ff372740a09dfd1f972d55d7eb: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.144s	user 0.055s	sys 0.005s Metrics: {"bytes_written":9548,"delete_count":0,"lbm_write_time_us":2821,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":6,"reinsert_count":0,"thread_start_us":9990,"threads_started":1,"update_count":370,"wal-append.queue_time_us":10484}
I20260709 13:21:32.430092 12521 maintenance_manager.cc:419] P 480284ff372740a09dfd1f972d55d7eb: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.475559
W20260709 13:21:32.431589 11517 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 12655)
W20260709 13:21:32.484126 13349 consensus_peers.cc:597] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:32.599107 13349 consensus_peers.cc:597] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:32.631742 13349 consensus_peers.cc:597] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:32.645169 13349 consensus_peers.cc:597] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:32.687704 12673 consensus_peers.cc:597] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:32.704183 12673 consensus_peers.cc:597] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:32.740625 12747 consensus_peers.cc:597] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:32.890533 12673 consensus_peers.cc:597] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:32.911693 12520 maintenance_manager.cc:643] P 480284ff372740a09dfd1f972d55d7eb: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.481s	user 0.131s	sys 0.030s Metrics: {"cfile_cache_miss":10,"cfile_cache_miss_bytes":31180,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":8234,"dirs.run_cpu_time_us":578,"dirs.run_wall_time_us":1645,"lbm_read_time_us":5910,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":25,"lbm_write_time_us":24609,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":12,"peak_mem_usage":607533,"reinsert_count":0,"thread_start_us":14887,"threads_started":2,"update_count":402}
I20260709 13:21:32.915408 12521 maintenance_manager.cc:419] P 480284ff372740a09dfd1f972d55d7eb: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260709 13:21:32.916640 12520 maintenance_manager.cc:643] P 480284ff372740a09dfd1f972d55d7eb: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {}
W20260709 13:21:32.942288 12673 consensus_peers.cc:597] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:32.968711 12673 consensus_peers.cc:597] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:32.993819 13349 consensus_peers.cc:597] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:32.997087 12747 consensus_peers.cc:597] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:33.069774 12673 consensus_peers.cc:597] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:33.074033 13349 consensus_peers.cc:597] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:33.098096 12673 consensus_peers.cc:597] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:33.143168 13349 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:45173, user_credentials={real_user=slave}} blocked reactor thread for 51290.2us
W20260709 13:21:33.172562 12747 consensus_peers.cc:597] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:33.249310 13349 consensus_peers.cc:597] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:33.274320 12747 consensus_peers.cc:597] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.414723 12673 consensus_peers.cc:597] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:33.482546 12673 consensus_peers.cc:597] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:33.502005 12747 consensus_peers.cc:597] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:33.504480 12747 consensus_peers.cc:597] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:33.648722 12747 consensus_peers.cc:597] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.706759 13349 consensus_peers.cc:597] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:21:33.858263 11517 tablet_server.cc:196] TabletServer@127.11.63.65:0 shutdown complete.
W20260709 13:21:33.895431 12747 consensus_peers.cc:597] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:34.059828 11517 tablet_server.cc:179] TabletServer@127.11.63.66:0 shutting down...
W20260709 13:21:34.115418 13349 consensus_peers.cc:597] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:34.119382 13349 consensus_peers.cc:597] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:34.127460 12747 consensus_peers.cc:597] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:34.128280 12747 consensus_peers.cc:597] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:34.154774 12747 consensus_peers.cc:597] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:34.162736 12747 consensus_peers.cc:597] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:34.165294 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:34.168525 11517 tablet_replica.cc:333] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.169430 11517 raft_consensus.cc:2243] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:21:34.171712 11517 raft_consensus.cc:2272] T 37e99a6b09d744a9b056b83be1033820 P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.174453 11517 tablet_replica.cc:333] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.175030 11517 raft_consensus.cc:2243] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.175549 11517 raft_consensus.cc:2272] T 8e0d0566acd7465f87e20053c75e28fa P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.178035 11517 tablet_replica.cc:333] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.178798 11517 raft_consensus.cc:2243] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.179544 11517 raft_consensus.cc:2272] T 522bcf0a155f4da196cb33f1aae90a82 P 639ab2db265f4747bb7afb93e833f530 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.184070 11517 tablet_replica.cc:333] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.184718 11517 raft_consensus.cc:2243] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:21:34.185775 11517 raft_consensus.cc:2272] T eb7fca96394f4ba9b4fe3d7a6e278878 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.188298 11517 tablet_replica.cc:333] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.189347 11517 raft_consensus.cc:2243] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.189913 11517 raft_consensus.cc:2272] T 18b2f67fae7f4963a4b4c562003df2fe P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.192178 11517 tablet_replica.cc:333] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.192919 11517 raft_consensus.cc:2243] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:21:34.194072 11517 raft_consensus.cc:2272] T c4e4ae9d38ba4fd0a700998e4f4750be P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.196754 11517 tablet_replica.cc:333] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
W20260709 13:21:34.197433 13349 consensus_peers.cc:597] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:34.197603 11517 raft_consensus.cc:2243] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.198378 11517 raft_consensus.cc:2272] T 6146f826f1904f6f8b10ad518d454c22 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.200519 11517 tablet_replica.cc:333] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.201087 11517 raft_consensus.cc:2243] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.201650 11517 raft_consensus.cc:2272] T ea604dddf4c244a181921c4dbdd0e720 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.204007 11517 tablet_replica.cc:333] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.205931 11517 raft_consensus.cc:2243] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.206600 11517 raft_consensus.cc:2272] T 49dad260ffe242d2b4e0976f119a19d7 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.210369 11517 tablet_replica.cc:333] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.211164 11517 raft_consensus.cc:2243] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:21:34.212024 11517 raft_consensus.cc:2272] T 98a43c5d78c945bb8c5541bc892f4865 P 639ab2db265f4747bb7afb93e833f530 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.214030 11517 tablet_replica.cc:333] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.214573 11517 raft_consensus.cc:2243] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:21:34.215741 11517 raft_consensus.cc:2272] T 8f91059867c444d89718e83475c089cd P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.218461 11517 tablet_replica.cc:333] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.219287 11517 raft_consensus.cc:2243] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:21:34.220568 11517 raft_consensus.cc:2272] T 1fa805ee07a7429abe6b520bbfb94cd9 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.224195 11517 tablet_replica.cc:333] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.225049 11517 raft_consensus.cc:2243] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.225728 11517 raft_consensus.cc:2272] T 1b13b87a5cea4263937041d123c5889d P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.228302 11517 tablet_replica.cc:333] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
W20260709 13:21:34.228516 13349 consensus_peers.cc:597] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:21:34.229135 11517 raft_consensus.cc:2243] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:21:34.230927 11517 raft_consensus.cc:2272] T bd1bc5f9b25e46cc866068bd346e11c6 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:34.233326 12747 consensus_peers.cc:597] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:34.233740 11517 tablet_replica.cc:333] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.234615 11517 raft_consensus.cc:2243] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.235267 11517 raft_consensus.cc:2272] T fb247fe4862541ccadc22bb79417d66c P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.237860 11517 tablet_replica.cc:333] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.238672 11517 raft_consensus.cc:2243] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:21:34.238966 12747 consensus_peers.cc:597] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:34.240511 13349 consensus_peers.cc:597] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:34.239316 11517 raft_consensus.cc:2272] T 612a4639a7a749be8856bbc60766e312 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:34.240098 12747 consensus_peers.cc:597] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:34.246658 11517 tablet_replica.cc:333] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.247491 11517 raft_consensus.cc:2243] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.248175 11517 raft_consensus.cc:2272] T 3b98bb10e3e04334b7fcf7f781593682 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:34.251554 13349 consensus_peers.cc:597] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:34.252084 11517 tablet_replica.cc:333] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.252717 11517 raft_consensus.cc:2243] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.253340 11517 raft_consensus.cc:2272] T af96e52b004d4c8c942cc0ec33c95c99 P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.255894 11517 tablet_replica.cc:333] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.256630 11517 raft_consensus.cc:2243] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:21:34.257807 11517 raft_consensus.cc:2272] T ccf3f471aa044537a9492a86bb48b0aa P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:34.260402 12747 consensus_peers.cc:597] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:34.261044 11517 tablet_replica.cc:333] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.261698 11517 raft_consensus.cc:2243] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:21:34.263020 11517 raft_consensus.cc:2272] T ad2866c2079c4c5b9df5ad6a072e192d P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.265790 11517 tablet_replica.cc:333] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.266604 11517 raft_consensus.cc:2243] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.267206 11517 raft_consensus.cc:2272] T e390d5b6aea04f09ab5dbf11b20a7b98 P 639ab2db265f4747bb7afb93e833f530 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:34.269217 12747 consensus_peers.cc:597] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:34.270150 11517 tablet_replica.cc:333] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.270735 11517 raft_consensus.cc:2243] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.271282 11517 raft_consensus.cc:2272] T ab5ddded6fd34f278dade99f2086c4e6 P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.273890 11517 tablet_replica.cc:333] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.274725 11517 raft_consensus.cc:2243] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.275429 11517 raft_consensus.cc:2272] T fa34731f37084c608660f0088a2dba09 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.278656 11517 tablet_replica.cc:333] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.279498 11517 raft_consensus.cc:2243] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:21:34.280668 11517 raft_consensus.cc:2272] T 5067d5ea86fa4774a6be5b882e5b289d P 639ab2db265f4747bb7afb93e833f530 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.283464 11517 tablet_replica.cc:333] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.284591 11517 raft_consensus.cc:2243] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.285280 11517 raft_consensus.cc:2272] T 199931c4ffc44fee980d8bb49cf37f00 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.295993 11517 tablet_replica.cc:333] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.297689 11517 raft_consensus.cc:2243] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:21:34.299985 11517 raft_consensus.cc:2272] T 5febc2df9d3c41359ceccc1bef8a54d8 P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:34.300737 12747 consensus_peers.cc:597] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:34.303594 12747 consensus_peers.cc:597] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:34.301802 13349 consensus_peers.cc:597] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:34.306113 11517 tablet_replica.cc:333] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.308558 11517 raft_consensus.cc:2243] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:21:34.310585 11517 raft_consensus.cc:2272] T 6609d5f93ab048558ea46e511ef3b2ec P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.312587 11517 tablet_replica.cc:333] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.313150 11517 raft_consensus.cc:2243] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:21:34.314139 11517 raft_consensus.cc:2272] T febe63321d6c417cbfc5ccbca7caeefa P 639ab2db265f4747bb7afb93e833f530 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.316248 11517 tablet_replica.cc:333] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.316815 11517 raft_consensus.cc:2243] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:21:34.317729 11517 raft_consensus.cc:2272] T f2f0a3339d324fd4902b6ff970ff0272 P 639ab2db265f4747bb7afb93e833f530 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.320459 11517 tablet_replica.cc:333] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.321235 11517 raft_consensus.cc:2243] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:21:34.323271 11517 raft_consensus.cc:2272] T 58239dd301f64758846dd758e449e4d8 P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.325860 11517 tablet_replica.cc:333] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.327091 11517 raft_consensus.cc:2243] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Raft consensus shutting down.
W20260709 13:21:34.327669 13349 consensus_peers.cc:597] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:34.329849 12747 consensus_peers.cc:597] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:34.347890 12747 consensus_peers.cc:597] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:34.330684 11517 raft_consensus.cc:2272] T 3ca762dedae144be828acb659b8d9e4b P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.609097 11517 tablet_replica.cc:333] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.610719 11517 raft_consensus.cc:2243] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.611693 11517 raft_consensus.cc:2272] T 15d86ce6de4b43719dc715ab6f4de742 P 639ab2db265f4747bb7afb93e833f530 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.614492 11517 tablet_replica.cc:333] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.615289 11517 raft_consensus.cc:2243] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.616032 11517 raft_consensus.cc:2272] T 3938a96da1794e32b484ac729226a8ce P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.619249 11517 tablet_replica.cc:333] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.620034 11517 raft_consensus.cc:2243] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.620637 11517 raft_consensus.cc:2272] T 4c13fa2aff324f65ad0bf6191ea68fab P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.626325 11517 tablet_replica.cc:333] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.627107 11517 raft_consensus.cc:2243] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.627803 11517 raft_consensus.cc:2272] T 866523bf2d2f442f971d504267ed200f P 639ab2db265f4747bb7afb93e833f530 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.630784 11517 tablet_replica.cc:333] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.631652 11517 raft_consensus.cc:2243] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.632339 11517 raft_consensus.cc:2272] T bc7827272b424bad88349b6b7437844b P 639ab2db265f4747bb7afb93e833f530 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.636135 11517 tablet_replica.cc:333] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.637105 11517 raft_consensus.cc:2243] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.637858 11517 raft_consensus.cc:2272] T c6cd31a55742463db9f0f0060bb60a77 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.641198 11517 tablet_replica.cc:333] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.642110 11517 raft_consensus.cc:2243] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.642819 11517 raft_consensus.cc:2272] T 18205d974a7c4592aec0153ef3e44c97 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.645587 11517 tablet_replica.cc:333] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.646917 11517 raft_consensus.cc:2243] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:21:34.648138 11517 raft_consensus.cc:2272] T 9119721f3ee04810b1f277c06c966a45 P 639ab2db265f4747bb7afb93e833f530 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.652228 11517 tablet_replica.cc:333] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.652999 11517 raft_consensus.cc:2243] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.653643 11517 raft_consensus.cc:2272] T 059429c0c4024dfb941d1ff325fe8f98 P 639ab2db265f4747bb7afb93e833f530 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.665668 11517 tablet_replica.cc:333] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.666428 11517 raft_consensus.cc:2243] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.667032 11517 raft_consensus.cc:2272] T 0db63389cf0d45de92e4b01621db07cd P 639ab2db265f4747bb7afb93e833f530 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.695410 11517 tablet_replica.cc:333] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.696200 11517 raft_consensus.cc:2243] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:34.696842 11517 raft_consensus.cc:2272] T a4d7de736149496b86a37ba2ca92670d P 639ab2db265f4747bb7afb93e833f530 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:34.731416 11517 tablet_replica.cc:333] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
I20260709 13:21:34.732170 11517 raft_consensus.cc:2243] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 19 LEADER]: Raft consensus shutting down.
W20260709 13:21:34.736145 13349 consensus_peers.cc:597] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:34.742853 13349 consensus_peers.cc:597] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:21:34.733383 11517 raft_consensus.cc:2272] T 414b06f3e5f8472ea4071ebdd0161b9d P 639ab2db265f4747bb7afb93e833f530 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:34.753593 13349 consensus_peers.cc:597] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:34.754293 13349 consensus_peers.cc:597] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:34.755800 13349 consensus_peers.cc:597] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:21:34.753238 11517 tablet_replica.cc:333] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530: stopping tablet replica
W20260709 13:21:34.755879 12747 consensus_peers.cc:597] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:21:34.770352 11517 raft_consensus.cc:2243] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Raft consensus shutting down.
W20260709 13:21:34.793535 13349 consensus_peers.cc:597] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:21:34.798635 11517 raft_consensus.cc:2272] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 639ab2db265f4747bb7afb93e833f530 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:34.810323 12747 consensus_peers.cc:597] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:34.813237 12747 consensus_peers.cc:597] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:34.815418 12747 consensus_peers.cc:597] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:34.815965 12747 consensus_peers.cc:597] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:34.816442 12747 consensus_peers.cc:597] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:34.817283 12747 consensus_peers.cc:597] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:34.825212 12747 consensus_peers.cc:597] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:34.825868 12747 consensus_peers.cc:597] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:34.843283 12747 consensus_peers.cc:597] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:34.843941 12747 consensus_peers.cc:597] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:34.847474 12747 consensus_peers.cc:597] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:34.933525 12747 consensus_peers.cc:597] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:35.126852 13349 consensus_peers.cc:597] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:21:35.263507 21086 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:35.264076 21086 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:35.266422 21086 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:35.271695 13349 leader_election.cc:336] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:35.277868 13349 consensus_peers.cc:597] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:35.280495 13349 consensus_peers.cc:597] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:35.281386 13349 leader_election.cc:336] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:35.281797 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:35.283052 21079 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:21:35.319919 12747 consensus_peers.cc:597] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:35.333489 12517 catalog_manager.cc:4755] TS 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 5067d5ea86fa4774a6be5b882e5b289d: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:35.336597 21086 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:35.337205 21086 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:35.339120 21086 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:35.344028 13349 leader_election.cc:336] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:35.344879 13349 leader_election.cc:336] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:35.345177 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:35.345832 21086 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:35.352229 21086 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:35.352795 21086 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:35.354038 21081 raft_consensus.cc:493] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:35.354686 21081 raft_consensus.cc:515] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:35.355469 21086 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:35.356623 21081 leader_election.cc:290] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:35.362812 13349 leader_election.cc:336] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:35.365207 12747 leader_election.cc:336] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
W20260709 13:21:35.366518 12747 leader_election.cc:336] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:35.367046 12747 leader_election.cc:304] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:35.370981 21082 raft_consensus.cc:2749] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:21:35.372530 13349 leader_election.cc:336] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:35.373050 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:35.374166 21086 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:21:35.380765 13349 consensus_peers.cc:597] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:21:35.390930 21086 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:35.391520 21086 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:35.393457 21086 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:35.396745 21086 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:35.397231 21086 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:35.399247 21086 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:35.400728 13349 leader_election.cc:336] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:35.403462 21082 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:35.404012 21082 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:35.406376 21082 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:35.406498 13349 leader_election.cc:336] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:35.406932 13349 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:35.407624 21086 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:21:35.408893 13349 leader_election.cc:336] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:35.408820 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
W20260709 13:21:35.412081 13349 leader_election.cc:336] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:35.412490 13349 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:35.412675 12747 leader_election.cc:336] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:35.413198 21086 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:35.413297 12747 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:35.414278 21082 raft_consensus.cc:2749] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:35.427165 21082 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:35.427645 21082 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:35.430604 21082 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:35.431308 21086 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:35.431876 21086 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:35.434374 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:35.434623 21086 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:35.435004 12747 leader_election.cc:336] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:35.436031 12746 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 522fd2ba0e9f402691b5ef29af8579cd, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:35.436324 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:35.437593 21082 raft_consensus.cc:2749] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:21:35.441432 13349 leader_election.cc:336] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:35.441891 13349 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:35.442472 21086 raft_consensus.cc:2749] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:35.463820 21082 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:35.464619 21082 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:35.467015 21082 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:35.471148 12747 leader_election.cc:336] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:35.471900 12747 leader_election.cc:336] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:35.472216 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:35.472813 21082 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:35.526710 21086 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:35.528151 21086 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:35.530030 21082 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:35.530635 21082 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:35.531656 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:35.532281 12783 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 16.
I20260709 13:21:35.531438 21086 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:35.533493 13348 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
I20260709 13:21:35.533735 21082 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
I20260709 13:21:35.534933 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd" is_pre_election: true
I20260709 13:21:35.534502 21086 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:35.535619 13382 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 15.
I20260709 13:21:35.535975 21086 raft_consensus.cc:493] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:35.536674 12746 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 
W20260709 13:21:35.535032 13349 leader_election.cc:336] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:35.536739 21086 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:35.537626 21082 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:35.538197 21082 raft_consensus.cc:493] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:35.538802 21082 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:35.539850 12747 leader_election.cc:336] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:35.544703 21086 raft_consensus.cc:515] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:35.547183 21086 leader_election.cc:290] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:35.546962 21082 raft_consensus.cc:515] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:35.548432 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98a43c5d78c945bb8c5541bc892f4865" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:35.549085 12783 raft_consensus.cc:3060] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:35.550410 21082 leader_election.cc:290] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68:36021)
W20260709 13:21:35.551746 13349 leader_election.cc:336] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:35.553577 13382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5067d5ea86fa4774a6be5b882e5b289d" candidate_uuid: "3d539f8d95d343d8b1696803f18b30d2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "522fd2ba0e9f402691b5ef29af8579cd"
I20260709 13:21:35.554215 13382 raft_consensus.cc:3060] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:35.556491 12783 raft_consensus.cc:2468] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 17.
I20260709 13:21:35.558209 13348 leader_election.cc:304] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:35.559378 21086 raft_consensus.cc:2804] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:35.560034 21086 raft_consensus.cc:697] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
W20260709 13:21:35.559674 12747 leader_election.cc:336] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:35.561290 21086 consensus_queue.cc:237] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:35.562219 13382 raft_consensus.cc:2468] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d539f8d95d343d8b1696803f18b30d2 in term 16.
I20260709 13:21:35.563426 12746 leader_election.cc:304] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:35.564258 21082 raft_consensus.cc:2804] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:21:35.565269 21082 raft_consensus.cc:697] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Becoming Leader. State: Replica: 3d539f8d95d343d8b1696803f18b30d2, State: Running, Role: LEADER
I20260709 13:21:35.568055 21082 consensus_queue.cc:237] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:35.570639 21086 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:35.571259 21086 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:35.577155 12525 catalog_manager.cc:5697] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 16 to 17, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 17 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:21:35.582442 13349 leader_election.cc:336] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:35.585615 21086 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:35.593030 21085 raft_consensus.cc:493] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:35.593328 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2" is_pre_election: true
I20260709 13:21:35.593699 21085 raft_consensus.cc:515] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:35.594240 12783 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 20.
I20260709 13:21:35.595494 13348 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:35.596047 21085 leader_election.cc:290] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:35.596874 21086 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:35.597458 21086 raft_consensus.cc:493] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting leader election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:35.600251 21086 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:35.609160 21086 raft_consensus.cc:515] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:35.611438 12730 debug-util.cc:398] Leaking SignalData structure 0x7b08002b8780 after lost signal to thread 11520
W20260709 13:21:35.612680 12730 debug-util.cc:398] Leaking SignalData structure 0x7b0800374a80 after lost signal to thread 12566
I20260709 13:21:35.612954 21086 leader_election.cc:290] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 election: Requested vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:35.613302 12730 debug-util.cc:398] Leaking SignalData structure 0x7b08001f05a0 after lost signal to thread 12733
W20260709 13:21:35.614642 12730 debug-util.cc:398] Leaking SignalData structure 0x7b08000e3500 after lost signal to thread 12807
W20260709 13:21:35.616010 12730 debug-util.cc:398] Leaking SignalData structure 0x7b08003cc0e0 after lost signal to thread 13410
W20260709 13:21:35.617089 13349 leader_election.cc:336] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
W20260709 13:21:35.618619 12747 leader_election.cc:336] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:35.619401 12747 consensus_peers.cc:597] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:35.635929 12747 leader_election.cc:336] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:35.636408 12747 leader_election.cc:304] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:35.637136 21081 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:35.637790 21081 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:35.640077 21081 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:35.640664 21081 raft_consensus.cc:2749] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:35.644547 12783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58239dd301f64758846dd758e449e4d8" candidate_uuid: "522fd2ba0e9f402691b5ef29af8579cd" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3d539f8d95d343d8b1696803f18b30d2"
I20260709 13:21:35.645160 12783 raft_consensus.cc:3060] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:35.647905 21086 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:35.648425 21086 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:35.650563 21086 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:35.642884 12531 catalog_manager.cc:5697] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 reported cstate change: term changed from 15 to 16, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67). New cstate: current_term: 16 leader_uuid: "3d539f8d95d343d8b1696803f18b30d2" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:35.656435 12783 raft_consensus.cc:2468] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522fd2ba0e9f402691b5ef29af8579cd in term 21.
I20260709 13:21:35.660651 21079 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
W20260709 13:21:35.661079 13349 leader_election.cc:336] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:35.661280 12747 leader_election.cc:336] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:35.661335 21079 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:35.662604 13348 leader_election.cc:304] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2, 522fd2ba0e9f402691b5ef29af8579cd; no voters: 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:35.663353 13349 leader_election.cc:336] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:35.663743 21079 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:35.665076 21079 raft_consensus.cc:2804] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:21:35.665202 12747 leader_election.cc:336] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:35.665712 21079 raft_consensus.cc:697] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 LEADER]: Becoming Leader. State: Replica: 522fd2ba0e9f402691b5ef29af8579cd, State: Running, Role: LEADER
I20260709 13:21:35.665728 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:35.670648 21082 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:35.669059 21079 consensus_queue.cc:237] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:35.663797 13349 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:35.674489 21086 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:21:35.685249 13349 leader_election.cc:336] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:35.693230 13349 leader_election.cc:336] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:35.693876 13349 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:35.694872 21086 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:35.701364 12531 catalog_manager.cc:5697] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd reported cstate change: term changed from 20 to 21, leader changed from 639ab2db265f4747bb7afb93e833f530 (127.11.63.66) to 522fd2ba0e9f402691b5ef29af8579cd (127.11.63.68). New cstate: current_term: 21 leader_uuid: "522fd2ba0e9f402691b5ef29af8579cd" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:35.723666 21086 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:35.724237 21086 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:35.726398 21086 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:35.734715 13349 consensus_peers.cc:597] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:35.746667 13349 leader_election.cc:336] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:35.748106 13349 leader_election.cc:336] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:35.748521 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:35.749297 21086 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:35.766489 21085 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:35.767122 21085 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:35.770210 21085 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:35.772584 12747 consensus_peers.cc:597] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:35.775470 12747 leader_election.cc:336] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:35.776638 12747 leader_election.cc:336] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:35.777060 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:35.777370 21085 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:35.777848 21081 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:21:35.778155 21085 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:35.780581 21085 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:35.787109 12747 leader_election.cc:336] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:35.788051 12747 leader_election.cc:336] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:35.788463 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:35.789255 21081 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:21:35.869874 13349 consensus_peers.cc:597] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:35.914875 12747 consensus_peers.cc:597] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:35.941695 13349 consensus_peers.cc:597] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:35.964474 11517 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 12730)
W20260709 13:21:36.072567 13349 consensus_peers.cc:597] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:36.134399 12779 raft_consensus.cc:1275] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:21:36.135988 21079 consensus_queue.cc:1048] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:21:36.143666 12747 consensus_peers.cc:597] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:21:36.176532 13384 raft_consensus.cc:1275] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Refusing update from remote peer 3d539f8d95d343d8b1696803f18b30d2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:21:36.181002 21085 consensus_queue.cc:1048] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:36.207432 12779 raft_consensus.cc:1275] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Refusing update from remote peer 522fd2ba0e9f402691b5ef29af8579cd: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:21:36.242022 21079 consensus_queue.cc:1048] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:21:36.248122 12747 consensus_peers.cc:597] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:36.261287 12747 consensus_peers.cc:597] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:36.269831 13349 consensus_peers.cc:597] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:36.272197 12747 consensus_peers.cc:597] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:21:36.276834 21082 raft_consensus.cc:493] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:36.277321 21082 raft_consensus.cc:515] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:36.279349 21082 leader_election.cc:290] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:36.286213 12747 consensus_peers.cc:597] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:36.292744 12747 leader_election.cc:336] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:36.293429 12747 leader_election.cc:336] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
W20260709 13:21:36.293903 13349 consensus_peers.cc:597] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:36.293871 12747 leader_election.cc:304] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:36.296103 21082 raft_consensus.cc:2749] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:21:36.496533 12747 consensus_peers.cc:597] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:36.628661 12747 consensus_peers.cc:597] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:36.653272 13349 consensus_peers.cc:597] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:36.667326 13349 consensus_peers.cc:597] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:36.692490 12747 consensus_peers.cc:597] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:36.747901 12747 consensus_peers.cc:597] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:36.753505 13349 consensus_peers.cc:597] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:36.784979 12747 consensus_peers.cc:597] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:36.802600 12747 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111) [suppressed 558 similar messages]
W20260709 13:21:36.811188 12747 consensus_peers.cc:597] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:36.824838 12747 consensus_peers.cc:597] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:36.825740 12747 consensus_peers.cc:597] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:36.827129 13349 consensus_peers.cc:597] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:36.837172 13349 consensus_peers.cc:597] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:36.837730 13349 consensus_peers.cc:597] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:36.838063 13349 consensus_peers.cc:597] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:36.848306 12747 consensus_peers.cc:597] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:36.855126 13349 consensus_peers.cc:597] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:36.866760 12747 consensus_peers.cc:597] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:36.912355 12747 consensus_peers.cc:597] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:36.956557 21086 raft_consensus.cc:493] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:36.957165 21086 raft_consensus.cc:515] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:36.959362 21086 leader_election.cc:290] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:36.961597 13349 consensus_peers.cc:597] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:36.972283 13349 leader_election.cc:336] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
W20260709 13:21:36.973073 13349 leader_election.cc:336] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:36.973501 13349 leader_election.cc:304] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:36.974483 21086 raft_consensus.cc:2749] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:21:36.981312 12747 consensus_peers.cc:597] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:36.986466 12747 consensus_peers.cc:597] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:36.988574 21086 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:36.989153 21086 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:36.991196 21086 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:37.000980 13349 leader_election.cc:336] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:37.001991 13349 leader_election.cc:336] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:37.002461 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:37.003479 21086 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:21:37.012343 12747 consensus_peers.cc:597] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:37.013911 12747 consensus_peers.cc:597] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:37.038872 12747 consensus_peers.cc:597] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:37.059177 13349 consensus_peers.cc:597] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:37.063759 13349 consensus_peers.cc:597] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:37.117439 12747 consensus_peers.cc:597] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:37.135339 21086 raft_consensus.cc:493] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:37.136036 21086 raft_consensus.cc:515] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:37.138432 21086 leader_election.cc:290] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:37.141525 21088 raft_consensus.cc:493] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:37.142167 21088 raft_consensus.cc:515] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:37.144515 21088 leader_election.cc:290] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:37.152031 13349 leader_election.cc:336] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:37.160283 13349 leader_election.cc:336] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:37.160741 13349 leader_election.cc:304] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:37.161522 21088 raft_consensus.cc:2749] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:21:37.164561 13349 leader_election.cc:336] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
W20260709 13:21:37.166759 13349 leader_election.cc:336] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:37.167172 13349 leader_election.cc:304] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:37.168006 21088 raft_consensus.cc:2749] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:21:37.168143 13349 consensus_peers.cc:597] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:37.174736 12747 consensus_peers.cc:597] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:37.175345 13349 consensus_peers.cc:597] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:37.188297 13349 consensus_peers.cc:597] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:37.201892 12747 consensus_peers.cc:597] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:37.216166 12747 consensus_peers.cc:597] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:37.230753 12747 consensus_peers.cc:597] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:37.270929 12747 consensus_peers.cc:597] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:21:37.303702 21082 raft_consensus.cc:493] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:37.304260 21082 raft_consensus.cc:515] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:37.306676 21082 leader_election.cc:290] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:37.313270 12747 leader_election.cc:336] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
W20260709 13:21:37.315042 12747 leader_election.cc:336] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:37.315529 12747 leader_election.cc:304] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:37.316325 21082 raft_consensus.cc:2749] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:21:37.341528 11517 tablet_server.cc:196] TabletServer@127.11.63.66:0 shutdown complete.
I20260709 13:21:37.375816 21082 raft_consensus.cc:493] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:37.376518 21082 raft_consensus.cc:515] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:37.379791 21082 leader_election.cc:290] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:37.387070 21088 raft_consensus.cc:493] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:37.387723 21088 raft_consensus.cc:515] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:37.388876 12747 leader_election.cc:336] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:37.390074 21088 leader_election.cc:290] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:37.393458 12747 leader_election.cc:336] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:37.393992 12747 leader_election.cc:304] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:37.396739 21082 raft_consensus.cc:2749] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:21:37.396885 13349 leader_election.cc:336] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:37.397095 12747 consensus_peers.cc:597] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:37.397960 13349 leader_election.cc:336] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:37.398416 13349 leader_election.cc:304] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:37.398903 21082 raft_consensus.cc:493] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:37.399199 21088 raft_consensus.cc:2749] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:37.399624 21082 raft_consensus.cc:515] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:37.401923 21082 leader_election.cc:290] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:37.416857 12747 leader_election.cc:336] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:37.419230 12747 leader_election.cc:336] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:37.419739 12747 leader_election.cc:304] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:37.420518 21082 raft_consensus.cc:2749] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:37.425706 21086 raft_consensus.cc:493] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:37.426568 21086 raft_consensus.cc:515] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:37.429528 21086 leader_election.cc:290] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:37.439301 12747 consensus_peers.cc:597] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:37.439579 13349 leader_election.cc:336] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
W20260709 13:21:37.440275 13349 leader_election.cc:336] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:37.440603 13349 leader_election.cc:304] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:37.441238 21086 raft_consensus.cc:2749] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:21:37.463335 12747 consensus_peers.cc:597] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:37.466503 11517 tablet_server.cc:179] TabletServer@127.11.63.67:0 shutting down...
W20260709 13:21:37.471603 13348 consensus_peers.cc:597] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:37.503340 21110 raft_consensus.cc:493] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:37.503930 21110 raft_consensus.cc:515] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } }
I20260709 13:21:37.506386 21110 leader_election.cc:290] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:37.509358 12747 leader_election.cc:336] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
W20260709 13:21:37.511508 12747 leader_election.cc:336] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:37.512001 12747 leader_election.cc:304] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d539f8d95d343d8b1696803f18b30d2; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:37.512871 21110 raft_consensus.cc:2749] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:37.526168 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:37.527166 11517 tablet_replica.cc:333] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.527915 11517 raft_consensus.cc:2243] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.528563 11517 raft_consensus.cc:2272] T 37e99a6b09d744a9b056b83be1033820 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.531064 11517 tablet_replica.cc:333] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
W20260709 13:21:37.531440 12747 consensus_peers.cc:597] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:37.531798 11517 raft_consensus.cc:2243] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.532749 11517 raft_consensus.cc:2272] T 8e0d0566acd7465f87e20053c75e28fa P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.535068 11517 tablet_replica.cc:333] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.535763 11517 raft_consensus.cc:2243] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.536796 11517 raft_consensus.cc:2272] T e2a417d650a44658bfd5e2f5305185ac P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.539081 11517 tablet_replica.cc:333] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.539606 11517 raft_consensus.cc:2243] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.540318 11517 raft_consensus.cc:2272] T 522bcf0a155f4da196cb33f1aae90a82 P 3d539f8d95d343d8b1696803f18b30d2 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.542152 11517 tablet_replica.cc:333] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.542737 11517 raft_consensus.cc:2243] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.543290 11517 raft_consensus.cc:2272] T 9119721f3ee04810b1f277c06c966a45 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.545928 11517 tablet_replica.cc:333] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.546613 11517 raft_consensus.cc:2243] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.547389 11517 raft_consensus.cc:2272] T 18205d974a7c4592aec0153ef3e44c97 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.549161 11517 tablet_replica.cc:333] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.549774 11517 raft_consensus.cc:2243] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.550664 11517 raft_consensus.cc:2272] T 49dad260ffe242d2b4e0976f119a19d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.553074 11517 tablet_replica.cc:333] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.553716 11517 raft_consensus.cc:2243] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.554567 11517 raft_consensus.cc:2272] T 9155962ca7f34a8c83a1780ce0018a59 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.556376 11517 tablet_replica.cc:333] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.556929 11517 raft_consensus.cc:2243] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.557885 11517 raft_consensus.cc:2272] T ea604dddf4c244a181921c4dbdd0e720 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.559774 11517 tablet_replica.cc:333] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.560312 11517 raft_consensus.cc:2243] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.561295 11517 raft_consensus.cc:2272] T 9132272ce4c847a99e395a53c1015230 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.563730 11517 tablet_replica.cc:333] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.564432 11517 raft_consensus.cc:2243] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.565286 11517 raft_consensus.cc:2272] T 20db0f202ea546699789c2325d241b35 P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.567113 11517 tablet_replica.cc:333] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.567641 11517 raft_consensus.cc:2243] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.568962 11517 raft_consensus.cc:2272] T 612a4639a7a749be8856bbc60766e312 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.571290 11517 tablet_replica.cc:333] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.572080 11517 raft_consensus.cc:2243] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.572785 11517 raft_consensus.cc:2272] T bd1bc5f9b25e46cc866068bd346e11c6 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:37.574954 12516 catalog_manager.cc:4755] TS 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): ChangeConfig:REMOVE_PEER RPC failed for tablet 612a4639a7a749be8856bbc60766e312: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:37.575807 11517 tablet_replica.cc:333] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.576450 11517 raft_consensus.cc:2243] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.577456 11517 raft_consensus.cc:2272] T 866523bf2d2f442f971d504267ed200f P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.579344 11517 tablet_replica.cc:333] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.579905 11517 raft_consensus.cc:2243] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.580690 11517 raft_consensus.cc:2272] T 4c13fa2aff324f65ad0bf6191ea68fab P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.582542 11517 tablet_replica.cc:333] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.583137 11517 raft_consensus.cc:2243] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.583629 11517 raft_consensus.cc:2272] T 981fb2f435e24d06a81fc02cf4591cc5 P 3d539f8d95d343d8b1696803f18b30d2 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.585501 11517 tablet_replica.cc:333] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.586084 11517 raft_consensus.cc:2243] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.586880 11517 raft_consensus.cc:2272] T 4679cef42d5540e9a5d3a66e52cdcaf9 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.588608 11517 tablet_replica.cc:333] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.589202 11517 raft_consensus.cc:2243] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.590304 11517 raft_consensus.cc:2272] T 98a43c5d78c945bb8c5541bc892f4865 P 3d539f8d95d343d8b1696803f18b30d2 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.592325 11517 tablet_replica.cc:333] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.592995 11517 raft_consensus.cc:2243] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.594324 11517 raft_consensus.cc:2272] T 94e3333498524f14a17661cdb38ef144 P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.596191 11517 tablet_replica.cc:333] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.596732 11517 raft_consensus.cc:2243] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.597497 11517 raft_consensus.cc:2272] T af96e52b004d4c8c942cc0ec33c95c99 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.600116 11517 tablet_replica.cc:333] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.600996 11517 raft_consensus.cc:2243] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.601543 11517 raft_consensus.cc:2272] T ccf3f471aa044537a9492a86bb48b0aa P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.603732 11517 tablet_replica.cc:333] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.604324 11517 raft_consensus.cc:2243] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.604933 11517 raft_consensus.cc:2272] T ad2866c2079c4c5b9df5ad6a072e192d P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.606948 11517 tablet_replica.cc:333] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.607684 11517 raft_consensus.cc:2243] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.608816 11517 raft_consensus.cc:2272] T e390d5b6aea04f09ab5dbf11b20a7b98 P 3d539f8d95d343d8b1696803f18b30d2 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:37.609815 13348 consensus_peers.cc:597] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:37.611613 11517 tablet_replica.cc:333] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.612428 11517 raft_consensus.cc:2243] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.612999 11517 raft_consensus.cc:2272] T b1054d44ad14423fb3d51a5243499bb9 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.615082 11517 tablet_replica.cc:333] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.615648 11517 raft_consensus.cc:2243] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.616498 11517 raft_consensus.cc:2272] T ab5ddded6fd34f278dade99f2086c4e6 P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.618451 11517 tablet_replica.cc:333] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.619014 11517 raft_consensus.cc:2243] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.619913 11517 raft_consensus.cc:2272] T 0db63389cf0d45de92e4b01621db07cd P 3d539f8d95d343d8b1696803f18b30d2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.621896 11517 tablet_replica.cc:333] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.622428 11517 raft_consensus.cc:2243] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.622838 11517 raft_consensus.cc:2272] T e1a0f7a1130449548d1659cd21760fc9 P 3d539f8d95d343d8b1696803f18b30d2 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.624958 11517 tablet_replica.cc:333] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.625489 11517 raft_consensus.cc:2243] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.625957 11517 raft_consensus.cc:2272] T 58239dd301f64758846dd758e449e4d8 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.628019 11517 tablet_replica.cc:333] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.628564 11517 raft_consensus.cc:2243] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.629058 11517 raft_consensus.cc:2272] T d46747dede234934bea0f0c324106c28 P 3d539f8d95d343d8b1696803f18b30d2 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.631019 11517 tablet_replica.cc:333] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.631659 11517 raft_consensus.cc:2243] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.632287 11517 raft_consensus.cc:2272] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 3d539f8d95d343d8b1696803f18b30d2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.634208 11517 tablet_replica.cc:333] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.634760 11517 raft_consensus.cc:2243] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.635547 11517 raft_consensus.cc:2272] T c6cd31a55742463db9f0f0060bb60a77 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.638255 11517 tablet_replica.cc:333] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
W20260709 13:21:37.638720 13348 consensus_peers.cc:597] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:37.639086 11517 raft_consensus.cc:2243] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.640074 11517 raft_consensus.cc:2272] T bc7827272b424bad88349b6b7437844b P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.642620 11517 tablet_replica.cc:333] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.643370 11517 raft_consensus.cc:2243] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.644289 11517 raft_consensus.cc:2272] T 199931c4ffc44fee980d8bb49cf37f00 P 3d539f8d95d343d8b1696803f18b30d2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.646337 11517 tablet_replica.cc:333] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.646867 11517 raft_consensus.cc:2243] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.647348 11517 raft_consensus.cc:2272] T 4d53f974eee5475b9cd2701948f533ac P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.649614 11517 tablet_replica.cc:333] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.650260 11517 raft_consensus.cc:2243] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.650892 11517 raft_consensus.cc:2272] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.653007 11517 tablet_replica.cc:333] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.653549 11517 raft_consensus.cc:2243] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.654117 11517 raft_consensus.cc:2272] T e79ffe63999346abaccd90ab600091d7 P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.656127 11517 tablet_replica.cc:333] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.656657 11517 raft_consensus.cc:2243] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.657164 11517 raft_consensus.cc:2272] T febe63321d6c417cbfc5ccbca7caeefa P 3d539f8d95d343d8b1696803f18b30d2 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.659178 11517 tablet_replica.cc:333] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.659729 11517 raft_consensus.cc:2243] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.660378 11517 raft_consensus.cc:2272] T 775128ed7b8d4283bf303c8d40247615 P 3d539f8d95d343d8b1696803f18b30d2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.662643 11517 tablet_replica.cc:333] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.663362 11517 raft_consensus.cc:2243] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.664311 11517 raft_consensus.cc:2272] T a4d7de736149496b86a37ba2ca92670d P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:37.665437 13348 consensus_peers.cc:597] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:37.666575 11517 tablet_replica.cc:333] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.667200 11517 raft_consensus.cc:2243] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.667691 11517 raft_consensus.cc:2272] T 414b06f3e5f8472ea4071ebdd0161b9d P 3d539f8d95d343d8b1696803f18b30d2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.669788 11517 tablet_replica.cc:333] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.670356 11517 raft_consensus.cc:2243] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:37.670855 11517 raft_consensus.cc:2272] T d1362166bcbb48e88ac7b5862a23517e P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.672863 11517 tablet_replica.cc:333] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.673436 11517 raft_consensus.cc:2243] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.674460 11517 raft_consensus.cc:2272] T 3938a96da1794e32b484ac729226a8ce P 3d539f8d95d343d8b1696803f18b30d2 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.676519 11517 tablet_replica.cc:333] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.677198 11517 raft_consensus.cc:2243] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.678226 11517 raft_consensus.cc:2272] T 059429c0c4024dfb941d1ff325fe8f98 P 3d539f8d95d343d8b1696803f18b30d2 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.680615 11517 tablet_replica.cc:333] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.681135 11517 raft_consensus.cc:2243] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.682204 11517 raft_consensus.cc:2272] T 5067d5ea86fa4774a6be5b882e5b289d P 3d539f8d95d343d8b1696803f18b30d2 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:37.683897 11517 tablet_replica.cc:333] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2: stopping tablet replica
I20260709 13:21:37.684350 11517 raft_consensus.cc:2243] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:21:37.685216 11517 raft_consensus.cc:2272] T fa34731f37084c608660f0088a2dba09 P 3d539f8d95d343d8b1696803f18b30d2 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:37.708547 13349 consensus_peers.cc:597] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:37.712716 13348 consensus_peers.cc:597] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:37.821236 13348 consensus_peers.cc:597] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.67:41181: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:37.837033 13348 consensus_peers.cc:597] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.67:41181: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:37.839685 13348 consensus_peers.cc:597] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.67:41181: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:37.840191 13348 consensus_peers.cc:597] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.67:41181: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:37.844383 13348 consensus_peers.cc:597] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.67:41181: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:37.898315 13349 consensus_peers.cc:597] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:37.914949 13349 consensus_peers.cc:597] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:21:37.926204 21088 raft_consensus.cc:493] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:37.926735 21088 raft_consensus.cc:515] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:37.928545 21088 leader_election.cc:290] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:37.933004 13349 leader_election.cc:336] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:37.933677 13349 leader_election.cc:336] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:37.934036 13349 leader_election.cc:304] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:37.934705 21088 raft_consensus.cc:2749] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:37.972681 21088 raft_consensus.cc:493] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:37.973359 21088 raft_consensus.cc:515] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:37.975515 21088 leader_election.cc:290] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
W20260709 13:21:37.980752 13349 leader_election.cc:336] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:37.983538 13349 leader_election.cc:336] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:37.984017 13349 leader_election.cc:304] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:37.984709 21088 raft_consensus.cc:2749] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:21:37.999414 13348 consensus_peers.cc:597] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.67:41181: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:38.033789 13349 consensus_peers.cc:597] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:38.053314 13348 consensus_peers.cc:597] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Couldn't send request to peer 3d539f8d95d343d8b1696803f18b30d2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.67:41181: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:38.067420 13349 consensus_peers.cc:597] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:38.257975 13349 consensus_peers.cc:597] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:38.299314 13349 consensus_peers.cc:597] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:38.599030 11517 tablet_server.cc:196] TabletServer@127.11.63.67:0 shutdown complete.
I20260709 13:21:38.622821 21088 raft_consensus.cc:493] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:38.623415 21088 raft_consensus.cc:515] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:38.625545 21088 leader_election.cc:290] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:38.626801 21088 raft_consensus.cc:493] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:38.627367 21088 raft_consensus.cc:515] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:38.630025 21086 raft_consensus.cc:493] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 639ab2db265f4747bb7afb93e833f530)
I20260709 13:21:38.630620 21086 raft_consensus.cc:515] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
W20260709 13:21:38.632010 13348 leader_election.cc:336] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Network error: Client connection negotiation failed: client connection to 127.11.63.67:41181: connect: Connection refused (error 111)
W20260709 13:21:38.632489 13349 leader_election.cc:336] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:38.632573 21086 leader_election.cc:290] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173)
I20260709 13:21:38.629611 21088 leader_election.cc:290] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:38.632944 13349 leader_election.cc:304] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
W20260709 13:21:38.634131 13348 leader_election.cc:336] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Network error: Client connection negotiation failed: client connection to 127.11.63.67:41181: connect: Connection refused (error 111)
I20260709 13:21:38.634785 21088 raft_consensus.cc:2749] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:21:38.639267 13349 leader_election.cc:336] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:38.639820 13349 leader_election.cc:304] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:38.640462 21088 raft_consensus.cc:2749] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:21:38.641172 13349 leader_election.cc:336] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:38.643167 13349 leader_election.cc:336] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:38.643502 13349 leader_election.cc:304] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:38.644209 13349 consensus_peers.cc:597] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:38.644151 21088 raft_consensus.cc:2749] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:38.657655 21086 raft_consensus.cc:493] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:38.658211 21086 raft_consensus.cc:515] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:38.660279 21086 leader_election.cc:290] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:38.664127 13349 leader_election.cc:336] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
W20260709 13:21:38.664616 13348 leader_election.cc:336] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Network error: Client connection negotiation failed: client connection to 127.11.63.67:41181: connect: Connection refused (error 111)
I20260709 13:21:38.665063 13348 leader_election.cc:304] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:38.665855 21086 raft_consensus.cc:2749] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:38.699558 11517 tablet_server.cc:179] TabletServer@127.11.63.68:0 shutting down...
I20260709 13:21:38.714840 21086 raft_consensus.cc:493] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:38.715570 21086 raft_consensus.cc:515] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:38.717901 21086 leader_election.cc:290] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:38.721750 13349 leader_election.cc:336] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
W20260709 13:21:38.722361 13348 leader_election.cc:336] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Network error: Client connection negotiation failed: client connection to 127.11.63.67:41181: connect: Connection refused (error 111)
I20260709 13:21:38.722761 13348 leader_election.cc:304] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:38.723538 21086 raft_consensus.cc:2749] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:38.733707 21086 raft_consensus.cc:493] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:38.734333 21086 raft_consensus.cc:515] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:38.736418 21086 leader_election.cc:290] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
W20260709 13:21:38.741168 13348 leader_election.cc:336] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Network error: Client connection negotiation failed: client connection to 127.11.63.67:41181: connect: Connection refused (error 111)
W20260709 13:21:38.754310 13349 leader_election.cc:336] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:38.754792 13349 leader_election.cc:304] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:38.755540 21086 raft_consensus.cc:2749] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:38.756078 21088 raft_consensus.cc:493] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:38.756662 21088 raft_consensus.cc:515] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } attrs { replace: true } } peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:38.759089 21088 leader_election.cc:290] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:38.760803 21088 raft_consensus.cc:493] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:38.761327 21088 raft_consensus.cc:515] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:38.762424 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:38.763345 11517 tablet_replica.cc:333] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.763550 21088 leader_election.cc:290] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:38.764266 11517 raft_consensus.cc:2243] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:38.765074 11517 raft_consensus.cc:2272] T 5067d5ea86fa4774a6be5b882e5b289d P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:38.767038 13348 leader_election.cc:336] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Network error: Client connection negotiation failed: client connection to 127.11.63.67:41181: connect: Connection refused (error 111)
I20260709 13:21:38.768011 11517 tablet_replica.cc:333] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
W20260709 13:21:38.768540 13349 consensus_peers.cc:597] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Couldn't send request to peer 19cd864c053744bebd96093a7982537e. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:21:38.769009 11517 raft_consensus.cc:2243] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Raft consensus shutting down.
W20260709 13:21:38.769348 13349 leader_election.cc:336] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:38.769872 13349 leader_election.cc:304] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:38.769867 11517 raft_consensus.cc:2272] T f2f0a3339d324fd4902b6ff970ff0272 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.771272 21088 raft_consensus.cc:2749] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:21:38.772560 13349 leader_election.cc:336] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
W20260709 13:21:38.773512 13349 leader_election.cc:336] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:38.774121 11517 tablet_replica.cc:333] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.774943 11517 raft_consensus.cc:2243] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:21:38.775445 13348 leader_election.cc:336] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Network error: Client connection negotiation failed: client connection to 127.11.63.67:41181: connect: Connection refused (error 111)
I20260709 13:21:38.775575 11517 raft_consensus.cc:2272] T 612a4639a7a749be8856bbc60766e312 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.775871 13348 leader_election.cc:304] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:38.777192 21088 raft_consensus.cc:2749] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:21:38.778967 11517 tablet_replica.cc:333] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.779994 11517 raft_consensus.cc:2243] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:21:38.780779 21088 raft_consensus.cc:493] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:38.781168 11517 raft_consensus.cc:2272] T 775128ed7b8d4283bf303c8d40247615 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.781308 21088 raft_consensus.cc:515] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:38.783757 21088 leader_election.cc:290] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:38.784538 11517 tablet_replica.cc:333] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.785368 11517 raft_consensus.cc:2243] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:38.786235 11517 raft_consensus.cc:2272] T 94e3333498524f14a17661cdb38ef144 P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:38.787755 13349 leader_election.cc:336] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:38.789373 11517 tablet_replica.cc:333] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.790064 21086 raft_consensus.cc:493] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:38.790431 11517 raft_consensus.cc:2243] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Raft consensus shutting down.
W20260709 13:21:38.790478 13348 leader_election.cc:336] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Network error: Client connection negotiation failed: client connection to 127.11.63.67:41181: connect: Connection refused (error 111)
I20260709 13:21:38.790557 21086 raft_consensus.cc:515] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:38.791332 11517 raft_consensus.cc:2272] T 49dad260ffe242d2b4e0976f119a19d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.791366 13348 leader_election.cc:304] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:38.792740 21113 raft_consensus.cc:2749] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:38.792905 21086 leader_election.cc:290] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:38.794730 11517 tablet_replica.cc:333] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.795709 11517 raft_consensus.cc:2243] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:38.796324 11517 raft_consensus.cc:2272] T c4e4ae9d38ba4fd0a700998e4f4750be P 522fd2ba0e9f402691b5ef29af8579cd [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:38.799510 13348 leader_election.cc:336] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Network error: Client connection negotiation failed: client connection to 127.11.63.67:41181: connect: Connection refused (error 111)
I20260709 13:21:38.799934 11517 tablet_replica.cc:333] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
W20260709 13:21:38.800140 13349 leader_election.cc:336] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:38.800580 13349 leader_election.cc:304] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
I20260709 13:21:38.800889 11517 raft_consensus.cc:2243] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:38.801347 21088 raft_consensus.cc:2749] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:38.801956 11517 raft_consensus.cc:2272] T af96e52b004d4c8c942cc0ec33c95c99 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.802055 21113 raft_consensus.cc:493] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:38.802711 21113 raft_consensus.cc:515] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "19cd864c053744bebd96093a7982537e" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 43603 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:38.804404 11517 tablet_replica.cc:333] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.804544 21113 leader_election.cc:290] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 19cd864c053744bebd96093a7982537e (127.11.63.65:43603), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:38.805172 11517 raft_consensus.cc:2243] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:21:38.806296 11517 raft_consensus.cc:2272] T 981fb2f435e24d06a81fc02cf4591cc5 P 522fd2ba0e9f402691b5ef29af8579cd [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.808804 11517 tablet_replica.cc:333] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
W20260709 13:21:38.808763 13348 leader_election.cc:336] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Network error: Client connection negotiation failed: client connection to 127.11.63.67:41181: connect: Connection refused (error 111)
W20260709 13:21:38.809500 13349 leader_election.cc:336] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
I20260709 13:21:38.809906 11517 raft_consensus.cc:2243] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:38.809911 13349 leader_election.cc:304] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 19cd864c053744bebd96093a7982537e, 3d539f8d95d343d8b1696803f18b30d2
I20260709 13:21:38.810855 11517 raft_consensus.cc:2272] T 4c13fa2aff324f65ad0bf6191ea68fab P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.810876 21113 raft_consensus.cc:2749] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:38.813172 11517 tablet_replica.cc:333] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.813856 11517 raft_consensus.cc:2243] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:38.814366 11517 raft_consensus.cc:2272] T 522bcf0a155f4da196cb33f1aae90a82 P 522fd2ba0e9f402691b5ef29af8579cd [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.816303 11517 tablet_replica.cc:333] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.816838 11517 raft_consensus.cc:2243] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:38.817273 11517 raft_consensus.cc:2272] T eb7fca96394f4ba9b4fe3d7a6e278878 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.819187 11517 tablet_replica.cc:333] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.819720 11517 raft_consensus.cc:2243] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:38.820185 11517 raft_consensus.cc:2272] T 8f91059867c444d89718e83475c089cd P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.823069 11517 tablet_replica.cc:333] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.823772 11517 raft_consensus.cc:2243] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:21:38.825174 11517 raft_consensus.cc:2272] T bb2fc32fdf0a4cd981b02de0a47a7f96 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.827356 11517 tablet_replica.cc:333] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.827957 11517 raft_consensus.cc:2243] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 13:21:38.828166 13349 consensus_peers.cc:597] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd -> Peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Couldn't send request to peer 639ab2db265f4747bb7afb93e833f530. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:38.828492 11517 raft_consensus.cc:2272] T 8e0d0566acd7465f87e20053c75e28fa P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.830439 11517 tablet_replica.cc:333] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.831027 11517 raft_consensus.cc:2243] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:21:38.831904 11517 raft_consensus.cc:2272] T 5fd9f4e830ed4a178665ad4e4ecc4452 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.834075 11517 tablet_replica.cc:333] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.834654 11517 raft_consensus.cc:2243] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:38.835095 11517 raft_consensus.cc:2272] T 866523bf2d2f442f971d504267ed200f P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.837039 11517 tablet_replica.cc:333] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.837664 11517 raft_consensus.cc:2243] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:21:38.838544 11517 raft_consensus.cc:2272] T 4d53f974eee5475b9cd2701948f533ac P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.840866 11517 tablet_replica.cc:333] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.841683 11517 raft_consensus.cc:2243] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:21:38.842608 11517 raft_consensus.cc:2272] T b1054d44ad14423fb3d51a5243499bb9 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.844444 11517 tablet_replica.cc:333] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.845041 11517 raft_consensus.cc:2243] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:38.845693 11517 raft_consensus.cc:2272] T 199931c4ffc44fee980d8bb49cf37f00 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.847702 11517 tablet_replica.cc:333] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.848239 11517 raft_consensus.cc:2243] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:21:38.849164 11517 raft_consensus.cc:2272] T fb247fe4862541ccadc22bb79417d66c P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.850966 11517 tablet_replica.cc:333] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.851521 11517 raft_consensus.cc:2243] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:38.851953 11517 raft_consensus.cc:2272] T e2a417d650a44658bfd5e2f5305185ac P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.854219 11517 tablet_replica.cc:333] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.854847 11517 raft_consensus.cc:2243] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:21:38.855651 11517 raft_consensus.cc:2272] T d1362166bcbb48e88ac7b5862a23517e P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.857882 11517 tablet_replica.cc:333] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.858404 11517 raft_consensus.cc:2243] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:38.858855 11517 raft_consensus.cc:2272] T 5febc2df9d3c41359ceccc1bef8a54d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.860695 11517 tablet_replica.cc:333] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.861279 11517 raft_consensus.cc:2243] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:21:38.862223 11517 raft_consensus.cc:2272] T 18b2f67fae7f4963a4b4c562003df2fe P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.864193 11517 tablet_replica.cc:333] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.864737 11517 raft_consensus.cc:2243] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:38.865262 11517 raft_consensus.cc:2272] T ab5ddded6fd34f278dade99f2086c4e6 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.867080 11517 tablet_replica.cc:333] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.867592 11517 raft_consensus.cc:2243] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 30 LEADER]: Raft consensus shutting down.
I20260709 13:21:38.868599 11517 raft_consensus.cc:2272] T 15d86ce6de4b43719dc715ab6f4de742 P 522fd2ba0e9f402691b5ef29af8579cd [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.870569 11517 tablet_replica.cc:333] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.871207 11517 raft_consensus.cc:2243] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:38.871747 11517 raft_consensus.cc:2272] T 9132272ce4c847a99e395a53c1015230 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.872169 21113 raft_consensus.cc:493] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3d539f8d95d343d8b1696803f18b30d2)
I20260709 13:21:38.872617 21113 raft_consensus.cc:515] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "639ab2db265f4747bb7afb93e833f530" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 45173 } } peers { permanent_uuid: "3d539f8d95d343d8b1696803f18b30d2" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41181 } } peers { permanent_uuid: "522fd2ba0e9f402691b5ef29af8579cd" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36021 } attrs { promote: false } }
I20260709 13:21:38.874382 11517 tablet_replica.cc:333] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.874540 21113 leader_election.cc:290] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173), 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181)
I20260709 13:21:38.875306 11517 raft_consensus.cc:2243] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:38.875969 11517 raft_consensus.cc:2272] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:38.878219 13349 leader_election.cc:336] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 639ab2db265f4747bb7afb93e833f530 (127.11.63.66:45173): Network error: Client connection negotiation failed: client connection to 127.11.63.66:45173: connect: Connection refused (error 111)
I20260709 13:21:38.878458 11517 tablet_replica.cc:333] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.879065 11517 raft_consensus.cc:2243] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:38.879622 11517 raft_consensus.cc:2272] T 18205d974a7c4592aec0153ef3e44c97 P 522fd2ba0e9f402691b5ef29af8579cd [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:38.879737 13348 leader_election.cc:336] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): Network error: Client connection negotiation failed: client connection to 127.11.63.67:41181: connect: Connection refused (error 111)
I20260709 13:21:38.880198 13348 leader_election.cc:304] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 522fd2ba0e9f402691b5ef29af8579cd; no voters: 3d539f8d95d343d8b1696803f18b30d2, 639ab2db265f4747bb7afb93e833f530
W20260709 13:21:38.880797 13348 raft_consensus.cc:2686] T a4d7de736149496b86a37ba2ca92670d P 522fd2ba0e9f402691b5ef29af8579cd: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:21:38.882071 11517 tablet_replica.cc:333] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.882634 11517 raft_consensus.cc:2243] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:21:38.883445 11517 raft_consensus.cc:2272] T 98a43c5d78c945bb8c5541bc892f4865 P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.885407 11517 tablet_replica.cc:333] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.886001 11517 raft_consensus.cc:2243] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:38.886406 11517 raft_consensus.cc:2272] T 1fa805ee07a7429abe6b520bbfb94cd9 P 522fd2ba0e9f402691b5ef29af8579cd [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.888154 11517 tablet_replica.cc:333] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.888648 11517 raft_consensus.cc:2243] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:38.889094 11517 raft_consensus.cc:2272] T 9155962ca7f34a8c83a1780ce0018a59 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.891347 11517 tablet_replica.cc:333] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.892025 11517 raft_consensus.cc:2243] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:21:38.893059 11517 raft_consensus.cc:2272] T d46747dede234934bea0f0c324106c28 P 522fd2ba0e9f402691b5ef29af8579cd [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.895253 11517 tablet_replica.cc:333] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.895816 11517 raft_consensus.cc:2243] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:21:38.896610 11517 raft_consensus.cc:2272] T 1b13b87a5cea4263937041d123c5889d P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.898463 11517 tablet_replica.cc:333] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.898977 11517 raft_consensus.cc:2243] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:21:38.899864 11517 raft_consensus.cc:2272] T e1a0f7a1130449548d1659cd21760fc9 P 522fd2ba0e9f402691b5ef29af8579cd [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.901818 11517 tablet_replica.cc:333] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.902349 11517 raft_consensus.cc:2243] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:38.902961 11517 raft_consensus.cc:2272] T 3ca762dedae144be828acb659b8d9e4b P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.905236 11517 tablet_replica.cc:333] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.905874 11517 raft_consensus.cc:2243] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:21:38.906697 11517 raft_consensus.cc:2272] T 58239dd301f64758846dd758e449e4d8 P 522fd2ba0e9f402691b5ef29af8579cd [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.908579 11517 tablet_replica.cc:333] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.909092 11517 raft_consensus.cc:2243] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:38.909739 11517 raft_consensus.cc:2272] T 20db0f202ea546699789c2325d241b35 P 522fd2ba0e9f402691b5ef29af8579cd [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.911861 11517 tablet_replica.cc:333] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.912415 11517 raft_consensus.cc:2243] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:21:38.913225 11517 raft_consensus.cc:2272] T 3b98bb10e3e04334b7fcf7f781593682 P 522fd2ba0e9f402691b5ef29af8579cd [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.915024 11517 tablet_replica.cc:333] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.915519 11517 raft_consensus.cc:2243] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:21:38.916325 11517 raft_consensus.cc:2272] T 6146f826f1904f6f8b10ad518d454c22 P 522fd2ba0e9f402691b5ef29af8579cd [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.918426 11517 tablet_replica.cc:333] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.918919 11517 raft_consensus.cc:2243] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:38.919370 11517 raft_consensus.cc:2272] T 6609d5f93ab048558ea46e511ef3b2ec P 522fd2ba0e9f402691b5ef29af8579cd [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.921365 11517 tablet_replica.cc:333] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.921880 11517 raft_consensus.cc:2243] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:21:38.922622 11517 raft_consensus.cc:2272] T e79ffe63999346abaccd90ab600091d7 P 522fd2ba0e9f402691b5ef29af8579cd [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:38.924288 11517 tablet_replica.cc:333] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd: stopping tablet replica
I20260709 13:21:38.924737 11517 raft_consensus.cc:2243] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:38.925226 11517 raft_consensus.cc:2272] T 059429c0c4024dfb941d1ff325fe8f98 P 522fd2ba0e9f402691b5ef29af8579cd [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.252004 11517 tablet_server.cc:196] TabletServer@127.11.63.68:0 shutdown complete.
I20260709 13:21:39.317943 11517 master.cc:562] Master@127.11.63.126:38321 shutting down...
W20260709 13:21:40.346589 12517 catalog_manager.cc:4755] TS 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 98a43c5d78c945bb8c5541bc892f4865: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:21:43.573261 12517 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111) [suppressed 242 similar messages]
W20260709 13:21:43.577044 12517 catalog_manager.cc:4755] TS 19cd864c053744bebd96093a7982537e (127.11.63.65:43603): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 5067d5ea86fa4774a6be5b882e5b289d: Network error: Client connection negotiation failed: client connection to 127.11.63.65:43603: connect: Connection refused (error 111)
W20260709 13:22:05.016306 12563 debug-util.cc:398] Leaking SignalData structure 0x7b08001b7c80 after lost signal to thread 11520
W20260709 13:22:05.017022 12563 debug-util.cc:398] Leaking SignalData structure 0x7b08003b3be0 after lost signal to thread 12566
W20260709 13:22:10.390156 12516 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.63.67:41181: connect: Connection refused (error 111)
W20260709 13:22:10.394055 12516 catalog_manager.cc:4755] TS 3d539f8d95d343d8b1696803f18b30d2 (127.11.63.67:41181): ChangeConfig:REMOVE_PEER RPC failed for tablet 612a4639a7a749be8856bbc60766e312: Network error: Client connection negotiation failed: client connection to 127.11.63.67:41181: connect: Connection refused (error 111)
I20260709 13:22:12.569165 11517 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:12.570377 11517 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:12.570770 11517 tablet_replica.cc:333] T 00000000000000000000000000000000 P 480284ff372740a09dfd1f972d55d7eb: stopping tablet replica
I20260709 13:22:12.766122 11517 master.cc:584] Master@127.11.63.126:38321 shutdown complete.
I20260709 13:22:12.821810 11517 test_util.cc:182] -----------------------------------------------
I20260709 13:22:12.822072 11517 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603019296230-11517-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (290121 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:22:12.825944 11517 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.11.63.126:41481
I20260709 13:22:12.827142 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:12.834358 21124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:12.834745 21125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:12.836602 21127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:12.839938 11517 server_base.cc:1061] running on GCE node
I20260709 13:22:12.844456 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:12.844746 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:12.844924 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603332844904 us; error 0 us; skew 500 ppm
I20260709 13:22:12.845439 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:12.848237 11517 webserver.cc:533] Webserver started at http://127.11.63.126:35373/ using document root <none> and password file <none>
I20260709 13:22:12.848779 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:12.848994 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:12.849272 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:12.850708 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/master-0-root/instance:
uuid: "3c0b86f1b1314d79b7547279c55b079e"
format_stamp: "Formatted at 2026-07-09 13:22:12 on dist-test-slave-qv7r"
I20260709 13:22:12.855733 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 13:22:12.859766 21132 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:12.861534 11517 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 13:22:12.861961 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/master-0-root
uuid: "3c0b86f1b1314d79b7547279c55b079e"
format_stamp: "Formatted at 2026-07-09 13:22:12 on dist-test-slave-qv7r"
I20260709 13:22:12.862315 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:12.879995 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:12.881265 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:12.926445 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.126:41481
I20260709 13:22:12.926685 21183 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.126:41481 every 8 connection(s)
I20260709 13:22:12.931309 21184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:12.943521 21184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e: Bootstrap starting.
I20260709 13:22:12.948904 21184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:12.954241 21184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e: No bootstrap required, opened a new log
I20260709 13:22:12.956590 21184 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c0b86f1b1314d79b7547279c55b079e" member_type: VOTER }
I20260709 13:22:12.957047 21184 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:12.957245 21184 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c0b86f1b1314d79b7547279c55b079e, State: Initialized, Role: FOLLOWER
I20260709 13:22:12.957942 21184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c0b86f1b1314d79b7547279c55b079e" member_type: VOTER }
I20260709 13:22:12.958551 21184 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:22:12.958842 21184 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:22:12.959146 21184 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:12.964191 21184 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c0b86f1b1314d79b7547279c55b079e" member_type: VOTER }
I20260709 13:22:12.964797 21184 leader_election.cc:304] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c0b86f1b1314d79b7547279c55b079e; no voters: 
I20260709 13:22:12.966427 21184 leader_election.cc:290] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:22:12.966823 21187 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:12.968333 21187 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e [term 1 LEADER]: Becoming Leader. State: Replica: 3c0b86f1b1314d79b7547279c55b079e, State: Running, Role: LEADER
I20260709 13:22:12.969105 21187 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c0b86f1b1314d79b7547279c55b079e" member_type: VOTER }
I20260709 13:22:12.969445 21184 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:22:12.979024 21189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c0b86f1b1314d79b7547279c55b079e. Latest consensus state: current_term: 1 leader_uuid: "3c0b86f1b1314d79b7547279c55b079e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c0b86f1b1314d79b7547279c55b079e" member_type: VOTER } }
I20260709 13:22:12.979609 21189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:12.979828 21188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c0b86f1b1314d79b7547279c55b079e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c0b86f1b1314d79b7547279c55b079e" member_type: VOTER } }
I20260709 13:22:12.980376 21188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:12.982795 21197 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:22:12.990334 21197 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:22:12.991477 11517 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:22:13.000118 21197 catalog_manager.cc:1383] Generated new cluster ID: dfe9a9b67082495ea0bc99caffd1138c
I20260709 13:22:13.000422 21197 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:22:13.018354 21197 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:22:13.019793 21197 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:22:13.042775 21197 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e: Generated new TSK 0
I20260709 13:22:13.043340 21197 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:22:13.058444 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:13.066192 21205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:13.066548 21206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:13.068385 21208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:13.070209 11517 server_base.cc:1061] running on GCE node
I20260709 13:22:13.073052 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:13.073274 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:13.073493 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603333073454 us; error 0 us; skew 500 ppm
I20260709 13:22:13.074205 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:13.076891 11517 webserver.cc:533] Webserver started at http://127.11.63.65:43837/ using document root <none> and password file <none>
I20260709 13:22:13.077386 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:13.077572 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:13.077911 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:13.079748 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/ts-0-root/instance:
uuid: "ec879c0528c44465916efc6c33dde636"
format_stamp: "Formatted at 2026-07-09 13:22:13 on dist-test-slave-qv7r"
I20260709 13:22:13.084992 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:22:13.088619 21213 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:13.089534 11517 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 13:22:13.089922 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/ts-0-root
uuid: "ec879c0528c44465916efc6c33dde636"
format_stamp: "Formatted at 2026-07-09 13:22:13 on dist-test-slave-qv7r"
I20260709 13:22:13.090286 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:13.142978 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:13.144392 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:13.146015 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:13.148514 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:13.148708 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:13.149001 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:13.149168 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:13.196779 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.65:35963
I20260709 13:22:13.196916 21275 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.65:35963 every 8 connection(s)
I20260709 13:22:13.203303 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:13.212399 21280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:13.214653 21281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:13.215792 21283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:13.217444 11517 server_base.cc:1061] running on GCE node
I20260709 13:22:13.220522 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:13.220798 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:13.220988 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603333220966 us; error 0 us; skew 500 ppm
I20260709 13:22:13.221792 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:13.224118 21276 heartbeater.cc:344] Connected to a master server at 127.11.63.126:41481
I20260709 13:22:13.224488 21276 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:13.225334 11517 webserver.cc:533] Webserver started at http://127.11.63.66:42675/ using document root <none> and password file <none>
I20260709 13:22:13.225405 21276 heartbeater.cc:507] Master 127.11.63.126:41481 requested a full tablet report, sending...
I20260709 13:22:13.226016 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:13.226294 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:13.226608 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:13.227921 21149 ts_manager.cc:194] Registered new tserver with Master: ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:13.228415 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/ts-1-root/instance:
uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
format_stamp: "Formatted at 2026-07-09 13:22:13 on dist-test-slave-qv7r"
I20260709 13:22:13.230664 21149 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47824
I20260709 13:22:13.234472 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.006s
I20260709 13:22:13.238982 21288 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:13.240098 11517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:22:13.240459 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/ts-1-root
uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
format_stamp: "Formatted at 2026-07-09 13:22:13 on dist-test-slave-qv7r"
I20260709 13:22:13.240761 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:13.250573 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:13.251842 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:13.253454 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:13.255714 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:13.255898 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:13.256191 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:13.256348 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:13.301013 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.66:38825
I20260709 13:22:13.301116 21350 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.66:38825 every 8 connection(s)
I20260709 13:22:13.306327 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:13.315060 21354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:13.318352 21355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:13.319067 21357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:13.320147 11517 server_base.cc:1061] running on GCE node
I20260709 13:22:13.321113 21351 heartbeater.cc:344] Connected to a master server at 127.11.63.126:41481
I20260709 13:22:13.321300 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:22:13.321547 21351 heartbeater.cc:461] Registering TS with master...
W20260709 13:22:13.321611 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:13.322059 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603333322037 us; error 0 us; skew 500 ppm
I20260709 13:22:13.322705 21351 heartbeater.cc:507] Master 127.11.63.126:41481 requested a full tablet report, sending...
I20260709 13:22:13.322892 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:13.325141 21149 ts_manager.cc:194] Registered new tserver with Master: 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:13.325747 11517 webserver.cc:533] Webserver started at http://127.11.63.67:34115/ using document root <none> and password file <none>
I20260709 13:22:13.326282 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:13.326504 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:13.326771 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:13.326777 21149 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47834
I20260709 13:22:13.328353 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/ts-2-root/instance:
uuid: "74e2ff1a68ac4c849ee6397ebb011375"
format_stamp: "Formatted at 2026-07-09 13:22:13 on dist-test-slave-qv7r"
I20260709 13:22:13.333904 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.003s
I20260709 13:22:13.337423 21362 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:13.338310 11517 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:22:13.338742 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/ts-2-root
uuid: "74e2ff1a68ac4c849ee6397ebb011375"
format_stamp: "Formatted at 2026-07-09 13:22:13 on dist-test-slave-qv7r"
I20260709 13:22:13.338994 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:13.351717 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:13.352952 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:13.354557 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:13.356739 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:13.356972 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:13.357213 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:13.357388 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:13.401448 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.67:43181
I20260709 13:22:13.401686 21424 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.67:43181 every 8 connection(s)
I20260709 13:22:13.406548 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:13.415189 21428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:13.417475 21429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:13.418903 11517 server_base.cc:1061] running on GCE node
W20260709 13:22:13.419773 21431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:13.420806 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:13.421077 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:13.421146 21425 heartbeater.cc:344] Connected to a master server at 127.11.63.126:41481
I20260709 13:22:13.421339 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603333421316 us; error 0 us; skew 500 ppm
I20260709 13:22:13.421803 21425 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:13.422339 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:13.422768 21425 heartbeater.cc:507] Master 127.11.63.126:41481 requested a full tablet report, sending...
I20260709 13:22:13.425133 21149 ts_manager.cc:194] Registered new tserver with Master: 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:13.426076 11517 webserver.cc:533] Webserver started at http://127.11.63.68:42397/ using document root <none> and password file <none>
I20260709 13:22:13.426622 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:13.426690 21149 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47836
I20260709 13:22:13.426927 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:13.427327 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:13.428912 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/ts-3-root/instance:
uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
format_stamp: "Formatted at 2026-07-09 13:22:13 on dist-test-slave-qv7r"
I20260709 13:22:13.434046 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:22:13.437723 21436 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:13.438627 11517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:22:13.438956 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/ts-3-root
uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
format_stamp: "Formatted at 2026-07-09 13:22:13 on dist-test-slave-qv7r"
I20260709 13:22:13.439270 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:13.461970 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:13.463325 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:13.464769 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:13.467274 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:13.467514 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:13.467774 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:13.467957 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:13.515264 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.68:36863
I20260709 13:22:13.515367 21498 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.68:36863 every 8 connection(s)
I20260709 13:22:13.529996 21499 heartbeater.cc:344] Connected to a master server at 127.11.63.126:41481
I20260709 13:22:13.530406 21499 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:13.531152 21499 heartbeater.cc:507] Master 127.11.63.126:41481 requested a full tablet report, sending...
I20260709 13:22:13.533252 21149 ts_manager.cc:194] Registered new tserver with Master: 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:13.533831 11517 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.015201797s
I20260709 13:22:13.535310 21149 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47842
I20260709 13:22:13.562472 21149 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47854:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:22:13.805492 21464 tablet_service.cc:1511] Processing CreateTablet for tablet 18a79b2b6d2c4326ab9e4332e0d7406f (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:22:13.805994 21316 tablet_service.cc:1511] Processing CreateTablet for tablet 18a79b2b6d2c4326ab9e4332e0d7406f (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:22:13.807478 21464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18a79b2b6d2c4326ab9e4332e0d7406f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.807608 21316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18a79b2b6d2c4326ab9e4332e0d7406f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.819808 21390 tablet_service.cc:1511] Processing CreateTablet for tablet 18a79b2b6d2c4326ab9e4332e0d7406f (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:22:13.821043 21315 tablet_service.cc:1511] Processing CreateTablet for tablet e532ca31c67547998ba2fb09244573b3 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:22:13.822177 21390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18a79b2b6d2c4326ab9e4332e0d7406f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.823153 21315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e532ca31c67547998ba2fb09244573b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.825802 21389 tablet_service.cc:1511] Processing CreateTablet for tablet e532ca31c67547998ba2fb09244573b3 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:22:13.827318 21389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e532ca31c67547998ba2fb09244573b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.831692 21463 tablet_service.cc:1511] Processing CreateTablet for tablet f301fe3cc72b4e439644a7d1617921aa (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:22:13.833249 21463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f301fe3cc72b4e439644a7d1617921aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.836741 21314 tablet_service.cc:1511] Processing CreateTablet for tablet f301fe3cc72b4e439644a7d1617921aa (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:22:13.838589 21314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f301fe3cc72b4e439644a7d1617921aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.842268 21388 tablet_service.cc:1511] Processing CreateTablet for tablet 7772ca3f93fe431e8108b4e7ab98786e (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:22:13.843761 21388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7772ca3f93fe431e8108b4e7ab98786e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.846442 21313 tablet_service.cc:1511] Processing CreateTablet for tablet 228f7ae94b2848ecb12f93717756ba8b (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:22:13.847376 21462 tablet_service.cc:1511] Processing CreateTablet for tablet 7772ca3f93fe431e8108b4e7ab98786e (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:22:13.849042 21462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7772ca3f93fe431e8108b4e7ab98786e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.850965 21387 tablet_service.cc:1511] Processing CreateTablet for tablet 228f7ae94b2848ecb12f93717756ba8b (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:22:13.852491 21387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 228f7ae94b2848ecb12f93717756ba8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.852680 21312 tablet_service.cc:1511] Processing CreateTablet for tablet ff163c4709bd4437bd70e20440389d62 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:22:13.854714 21386 tablet_service.cc:1511] Processing CreateTablet for tablet 347c76112a0e430ca7d607278f20c811 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:22:13.855234 21461 tablet_service.cc:1511] Processing CreateTablet for tablet ff163c4709bd4437bd70e20440389d62 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:22:13.856184 21386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 347c76112a0e430ca7d607278f20c811. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.856588 21461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff163c4709bd4437bd70e20440389d62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.847944 21313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 228f7ae94b2848ecb12f93717756ba8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.860846 21460 tablet_service.cc:1511] Processing CreateTablet for tablet 347c76112a0e430ca7d607278f20c811 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:22:13.863487 21385 tablet_service.cc:1511] Processing CreateTablet for tablet d7c665892b494d9db72239758708b947 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:22:13.864408 21311 tablet_service.cc:1511] Processing CreateTablet for tablet d7c665892b494d9db72239758708b947 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:22:13.865087 21385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7c665892b494d9db72239758708b947. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.866066 21312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff163c4709bd4437bd70e20440389d62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.866987 21311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7c665892b494d9db72239758708b947. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.867818 21460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 347c76112a0e430ca7d607278f20c811. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.889338 21241 tablet_service.cc:1511] Processing CreateTablet for tablet e532ca31c67547998ba2fb09244573b3 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:22:13.891001 21241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e532ca31c67547998ba2fb09244573b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.889338 21240 tablet_service.cc:1511] Processing CreateTablet for tablet f301fe3cc72b4e439644a7d1617921aa (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:22:13.894336 21240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f301fe3cc72b4e439644a7d1617921aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.894150 21237 tablet_service.cc:1511] Processing CreateTablet for tablet ff163c4709bd4437bd70e20440389d62 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:22:13.896648 21236 tablet_service.cc:1511] Processing CreateTablet for tablet 347c76112a0e430ca7d607278f20c811 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:22:13.892701 21238 tablet_service.cc:1511] Processing CreateTablet for tablet 228f7ae94b2848ecb12f93717756ba8b (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:22:13.890618 21239 tablet_service.cc:1511] Processing CreateTablet for tablet 7772ca3f93fe431e8108b4e7ab98786e (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:22:13.900502 21237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff163c4709bd4437bd70e20440389d62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.906034 21236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 347c76112a0e430ca7d607278f20c811. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.910027 21238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 228f7ae94b2848ecb12f93717756ba8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.910764 21239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7772ca3f93fe431e8108b4e7ab98786e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.939352 21459 tablet_service.cc:1511] Processing CreateTablet for tablet d7c665892b494d9db72239758708b947 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:22:13.941008 21459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7c665892b494d9db72239758708b947. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.969419 21235 tablet_service.cc:1511] Processing CreateTablet for tablet 49acbb28158c4780add00f91662769da (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:22:13.970569 21310 tablet_service.cc:1511] Processing CreateTablet for tablet 49acbb28158c4780add00f91662769da (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:22:13.971062 21235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49acbb28158c4780add00f91662769da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:13.972005 21310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49acbb28158c4780add00f91662769da. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:22:13.981312 21196 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:13.984555 21518 tablet_bootstrap.cc:492] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:14.005656 21384 tablet_service.cc:1511] Processing CreateTablet for tablet 49acbb28158c4780add00f91662769da (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:22:14.007180 21384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49acbb28158c4780add00f91662769da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.020166 21518 tablet_bootstrap.cc:654] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:14.024924 21463 tablet_service.cc:1511] Processing CreateTablet for tablet 6b569ec3d17d412c8c9087446e64bb77 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:22:14.026335 21463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b569ec3d17d412c8c9087446e64bb77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.044013 21519 tablet_bootstrap.cc:492] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:14.056938 21519 tablet_bootstrap.cc:654] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:14.063712 21236 tablet_service.cc:1511] Processing CreateTablet for tablet 6b569ec3d17d412c8c9087446e64bb77 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:22:14.065979 21236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b569ec3d17d412c8c9087446e64bb77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.069433 21520 tablet_bootstrap.cc:492] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:14.072748 21521 tablet_bootstrap.cc:492] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:14.072916 21387 tablet_service.cc:1511] Processing CreateTablet for tablet 6558bd3928cd424fa13c61523335998a (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:22:14.074330 21387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6558bd3928cd424fa13c61523335998a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.074165 21314 tablet_service.cc:1511] Processing CreateTablet for tablet 6b569ec3d17d412c8c9087446e64bb77 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:22:14.075397 21518 tablet_bootstrap.cc:492] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:14.075544 21314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b569ec3d17d412c8c9087446e64bb77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.075932 21518 ts_tablet_manager.cc:1403] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.092s	user 0.014s	sys 0.000s
I20260709 13:22:14.083165 21518 raft_consensus.cc:359] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:14.084066 21518 raft_consensus.cc:385] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:14.084465 21518 raft_consensus.cc:740] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:14.085695 21520 tablet_bootstrap.cc:654] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:14.084957 21462 tablet_service.cc:1511] Processing CreateTablet for tablet 6558bd3928cd424fa13c61523335998a (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:22:14.085314 21518 consensus_queue.cc:260] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:14.089620 21239 tablet_service.cc:1511] Processing CreateTablet for tablet 6558bd3928cd424fa13c61523335998a (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:22:14.090727 21462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6558bd3928cd424fa13c61523335998a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.091197 21239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6558bd3928cd424fa13c61523335998a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.086112 21521 tablet_bootstrap.cc:654] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:14.105646 21312 tablet_service.cc:1511] Processing CreateTablet for tablet 5957ec3bcdcf40838d52d01a2a65281f (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:22:14.106614 21384 tablet_service.cc:1511] Processing CreateTablet for tablet 5957ec3bcdcf40838d52d01a2a65281f (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:22:14.107079 21312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5957ec3bcdcf40838d52d01a2a65281f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.108024 21384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5957ec3bcdcf40838d52d01a2a65281f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.111521 21519 tablet_bootstrap.cc:492] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:14.112075 21519 ts_tablet_manager.cc:1403] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.068s	user 0.011s	sys 0.008s
I20260709 13:22:14.115587 21519 raft_consensus.cc:359] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:14.116346 21519 raft_consensus.cc:385] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:14.116676 21519 raft_consensus.cc:740] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:14.117537 21519 consensus_queue.cc:260] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:14.119031 21518 ts_tablet_manager.cc:1434] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.043s	user 0.012s	sys 0.000s
I20260709 13:22:14.128037 21425 heartbeater.cc:499] Master 127.11.63.126:41481 was elected leader, sending a full tablet report...
I20260709 13:22:14.133684 21518 tablet_bootstrap.cc:492] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:14.145313 21462 tablet_service.cc:1511] Processing CreateTablet for tablet 5957ec3bcdcf40838d52d01a2a65281f (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:22:14.148754 21236 tablet_service.cc:1511] Processing CreateTablet for tablet 01967f3f14be4ecaa1cc2469a50fc156 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:22:14.155265 21462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5957ec3bcdcf40838d52d01a2a65281f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:22:14.156093 21426 tablet.cc:2406] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:22:14.152807 21521 tablet_bootstrap.cc:492] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:14.158435 21521 ts_tablet_manager.cc:1403] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.086s	user 0.013s	sys 0.000s
I20260709 13:22:14.158514 21236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01967f3f14be4ecaa1cc2469a50fc156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.165511 21314 tablet_service.cc:1511] Processing CreateTablet for tablet 01967f3f14be4ecaa1cc2469a50fc156 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:22:14.175906 21384 tablet_service.cc:1511] Processing CreateTablet for tablet 01967f3f14be4ecaa1cc2469a50fc156 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:22:14.176824 21314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01967f3f14be4ecaa1cc2469a50fc156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.177321 21384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01967f3f14be4ecaa1cc2469a50fc156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.169512 21463 tablet_service.cc:1511] Processing CreateTablet for tablet d4eebc08c8f342a69d7a1248e7df69a1 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:22:14.170470 21240 tablet_service.cc:1511] Processing CreateTablet for tablet d4eebc08c8f342a69d7a1248e7df69a1 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:22:14.172540 21316 tablet_service.cc:1511] Processing CreateTablet for tablet d4eebc08c8f342a69d7a1248e7df69a1 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:22:14.181080 21520 tablet_bootstrap.cc:492] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:14.189100 21520 ts_tablet_manager.cc:1403] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.120s	user 0.031s	sys 0.024s
I20260709 13:22:14.182797 21390 tablet_service.cc:1511] Processing CreateTablet for tablet 8188c780f29f41218f57313c157dae63 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:22:14.193171 21390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8188c780f29f41218f57313c157dae63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.193377 21387 tablet_service.cc:1511] Processing CreateTablet for tablet 97dbdec3b8bc40898dcd64ba19a4ec5d (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:22:14.195022 21387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97dbdec3b8bc40898dcd64ba19a4ec5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.185734 21463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4eebc08c8f342a69d7a1248e7df69a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.175601 21518 tablet_bootstrap.cc:654] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:14.199817 21499 heartbeater.cc:499] Master 127.11.63.126:41481 was elected leader, sending a full tablet report...
I20260709 13:22:14.202276 21520 raft_consensus.cc:359] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:14.203073 21520 raft_consensus.cc:385] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:14.202759 21459 tablet_service.cc:1511] Processing CreateTablet for tablet 97dbdec3b8bc40898dcd64ba19a4ec5d (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:22:14.203379 21520 raft_consensus.cc:740] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:14.204098 21459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97dbdec3b8bc40898dcd64ba19a4ec5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.204330 21520 consensus_queue.cc:260] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:14.210045 21239 tablet_service.cc:1511] Processing CreateTablet for tablet d31c45d90b684fb18b03156866654665 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:22:14.211529 21312 tablet_service.cc:1511] Processing CreateTablet for tablet d31c45d90b684fb18b03156866654665 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:22:14.214644 21464 tablet_service.cc:1511] Processing CreateTablet for tablet 5fec8ff4bc5a4d558263e38ae75ece75 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:22:14.216182 21464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fec8ff4bc5a4d558263e38ae75ece75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.188412 21316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4eebc08c8f342a69d7a1248e7df69a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.217592 21313 tablet_service.cc:1511] Processing CreateTablet for tablet 5fec8ff4bc5a4d558263e38ae75ece75 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:22:14.218953 21313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fec8ff4bc5a4d558263e38ae75ece75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.185019 21241 tablet_service.cc:1511] Processing CreateTablet for tablet 8188c780f29f41218f57313c157dae63 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:22:14.222676 21461 tablet_service.cc:1511] Processing CreateTablet for tablet cf088eed547f42aea083b31277ecece1 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:22:14.223563 21315 tablet_service.cc:1511] Processing CreateTablet for tablet a85e6742ffca4d9c962b8613986c3896 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:22:14.224858 21315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a85e6742ffca4d9c962b8613986c3896. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.224601 21385 tablet_service.cc:1511] Processing CreateTablet for tablet d31c45d90b684fb18b03156866654665 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:22:14.225257 21519 ts_tablet_manager.cc:1434] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.113s	user 0.021s	sys 0.030s
I20260709 13:22:14.226063 21385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d31c45d90b684fb18b03156866654665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.226455 21519 tablet_bootstrap.cc:492] T 347c76112a0e430ca7d607278f20c811 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:14.229481 21311 tablet_service.cc:1511] Processing CreateTablet for tablet d275cc2417214c6f80ff88e7fc3916b7 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:22:14.230800 21311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d275cc2417214c6f80ff88e7fc3916b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.187561 21310 tablet_service.cc:1511] Processing CreateTablet for tablet 97dbdec3b8bc40898dcd64ba19a4ec5d (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:22:14.187181 21240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4eebc08c8f342a69d7a1248e7df69a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.232033 21310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97dbdec3b8bc40898dcd64ba19a4ec5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.233800 21276 heartbeater.cc:499] Master 127.11.63.126:41481 was elected leader, sending a full tablet report...
I20260709 13:22:14.178943 21521 raft_consensus.cc:359] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:14.234330 21521 raft_consensus.cc:385] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:14.234692 21521 raft_consensus.cc:740] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:14.234882 21458 tablet_service.cc:1511] Processing CreateTablet for tablet 200a5b44be554a6bb18a21a29836902e (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:22:14.235606 21521 consensus_queue.cc:260] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:14.238068 21309 tablet_service.cc:1511] Processing CreateTablet for tablet 200a5b44be554a6bb18a21a29836902e (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:22:14.239511 21309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 200a5b44be554a6bb18a21a29836902e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.239570 21235 tablet_service.cc:1511] Processing CreateTablet for tablet 5fec8ff4bc5a4d558263e38ae75ece75 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:22:14.241103 21235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fec8ff4bc5a4d558263e38ae75ece75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.241796 21461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf088eed547f42aea083b31277ecece1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.242686 21238 tablet_service.cc:1511] Processing CreateTablet for tablet cf088eed547f42aea083b31277ecece1 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:22:14.244117 21238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf088eed547f42aea083b31277ecece1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.245633 21237 tablet_service.cc:1511] Processing CreateTablet for tablet d275cc2417214c6f80ff88e7fc3916b7 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:22:14.247048 21237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d275cc2417214c6f80ff88e7fc3916b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.247519 21308 tablet_service.cc:1511] Processing CreateTablet for tablet 799c8e732d4d46e3b26dcd77f458dcbe (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:22:14.248880 21308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 799c8e732d4d46e3b26dcd77f458dcbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.251040 21457 tablet_service.cc:1511] Processing CreateTablet for tablet ad13ab5b32d74d80990d10ba376cc2c7 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:22:14.251616 21307 tablet_service.cc:1511] Processing CreateTablet for tablet 311e1d9a839a418296e3f320abad6c88 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:22:14.253023 21307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 311e1d9a839a418296e3f320abad6c88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.253674 21456 tablet_service.cc:1511] Processing CreateTablet for tablet 799c8e732d4d46e3b26dcd77f458dcbe (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:22:14.256399 21455 tablet_service.cc:1511] Processing CreateTablet for tablet d2b503b6f79842efbb8a7ecd5c5851e0 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:22:14.259819 21386 tablet_service.cc:1511] Processing CreateTablet for tablet d275cc2417214c6f80ff88e7fc3916b7 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:22:14.261089 21386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d275cc2417214c6f80ff88e7fc3916b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.261018 21389 tablet_service.cc:1511] Processing CreateTablet for tablet a85e6742ffca4d9c962b8613986c3896 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:22:14.264145 21383 tablet_service.cc:1511] Processing CreateTablet for tablet ad13ab5b32d74d80990d10ba376cc2c7 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:22:14.230623 21460 tablet_service.cc:1511] Processing CreateTablet for tablet a85e6742ffca4d9c962b8613986c3896 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:22:14.248785 21234 tablet_service.cc:1511] Processing CreateTablet for tablet 200a5b44be554a6bb18a21a29836902e (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:22:14.265865 21525 raft_consensus.cc:493] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:14.268863 21382 tablet_service.cc:1511] Processing CreateTablet for tablet 799c8e732d4d46e3b26dcd77f458dcbe (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:22:14.270944 21381 tablet_service.cc:1511] Processing CreateTablet for tablet 311e1d9a839a418296e3f320abad6c88 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:22:14.230623 21388 tablet_service.cc:1511] Processing CreateTablet for tablet cf088eed547f42aea083b31277ecece1 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:22:14.272420 21241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8188c780f29f41218f57313c157dae63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.282953 21525 raft_consensus.cc:515] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:14.280207 21383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad13ab5b32d74d80990d10ba376cc2c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.290663 21462 tablet_service.cc:1511] Processing CreateTablet for tablet 8188c780f29f41218f57313c157dae63 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:22:14.285225 21312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d31c45d90b684fb18b03156866654665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.301123 21519 tablet_bootstrap.cc:654] T 347c76112a0e430ca7d607278f20c811 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:14.301393 21458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 200a5b44be554a6bb18a21a29836902e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.314254 21314 tablet_service.cc:1511] Processing CreateTablet for tablet d2b503b6f79842efbb8a7ecd5c5851e0 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:22:14.318624 21234 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 200a5b44be554a6bb18a21a29836902e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.349685 21388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf088eed547f42aea083b31277ecece1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.345255 21351 heartbeater.cc:499] Master 127.11.63.126:41481 was elected leader, sending a full tablet report...
I20260709 13:22:14.373095 21455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2b503b6f79842efbb8a7ecd5c5851e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.385035 21462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8188c780f29f41218f57313c157dae63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.388545 21460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a85e6742ffca4d9c962b8613986c3896. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.368532 21389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a85e6742ffca4d9c962b8613986c3896. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.388953 21314 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2b503b6f79842efbb8a7ecd5c5851e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.394016 21381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 311e1d9a839a418296e3f320abad6c88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.394107 21233 tablet_service.cc:1511] Processing CreateTablet for tablet 311e1d9a839a418296e3f320abad6c88 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:22:14.395236 21232 tablet_service.cc:1511] Processing CreateTablet for tablet d2b503b6f79842efbb8a7ecd5c5851e0 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:22:14.368345 21239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d31c45d90b684fb18b03156866654665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.397133 21382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 799c8e732d4d46e3b26dcd77f458dcbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.402392 21525 leader_election.cc:290] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:14.397730 21233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 311e1d9a839a418296e3f320abad6c88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.377432 21384 tablet_service.cc:1511] Processing CreateTablet for tablet 7a5e0169610b4059877434f37effb058 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:22:14.410039 21457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad13ab5b32d74d80990d10ba376cc2c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.399972 21236 tablet_service.cc:1511] Processing CreateTablet for tablet ad13ab5b32d74d80990d10ba376cc2c7 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:22:14.414541 21456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 799c8e732d4d46e3b26dcd77f458dcbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.374802 21529 raft_consensus.cc:493] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:14.410411 21384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a5e0169610b4059877434f37effb058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.417498 21529 raft_consensus.cc:515] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:14.418202 21240 tablet_service.cc:1511] Processing CreateTablet for tablet 51766d5a86a6498b8e30249efa0ac990 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:22:14.420454 21520 ts_tablet_manager.cc:1434] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.231s	user 0.017s	sys 0.021s
I20260709 13:22:14.423781 21529 leader_election.cc:290] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:14.431007 21232 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2b503b6f79842efbb8a7ecd5c5851e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.448926 21520 tablet_bootstrap.cc:492] T d7c665892b494d9db72239758708b947 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:14.457219 21236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad13ab5b32d74d80990d10ba376cc2c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.475835 21311 tablet_service.cc:1511] Processing CreateTablet for tablet 7a5e0169610b4059877434f37effb058 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:22:14.479727 21521 ts_tablet_manager.cc:1434] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.304s	user 0.007s	sys 0.017s
I20260709 13:22:14.480906 21521 tablet_bootstrap.cc:492] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:14.477433 21311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a5e0169610b4059877434f37effb058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.479344 21390 tablet_service.cc:1511] Processing CreateTablet for tablet 5c0c8b12f7474d579a8805d344efb092 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:22:14.495146 21518 tablet_bootstrap.cc:492] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:14.495769 21518 ts_tablet_manager.cc:1403] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.362s	user 0.019s	sys 0.036s
I20260709 13:22:14.499116 21520 tablet_bootstrap.cc:654] T d7c665892b494d9db72239758708b947 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:14.501821 21309 tablet_service.cc:1511] Processing CreateTablet for tablet 51766d5a86a6498b8e30249efa0ac990 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:22:14.502820 21390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c0c8b12f7474d579a8805d344efb092. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.503288 21309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51766d5a86a6498b8e30249efa0ac990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.504046 21310 tablet_service.cc:1511] Processing CreateTablet for tablet a1bf171a09b745ec963d790f1eefe432 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:22:14.505311 21310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1bf171a09b745ec963d790f1eefe432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.506796 21459 tablet_service.cc:1511] Processing CreateTablet for tablet 7a5e0169610b4059877434f37effb058 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:22:14.508098 21387 tablet_service.cc:1511] Processing CreateTablet for tablet 51766d5a86a6498b8e30249efa0ac990 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:22:14.513209 21387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51766d5a86a6498b8e30249efa0ac990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.516286 21383 tablet_service.cc:1511] Processing CreateTablet for tablet bc40f11dce5540d1a056e9bacba075e2 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:22:14.508095 21459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a5e0169610b4059877434f37effb058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.530759 21237 tablet_service.cc:1511] Processing CreateTablet for tablet a1bf171a09b745ec963d790f1eefe432 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:22:14.517935 21464 tablet_service.cc:1511] Processing CreateTablet for tablet 5c0c8b12f7474d579a8805d344efb092 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:22:14.534492 21464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c0c8b12f7474d579a8805d344efb092. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.518795 21518 raft_consensus.cc:359] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:14.537778 21518 raft_consensus.cc:385] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:14.538157 21518 raft_consensus.cc:740] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:14.539026 21518 consensus_queue.cc:260] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:14.511567 21241 tablet_service.cc:1511] Processing CreateTablet for tablet 5c0c8b12f7474d579a8805d344efb092 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:22:14.543711 21518 ts_tablet_manager.cc:1434] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.047s	user 0.010s	sys 0.001s
I20260709 13:22:14.544843 21518 tablet_bootstrap.cc:492] T 7772ca3f93fe431e8108b4e7ab98786e P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:14.498921 21463 tablet_service.cc:1511] Processing CreateTablet for tablet bc40f11dce5540d1a056e9bacba075e2 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:22:14.546017 21463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc40f11dce5540d1a056e9bacba075e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.534415 21519 tablet_bootstrap.cc:492] T 347c76112a0e430ca7d607278f20c811 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:14.556797 21240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51766d5a86a6498b8e30249efa0ac990. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.565725 21519 ts_tablet_manager.cc:1403] T 347c76112a0e430ca7d607278f20c811 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.340s	user 0.023s	sys 0.013s
I20260709 13:22:14.571614 21241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c0c8b12f7474d579a8805d344efb092. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.547658 21455 tablet_service.cc:1511] Processing CreateTablet for tablet a1bf171a09b745ec963d790f1eefe432 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:22:14.584117 21455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1bf171a09b745ec963d790f1eefe432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.585915 21237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1bf171a09b745ec963d790f1eefe432. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.586220 21385 tablet_service.cc:1511] Processing CreateTablet for tablet 37a3c02685464cf1af805559b46c4356 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:22:14.586586 21235 tablet_service.cc:1511] Processing CreateTablet for tablet bc40f11dce5540d1a056e9bacba075e2 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:22:14.587589 21385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37a3c02685464cf1af805559b46c4356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.587833 21235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc40f11dce5540d1a056e9bacba075e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.590415 21312 tablet_service.cc:1511] Processing CreateTablet for tablet 37a3c02685464cf1af805559b46c4356 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:22:14.591758 21312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37a3c02685464cf1af805559b46c4356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.569137 21519 raft_consensus.cc:359] T 347c76112a0e430ca7d607278f20c811 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:14.602819 21527 raft_consensus.cc:493] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:14.603041 21383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc40f11dce5540d1a056e9bacba075e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.603435 21527 raft_consensus.cc:515] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:14.600909 21519 raft_consensus.cc:385] T 347c76112a0e430ca7d607278f20c811 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:14.623651 21239 tablet_service.cc:1511] Processing CreateTablet for tablet 34c6754d9a8e41ceb1fb416dcbbf0234 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:22:14.625104 21239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34c6754d9a8e41ceb1fb416dcbbf0234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.610764 21527 leader_election.cc:290] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:14.639644 21458 tablet_service.cc:1511] Processing CreateTablet for tablet 37a3c02685464cf1af805559b46c4356 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:22:14.641219 21458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37a3c02685464cf1af805559b46c4356. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.645789 21316 tablet_service.cc:1511] Processing CreateTablet for tablet 34c6754d9a8e41ceb1fb416dcbbf0234 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:22:14.647454 21316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34c6754d9a8e41ceb1fb416dcbbf0234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.648200 21521 tablet_bootstrap.cc:654] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:14.671200 21532 raft_consensus.cc:493] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:14.671851 21532 raft_consensus.cc:515] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:14.674721 21315 tablet_service.cc:1511] Processing CreateTablet for tablet e4905b7b4db2444484e9403fed337486 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:22:14.676143 21315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4905b7b4db2444484e9403fed337486. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.678792 21532 leader_election.cc:290] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:14.618794 21519 raft_consensus.cc:740] T 347c76112a0e430ca7d607278f20c811 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:14.719709 21519 consensus_queue.cc:260] T 347c76112a0e430ca7d607278f20c811 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:14.722844 21519 ts_tablet_manager.cc:1434] T 347c76112a0e430ca7d607278f20c811 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.157s	user 0.003s	sys 0.005s
I20260709 13:22:14.724192 21519 tablet_bootstrap.cc:492] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:14.730311 21238 tablet_service.cc:1511] Processing CreateTablet for tablet e4905b7b4db2444484e9403fed337486 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:22:14.731698 21238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4905b7b4db2444484e9403fed337486. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.739017 21520 tablet_bootstrap.cc:492] T d7c665892b494d9db72239758708b947 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:14.739575 21520 ts_tablet_manager.cc:1403] T d7c665892b494d9db72239758708b947 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.293s	user 0.014s	sys 0.005s
I20260709 13:22:14.748041 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a79b2b6d2c4326ab9e4332e0d7406f" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
W20260709 13:22:14.750501 21289 leader_election.cc:343] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:14.755535 21385 tablet_service.cc:1511] Processing CreateTablet for tablet f4e1cba0e20343aeb61c31c917dd1f50 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:22:14.757922 21520 raft_consensus.cc:359] T d7c665892b494d9db72239758708b947 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:14.758738 21520 raft_consensus.cc:385] T d7c665892b494d9db72239758708b947 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:14.759037 21520 raft_consensus.cc:740] T d7c665892b494d9db72239758708b947 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:14.759925 21520 consensus_queue.cc:260] T d7c665892b494d9db72239758708b947 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:14.765097 21518 tablet_bootstrap.cc:654] T 7772ca3f93fe431e8108b4e7ab98786e P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:14.770964 21520 ts_tablet_manager.cc:1434] T d7c665892b494d9db72239758708b947 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
I20260709 13:22:14.772059 21520 tablet_bootstrap.cc:492] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:14.776824 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7772ca3f93fe431e8108b4e7ab98786e" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
W20260709 13:22:14.779644 21134 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.11.63.66:38825, user_credentials={real_user=slave}} blocked reactor thread for 48984us
W20260709 13:22:14.791534 21439 leader_election.cc:343] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:14.822517 21309 tablet_service.cc:1511] Processing CreateTablet for tablet 9805db96a0ab4ac3a686eb8ec904fb24 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:22:14.824002 21309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9805db96a0ab4ac3a686eb8ec904fb24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.831130 21525 raft_consensus.cc:493] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:14.831712 21525 raft_consensus.cc:515] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:14.843334 21385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4e1cba0e20343aeb61c31c917dd1f50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.838325 21525 leader_election.cc:290] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:14.846474 21383 tablet_service.cc:1511] Processing CreateTablet for tablet 9805db96a0ab4ac3a686eb8ec904fb24 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:22:14.849699 21461 tablet_service.cc:1511] Processing CreateTablet for tablet f4e1cba0e20343aeb61c31c917dd1f50 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:22:14.851356 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7772ca3f93fe431e8108b4e7ab98786e" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:14.852180 21251 raft_consensus.cc:2468] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 0.
I20260709 13:22:14.848302 21383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9805db96a0ab4ac3a686eb8ec904fb24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.854537 21310 tablet_service.cc:1511] Processing CreateTablet for tablet 6898b14b98d944278cfd7cb32d964074 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:22:14.855839 21310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6898b14b98d944278cfd7cb32d964074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.856345 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "347c76112a0e430ca7d607278f20c811" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:14.857014 21474 raft_consensus.cc:2468] T 347c76112a0e430ca7d607278f20c811 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
I20260709 13:22:14.858778 21461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4e1cba0e20343aeb61c31c917dd1f50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.863345 21519 tablet_bootstrap.cc:654] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:14.867599 21363 leader_election.cc:304] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:14.876813 21525 raft_consensus.cc:2804] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:14.877211 21525 raft_consensus.cc:493] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:14.877586 21525 raft_consensus.cc:3060] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:14.878221 21462 tablet_service.cc:1511] Processing CreateTablet for tablet e4905b7b4db2444484e9403fed337486 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:22:14.879635 21462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4905b7b4db2444484e9403fed337486. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.881224 21384 tablet_service.cc:1511] Processing CreateTablet for tablet 6898b14b98d944278cfd7cb32d964074 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:22:14.882606 21384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6898b14b98d944278cfd7cb32d964074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.885365 21525 raft_consensus.cc:515] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:14.890204 21382 tablet_service.cc:1511] Processing CreateTablet for tablet 34c6754d9a8e41ceb1fb416dcbbf0234 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:22:14.891120 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a79b2b6d2c4326ab9e4332e0d7406f" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:14.891608 21382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34c6754d9a8e41ceb1fb416dcbbf0234. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.893103 21525 leader_election.cc:290] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:14.895610 21239 tablet_service.cc:1511] Processing CreateTablet for tablet f4e1cba0e20343aeb61c31c917dd1f50 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:22:14.896867 21239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4e1cba0e20343aeb61c31c917dd1f50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.898008 21437 leader_election.cc:304] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:14.899183 21527 raft_consensus.cc:2804] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:14.899708 21527 raft_consensus.cc:493] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:14.900147 21527 raft_consensus.cc:3060] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:14.909226 21456 tablet_service.cc:1511] Processing CreateTablet for tablet 9805db96a0ab4ac3a686eb8ec904fb24 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:22:14.913328 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "347c76112a0e430ca7d607278f20c811" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:14.913976 21474 raft_consensus.cc:3060] T 347c76112a0e430ca7d607278f20c811 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:14.914073 21473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7772ca3f93fe431e8108b4e7ab98786e" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:14.924129 21464 tablet_service.cc:1511] Processing CreateTablet for tablet 450b7111fe014a51b6e801869913ec52 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:22:14.944262 21521 tablet_bootstrap.cc:492] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:14.934127 21315 tablet_service.cc:1511] Processing CreateTablet for tablet 450b7111fe014a51b6e801869913ec52 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:22:14.946802 21521 ts_tablet_manager.cc:1403] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.466s	user 0.021s	sys 0.008s
I20260709 13:22:14.947631 21315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 450b7111fe014a51b6e801869913ec52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.937870 21388 tablet_service.cc:1511] Processing CreateTablet for tablet 73fe3abd2b2647d384d0e147d1932c0a (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:22:14.948916 21388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73fe3abd2b2647d384d0e147d1932c0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.930239 21527 raft_consensus.cc:515] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:14.951390 21459 tablet_service.cc:1511] Processing CreateTablet for tablet 73fe3abd2b2647d384d0e147d1932c0a (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:22:14.953708 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7772ca3f93fe431e8108b4e7ab98786e" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:14.954340 21520 tablet_bootstrap.cc:654] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:14.928936 21235 tablet_service.cc:1511] Processing CreateTablet for tablet 450b7111fe014a51b6e801869913ec52 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
W20260709 13:22:14.955268 21439 leader_election.cc:343] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:14.953939 21473 raft_consensus.cc:2393] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec879c0528c44465916efc6c33dde636 in current term 1: Already voted for candidate 43cd181efe6b4b2fb79fd3e997a401ea in this term.
I20260709 13:22:14.955864 21235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 450b7111fe014a51b6e801869913ec52. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:22:14.926927 21291 leader_election.cc:343] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:14.943853 21518 tablet_bootstrap.cc:492] T 7772ca3f93fe431e8108b4e7ab98786e P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:14.958086 21518 ts_tablet_manager.cc:1403] T 7772ca3f93fe431e8108b4e7ab98786e P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.413s	user 0.019s	sys 0.006s
I20260709 13:22:14.958995 21527 leader_election.cc:290] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:14.960546 21530 raft_consensus.cc:493] T d7c665892b494d9db72239758708b947 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:14.961124 21530 raft_consensus.cc:515] T d7c665892b494d9db72239758708b947 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:14.961297 21518 raft_consensus.cc:359] T 7772ca3f93fe431e8108b4e7ab98786e P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:14.962025 21518 raft_consensus.cc:385] T 7772ca3f93fe431e8108b4e7ab98786e P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:14.962347 21518 raft_consensus.cc:740] T 7772ca3f93fe431e8108b4e7ab98786e P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:14.962267 21521 raft_consensus.cc:359] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:14.963565 21521 raft_consensus.cc:385] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:14.963517 21518 consensus_queue.cc:260] T 7772ca3f93fe431e8108b4e7ab98786e P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:14.964287 21521 raft_consensus.cc:740] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:14.964661 21530 leader_election.cc:290] T d7c665892b494d9db72239758708b947 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:14.965476 21473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7c665892b494d9db72239758708b947" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:14.965669 21521 consensus_queue.cc:260] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
W20260709 13:22:14.966858 21289 leader_election.cc:343] T d7c665892b494d9db72239758708b947 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:14.935211 21456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9805db96a0ab4ac3a686eb8ec904fb24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.976621 21518 ts_tablet_manager.cc:1434] T 7772ca3f93fe431e8108b4e7ab98786e P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 13:22:14.977412 21238 tablet_service.cc:1511] Processing CreateTablet for tablet 73fe3abd2b2647d384d0e147d1932c0a (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:22:14.978839 21238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73fe3abd2b2647d384d0e147d1932c0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.980854 21309 tablet_service.cc:1511] Processing CreateTablet for tablet cdc82236ff024407ae854c96dd8c89ec (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:22:14.939039 21241 tablet_service.cc:1511] Processing CreateTablet for tablet 6898b14b98d944278cfd7cb32d964074 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:22:14.982172 21309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdc82236ff024407ae854c96dd8c89ec. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:22:14.984020 21196 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:14.974069 21291 leader_election.cc:304] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375
W20260709 13:22:14.986660 21291 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:43181, user_credentials={real_user=slave}} blocked reactor thread for 60713.8us
I20260709 13:22:14.988282 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7c665892b494d9db72239758708b947" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:14.989670 21530 raft_consensus.cc:2749] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:14.982173 21518 tablet_bootstrap.cc:492] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
W20260709 13:22:14.997834 21291 leader_election.cc:343] T d7c665892b494d9db72239758708b947 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:14.998476 21291 leader_election.cc:304] T d7c665892b494d9db72239758708b947 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:14.999459 21241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6898b14b98d944278cfd7cb32d964074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:14.999392 21530 raft_consensus.cc:2749] T d7c665892b494d9db72239758708b947 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:15.004751 21474 raft_consensus.cc:2468] T 347c76112a0e430ca7d607278f20c811 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:15.005718 21387 tablet_service.cc:1511] Processing CreateTablet for tablet cdc82236ff024407ae854c96dd8c89ec (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:22:15.006661 21521 ts_tablet_manager.cc:1434] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.060s	user 0.010s	sys 0.000s
I20260709 13:22:15.007195 21387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdc82236ff024407ae854c96dd8c89ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.007896 21521 tablet_bootstrap.cc:492] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:15.015295 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7772ca3f93fe431e8108b4e7ab98786e" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:15.015960 21251 raft_consensus.cc:3060] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:15.026837 21459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73fe3abd2b2647d384d0e147d1932c0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.027771 21240 tablet_service.cc:1511] Processing CreateTablet for tablet 8f2a6063e86146e8ba5f721d593edf0a (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:22:15.029112 21240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f2a6063e86146e8ba5f721d593edf0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.031222 21518 tablet_bootstrap.cc:654] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:15.033627 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7772ca3f93fe431e8108b4e7ab98786e" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:15.034337 21400 raft_consensus.cc:2468] T 7772ca3f93fe431e8108b4e7ab98786e P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:15.035768 21216 leader_election.cc:304] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:15.037186 21363 leader_election.cc:304] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:15.038158 21525 raft_consensus.cc:2804] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:15.039987 21461 tablet_service.cc:1511] Processing CreateTablet for tablet cdc82236ff024407ae854c96dd8c89ec (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:22:15.041275 21464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 450b7111fe014a51b6e801869913ec52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.042124 21461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdc82236ff024407ae854c96dd8c89ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.046836 21250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "347c76112a0e430ca7d607278f20c811" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:15.048516 21249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e532ca31c67547998ba2fb09244573b3" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:15.050182 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "347c76112a0e430ca7d607278f20c811" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
W20260709 13:22:15.055003 21363 leader_election.cc:343] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:15.055509 21326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e532ca31c67547998ba2fb09244573b3" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
W20260709 13:22:15.056098 21363 leader_election.cc:343] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:15.056855 21363 leader_election.cc:343] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:15.048861 21520 tablet_bootstrap.cc:492] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:15.059566 21310 tablet_service.cc:1511] Processing CreateTablet for tablet 8f2a6063e86146e8ba5f721d593edf0a (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:22:15.060303 21387 tablet_service.cc:1511] Processing CreateTablet for tablet 8f2a6063e86146e8ba5f721d593edf0a (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:22:15.060978 21310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f2a6063e86146e8ba5f721d593edf0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.061967 21387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f2a6063e86146e8ba5f721d593edf0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.062793 21251 raft_consensus.cc:2468] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:15.063339 21532 raft_consensus.cc:2764] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:22:15.069861 21520 ts_tablet_manager.cc:1403] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.298s	user 0.014s	sys 0.003s
I20260709 13:22:15.082489 21455 tablet_service.cc:1511] Processing CreateTablet for tablet 1ddeabcec9b64fc68b40c70df90b80ef (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:22:15.084501 21455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ddeabcec9b64fc68b40c70df90b80ef. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:22:15.093837 21364 leader_election.cc:343] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:15.094385 21364 leader_election.cc:304] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:15.094750 21437 leader_election.cc:304] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:15.079020 21525 raft_consensus.cc:697] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Leader. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:15.103775 21554 raft_consensus.cc:2749] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:15.108704 21521 tablet_bootstrap.cc:654] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:15.126125 21525 consensus_queue.cc:237] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:15.076673 21520 raft_consensus.cc:359] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.124724 21527 raft_consensus.cc:2804] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:15.126773 21519 tablet_bootstrap.cc:492] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:15.131713 21520 raft_consensus.cc:385] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:15.135829 21520 raft_consensus.cc:740] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:15.138650 21520 consensus_queue.cc:260] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.139164 21519 ts_tablet_manager.cc:1403] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.415s	user 0.024s	sys 0.033s
I20260709 13:22:15.142436 21519 raft_consensus.cc:359] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:15.143149 21519 raft_consensus.cc:385] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:15.143420 21519 raft_consensus.cc:740] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:15.137220 21239 tablet_service.cc:1511] Processing CreateTablet for tablet 1ddeabcec9b64fc68b40c70df90b80ef (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:22:15.147404 21519 consensus_queue.cc:260] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:15.153291 21527 raft_consensus.cc:697] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:15.157616 21518 tablet_bootstrap.cc:492] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:15.158700 21312 tablet_service.cc:1511] Processing CreateTablet for tablet 1ddeabcec9b64fc68b40c70df90b80ef (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
W20260709 13:22:15.132489 21364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:38825, user_credentials={real_user=slave}} blocked reactor thread for 54848.5us
I20260709 13:22:15.168881 21239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ddeabcec9b64fc68b40c70df90b80ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.171671 21520 ts_tablet_manager.cc:1434] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.101s	user 0.007s	sys 0.002s
I20260709 13:22:15.172078 21312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ddeabcec9b64fc68b40c70df90b80ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.172716 21238 tablet_service.cc:1511] Processing CreateTablet for tablet dcf55c037e6c48bc9d73f87a54b26f92 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:22:15.178429 21238 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcf55c037e6c48bc9d73f87a54b26f92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.178072 21309 tablet_service.cc:1511] Processing CreateTablet for tablet 033f24b2a37f421cbea89f5c5c61d0a3 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:22:15.182518 21519 ts_tablet_manager.cc:1434] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.006s
I20260709 13:22:15.174129 21518 ts_tablet_manager.cc:1403] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.192s	user 0.026s	sys 0.034s
I20260709 13:22:15.183653 21461 tablet_service.cc:1511] Processing CreateTablet for tablet 033f24b2a37f421cbea89f5c5c61d0a3 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:22:15.185022 21461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 033f24b2a37f421cbea89f5c5c61d0a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.161087 21527 consensus_queue.cc:237] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:15.187757 21383 tablet_service.cc:1511] Processing CreateTablet for tablet dcf55c037e6c48bc9d73f87a54b26f92 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:22:15.191143 21519 tablet_bootstrap.cc:492] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:15.191531 21383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcf55c037e6c48bc9d73f87a54b26f92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.194240 21309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 033f24b2a37f421cbea89f5c5c61d0a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.193867 21384 tablet_service.cc:1511] Processing CreateTablet for tablet 033f24b2a37f421cbea89f5c5c61d0a3 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:22:15.211504 21384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 033f24b2a37f421cbea89f5c5c61d0a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.218250 21464 tablet_service.cc:1511] Processing CreateTablet for tablet dcf55c037e6c48bc9d73f87a54b26f92 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:22:15.220340 21148 catalog_manager.cc:5697] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "74e2ff1a68ac4c849ee6397ebb011375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:15.226187 21315 tablet_service.cc:1511] Processing CreateTablet for tablet 08649134c0c94e7caa55fcc7513eec5d (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:22:15.213054 21240 tablet_service.cc:1511] Processing CreateTablet for tablet 08649134c0c94e7caa55fcc7513eec5d (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:22:15.228147 21240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08649134c0c94e7caa55fcc7513eec5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.228266 21315 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08649134c0c94e7caa55fcc7513eec5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.227991 21518 raft_consensus.cc:359] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:15.228883 21518 raft_consensus.cc:385] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:15.229203 21518 raft_consensus.cc:740] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:15.237030 21388 tablet_service.cc:1511] Processing CreateTablet for tablet 08649134c0c94e7caa55fcc7513eec5d (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:22:15.238547 21388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08649134c0c94e7caa55fcc7513eec5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.240703 21520 tablet_bootstrap.cc:492] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:15.253695 21464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcf55c037e6c48bc9d73f87a54b26f92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.267798 21235 tablet_service.cc:1511] Processing CreateTablet for tablet 7acd3d8dbc204f82b3f0f75ea8b68c57 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:22:15.269183 21235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7acd3d8dbc204f82b3f0f75ea8b68c57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.282052 21456 tablet_service.cc:1511] Processing CreateTablet for tablet 99478ce52e24419ea31804ba62f7a46a (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:22:15.283432 21456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99478ce52e24419ea31804ba62f7a46a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.286371 21455 tablet_service.cc:1511] Processing CreateTablet for tablet 7acd3d8dbc204f82b3f0f75ea8b68c57 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:22:15.287832 21455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7acd3d8dbc204f82b3f0f75ea8b68c57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.289659 21147 catalog_manager.cc:5697] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 1 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:15.292675 21521 tablet_bootstrap.cc:492] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:15.292599 21310 tablet_service.cc:1511] Processing CreateTablet for tablet 7acd3d8dbc204f82b3f0f75ea8b68c57 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:22:15.293406 21521 ts_tablet_manager.cc:1403] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.286s	user 0.026s	sys 0.020s
I20260709 13:22:15.293207 21241 tablet_service.cc:1511] Processing CreateTablet for tablet 99478ce52e24419ea31804ba62f7a46a (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:22:15.294709 21241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99478ce52e24419ea31804ba62f7a46a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.296662 21521 raft_consensus.cc:359] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.297587 21521 raft_consensus.cc:385] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:15.297956 21521 raft_consensus.cc:740] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:15.297885 21387 tablet_service.cc:1511] Processing CreateTablet for tablet 99478ce52e24419ea31804ba62f7a46a (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:22:15.298976 21521 consensus_queue.cc:260] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.278653 21518 consensus_queue.cc:260] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:15.304042 21316 tablet_service.cc:1511] Processing CreateTablet for tablet d2bf69be9e9240b5ac99adc49258432e (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:22:15.305672 21518 ts_tablet_manager.cc:1434] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.122s	user 0.008s	sys 0.001s
I20260709 13:22:15.307307 21459 tablet_service.cc:1511] Processing CreateTablet for tablet d2bf69be9e9240b5ac99adc49258432e (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:22:15.308362 21519 tablet_bootstrap.cc:654] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:15.308836 21459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2bf69be9e9240b5ac99adc49258432e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.305433 21316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2bf69be9e9240b5ac99adc49258432e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.317477 21385 tablet_service.cc:1511] Processing CreateTablet for tablet d2bf69be9e9240b5ac99adc49258432e (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:22:15.320921 21237 tablet_service.cc:1511] Processing CreateTablet for tablet a169fb713efc445fb1f0338c7c11203a (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:22:15.322206 21310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7acd3d8dbc204f82b3f0f75ea8b68c57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.322417 21237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a169fb713efc445fb1f0338c7c11203a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.329519 21385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2bf69be9e9240b5ac99adc49258432e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.331861 21382 tablet_service.cc:1511] Processing CreateTablet for tablet a169fb713efc445fb1f0338c7c11203a (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:22:15.333190 21382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a169fb713efc445fb1f0338c7c11203a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.339697 21462 tablet_service.cc:1511] Processing CreateTablet for tablet 1fff6b36d1984d38b6f791e0ac15f56b (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:22:15.341146 21462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fff6b36d1984d38b6f791e0ac15f56b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.343734 21520 tablet_bootstrap.cc:654] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:15.350097 21387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99478ce52e24419ea31804ba62f7a46a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.358068 21518 tablet_bootstrap.cc:492] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:15.358328 21312 tablet_service.cc:1511] Processing CreateTablet for tablet a169fb713efc445fb1f0338c7c11203a (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:22:15.359661 21312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a169fb713efc445fb1f0338c7c11203a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.365422 21521 ts_tablet_manager.cc:1434] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.072s	user 0.007s	sys 0.000s
I20260709 13:22:15.379340 21521 tablet_bootstrap.cc:492] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:15.382066 21311 tablet_service.cc:1511] Processing CreateTablet for tablet 1fff6b36d1984d38b6f791e0ac15f56b (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:22:15.384012 21311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fff6b36d1984d38b6f791e0ac15f56b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.385782 21239 tablet_service.cc:1511] Processing CreateTablet for tablet 1fff6b36d1984d38b6f791e0ac15f56b (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:22:15.387770 21239 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fff6b36d1984d38b6f791e0ac15f56b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.392197 21527 raft_consensus.cc:493] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:15.392742 21527 raft_consensus.cc:515] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:15.395082 21527 leader_election.cc:290] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:15.395481 21518 tablet_bootstrap.cc:654] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:15.398102 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f301fe3cc72b4e439644a7d1617921aa" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
W20260709 13:22:15.399595 21437 leader_election.cc:343] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:15.424759 21532 raft_consensus.cc:493] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:15.425381 21532 raft_consensus.cc:515] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:15.432579 21532 leader_election.cc:290] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:15.433481 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff163c4709bd4437bd70e20440389d62" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:15.437806 21383 tablet_service.cc:1511] Processing CreateTablet for tablet d37dadd42cd04be5a0e0b339df630a04 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
W20260709 13:22:15.438273 21214 leader_election.cc:343] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:15.439239 21383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d37dadd42cd04be5a0e0b339df630a04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.439963 21519 tablet_bootstrap.cc:492] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:15.440490 21519 ts_tablet_manager.cc:1403] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.250s	user 0.006s	sys 0.030s
I20260709 13:22:15.453208 21520 tablet_bootstrap.cc:492] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:15.452947 21459 tablet_service.cc:1511] Processing CreateTablet for tablet d37dadd42cd04be5a0e0b339df630a04 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:22:15.454030 21520 ts_tablet_manager.cc:1403] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.232s	user 0.013s	sys 0.005s
I20260709 13:22:15.454519 21459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d37dadd42cd04be5a0e0b339df630a04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.455950 21519 raft_consensus.cc:359] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:15.456736 21519 raft_consensus.cc:385] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:15.457032 21519 raft_consensus.cc:740] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:15.460562 21237 tablet_service.cc:1511] Processing CreateTablet for tablet d37dadd42cd04be5a0e0b339df630a04 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:22:15.461982 21237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d37dadd42cd04be5a0e0b339df630a04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.462605 21519 consensus_queue.cc:260] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:15.467736 21532 raft_consensus.cc:493] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:15.468335 21532 raft_consensus.cc:515] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.468999 21521 tablet_bootstrap.cc:654] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:15.470625 21532 leader_election.cc:290] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:15.472978 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e532ca31c67547998ba2fb09244573b3" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:15.481711 21400 raft_consensus.cc:2468] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:15.486349 21216 leader_election.cc:304] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:15.487339 21532 raft_consensus.cc:2804] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:15.487767 21532 raft_consensus.cc:493] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:15.488189 21532 raft_consensus.cc:3060] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:15.490278 21387 tablet_service.cc:1511] Processing CreateTablet for tablet 665861bf917f4d42aa1252c374a656ea (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:22:15.491676 21387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 665861bf917f4d42aa1252c374a656ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.494451 21241 tablet_service.cc:1511] Processing CreateTablet for tablet 3449b9c781a847b9ba63813b577eef81 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:22:15.495970 21241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3449b9c781a847b9ba63813b577eef81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.499794 21525 raft_consensus.cc:493] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:15.500434 21525 raft_consensus.cc:515] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:15.500922 21326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f301fe3cc72b4e439644a7d1617921aa" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:15.470631 21520 raft_consensus.cc:359] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.502600 21520 raft_consensus.cc:385] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:22:15.502709 21438 leader_election.cc:343] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:15.502959 21520 raft_consensus.cc:740] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:15.503341 21438 leader_election.cc:304] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:15.503891 21520 consensus_queue.cc:260] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.504815 21527 raft_consensus.cc:2749] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:15.505259 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7c665892b494d9db72239758708b947" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:15.506606 21326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7c665892b494d9db72239758708b947" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:15.507305 21326 raft_consensus.cc:2468] T d7c665892b494d9db72239758708b947 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
I20260709 13:22:15.506825 21519 ts_tablet_manager.cc:1434] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.066s	user 0.006s	sys 0.001s
I20260709 13:22:15.511293 21384 tablet_service.cc:1511] Processing CreateTablet for tablet 3449b9c781a847b9ba63813b577eef81 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:22:15.511861 21519 tablet_bootstrap.cc:492] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:15.512826 21384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3449b9c781a847b9ba63813b577eef81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.515379 21364 leader_election.cc:304] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:15.515295 21310 tablet_service.cc:1511] Processing CreateTablet for tablet 665861bf917f4d42aa1252c374a656ea (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:22:15.516513 21556 raft_consensus.cc:2804] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:15.516729 21310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 665861bf917f4d42aa1252c374a656ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.516988 21556 raft_consensus.cc:493] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:15.517469 21556 raft_consensus.cc:3060] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:15.519022 21520 ts_tablet_manager.cc:1434] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.065s	user 0.007s	sys 0.001s
I20260709 13:22:15.520468 21455 tablet_service.cc:1511] Processing CreateTablet for tablet 665861bf917f4d42aa1252c374a656ea (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:22:15.520602 21240 tablet_service.cc:1511] Processing CreateTablet for tablet 280cfe3edb3f490c8de46d0c80f6da5a (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:22:15.521831 21455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 665861bf917f4d42aa1252c374a656ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.522223 21240 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 280cfe3edb3f490c8de46d0c80f6da5a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:22:15.523115 21363 leader_election.cc:343] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:15.525056 21520 tablet_bootstrap.cc:492] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:15.526321 21462 tablet_service.cc:1511] Processing CreateTablet for tablet 280cfe3edb3f490c8de46d0c80f6da5a (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:22:15.527745 21462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 280cfe3edb3f490c8de46d0c80f6da5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.529636 21316 tablet_service.cc:1511] Processing CreateTablet for tablet 3449b9c781a847b9ba63813b577eef81 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:22:15.529992 21311 tablet_service.cc:1511] Processing CreateTablet for tablet 280cfe3edb3f490c8de46d0c80f6da5a (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:22:15.531234 21316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3449b9c781a847b9ba63813b577eef81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.530969 21521 tablet_bootstrap.cc:492] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:15.531678 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e532ca31c67547998ba2fb09244573b3" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:15.532577 21311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 280cfe3edb3f490c8de46d0c80f6da5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.532667 21325 raft_consensus.cc:2468] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:15.535189 21554 consensus_queue.cc:1048] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:15.536479 21521 ts_tablet_manager.cc:1403] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.162s	user 0.014s	sys 0.004s
I20260709 13:22:15.542166 21525 leader_election.cc:290] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:15.546177 21519 tablet_bootstrap.cc:654] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:15.538516 21326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff163c4709bd4437bd70e20440389d62" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:15.552635 21521 raft_consensus.cc:359] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.556591 21388 tablet_service.cc:1511] Processing CreateTablet for tablet 2d5ddbaefa064931963bc1e27ac99903 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:22:15.560678 21521 raft_consensus.cc:385] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:15.561319 21388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d5ddbaefa064931963bc1e27ac99903. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.561474 21521 raft_consensus.cc:740] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:15.566329 21518 tablet_bootstrap.cc:492] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:15.566996 21518 ts_tablet_manager.cc:1403] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.209s	user 0.023s	sys 0.009s
I20260709 13:22:15.563015 21521 consensus_queue.cc:260] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.573956 21521 ts_tablet_manager.cc:1434] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.001s
I20260709 13:22:15.575017 21235 tablet_service.cc:1511] Processing CreateTablet for tablet 2d5ddbaefa064931963bc1e27ac99903 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:22:15.580806 21387 tablet_service.cc:1511] Processing CreateTablet for tablet 3f3bf699b02c46c78d330dc7f9d3314b (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:22:15.582297 21387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f3bf699b02c46c78d330dc7f9d3314b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:22:15.582839 21215 leader_election.cc:343] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:15.583341 21215 leader_election.cc:304] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec879c0528c44465916efc6c33dde636; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:15.552925 21532 raft_consensus.cc:515] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.586066 21574 raft_consensus.cc:2749] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:15.586683 21459 tablet_service.cc:1511] Processing CreateTablet for tablet 2d5ddbaefa064931963bc1e27ac99903 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:22:15.588265 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e532ca31c67547998ba2fb09244573b3" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:15.588887 21400 raft_consensus.cc:3060] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:15.593789 21532 leader_election.cc:290] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Requested vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:15.593972 21520 tablet_bootstrap.cc:654] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:15.594054 21312 tablet_service.cc:1511] Processing CreateTablet for tablet 3f3bf699b02c46c78d330dc7f9d3314b (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:22:15.595595 21312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f3bf699b02c46c78d330dc7f9d3314b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.588312 21459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d5ddbaefa064931963bc1e27ac99903. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.597410 21400 raft_consensus.cc:2468] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:15.601675 21216 leader_election.cc:304] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:15.603499 21532 raft_consensus.cc:2804] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:15.603976 21532 raft_consensus.cc:697] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:15.604770 21532 consensus_queue.cc:237] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.607543 21518 raft_consensus.cc:359] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.608282 21518 raft_consensus.cc:385] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:15.608544 21518 raft_consensus.cc:740] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:15.601851 21556 raft_consensus.cc:515] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:15.615221 21326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e532ca31c67547998ba2fb09244573b3" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:15.615924 21326 raft_consensus.cc:3060] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:15.618391 21456 tablet_service.cc:1511] Processing CreateTablet for tablet 3f3bf699b02c46c78d330dc7f9d3314b (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:22:15.619912 21456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f3bf699b02c46c78d330dc7f9d3314b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.620699 21519 tablet_bootstrap.cc:492] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:15.621766 21519 ts_tablet_manager.cc:1403] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.110s	user 0.016s	sys 0.004s
I20260709 13:22:15.637344 21519 raft_consensus.cc:359] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.638810 21519 raft_consensus.cc:385] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:15.639428 21519 raft_consensus.cc:740] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:15.652088 21235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d5ddbaefa064931963bc1e27ac99903. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.652299 21326 raft_consensus.cc:2468] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:15.652823 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7c665892b494d9db72239758708b947" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:15.653776 21325 raft_consensus.cc:3060] T d7c665892b494d9db72239758708b947 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:15.609385 21518 consensus_queue.cc:260] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.662205 21518 ts_tablet_manager.cc:1434] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.095s	user 0.008s	sys 0.000s
I20260709 13:22:15.663290 21518 tablet_bootstrap.cc:492] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:15.664245 21556 leader_election.cc:290] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Requested vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:15.665884 21473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7c665892b494d9db72239758708b947" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
W20260709 13:22:15.667606 21363 leader_election.cc:343] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:15.668188 21574 raft_consensus.cc:493] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:15.661441 21521 tablet_bootstrap.cc:492] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:15.690083 21241 tablet_service.cc:1511] Processing CreateTablet for tablet aa2f735fa7d14b1e876c094ffbfb2c9d (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:22:15.691428 21241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa2f735fa7d14b1e876c094ffbfb2c9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.684515 21574 raft_consensus.cc:515] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.695628 21574 leader_election.cc:290] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:15.698055 21326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228f7ae94b2848ecb12f93717756ba8b" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:15.699012 21326 raft_consensus.cc:2468] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:15.700120 21518 tablet_bootstrap.cc:654] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:15.700762 21215 leader_election.cc:304] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:15.703477 21520 tablet_bootstrap.cc:492] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:15.704025 21520 ts_tablet_manager.cc:1403] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.179s	user 0.014s	sys 0.002s
W20260709 13:22:15.706244 21363 consensus_peers.cc:597] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:15.707114 21520 raft_consensus.cc:359] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.707865 21520 raft_consensus.cc:385] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:15.708143 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228f7ae94b2848ecb12f93717756ba8b" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
W20260709 13:22:15.710367 21216 leader_election.cc:343] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:15.718294 21574 raft_consensus.cc:2804] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:15.708145 21520 raft_consensus.cc:740] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:15.640360 21519 consensus_queue.cc:260] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.731173 21387 tablet_service.cc:1511] Processing CreateTablet for tablet aa2f735fa7d14b1e876c094ffbfb2c9d (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:22:15.731612 21527 consensus_queue.cc:1048] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:15.722976 21574 raft_consensus.cc:493] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:15.745410 21574 raft_consensus.cc:3060] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:15.734596 21316 tablet_service.cc:1511] Processing CreateTablet for tablet aa2f735fa7d14b1e876c094ffbfb2c9d (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:22:15.736220 21455 tablet_service.cc:1511] Processing CreateTablet for tablet 05b6856cbfc84013bdfa6979b0366468 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:22:15.754503 21455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05b6856cbfc84013bdfa6979b0366468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.754632 21316 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa2f735fa7d14b1e876c094ffbfb2c9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.726781 21325 raft_consensus.cc:2468] T d7c665892b494d9db72239758708b947 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:15.738106 21241 tablet_service.cc:1511] Processing CreateTablet for tablet 05b6856cbfc84013bdfa6979b0366468 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:22:15.756716 21241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05b6856cbfc84013bdfa6979b0366468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.742009 21387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa2f735fa7d14b1e876c094ffbfb2c9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.757618 21364 leader_election.cc:304] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 43cd181efe6b4b2fb79fd3e997a401ea
I20260709 13:22:15.758206 21519 ts_tablet_manager.cc:1434] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.135s	user 0.017s	sys 0.000s
I20260709 13:22:15.759281 21519 tablet_bootstrap.cc:492] T d7c665892b494d9db72239758708b947 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:15.759239 21556 raft_consensus.cc:2804] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:15.761962 21556 raft_consensus.cc:697] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Leader. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:15.764760 21556 consensus_queue.cc:237] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:15.769233 21311 tablet_service.cc:1511] Processing CreateTablet for tablet 05b6856cbfc84013bdfa6979b0366468 (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:22:15.770792 21311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05b6856cbfc84013bdfa6979b0366468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.770417 21383 tablet_service.cc:1511] Processing CreateTablet for tablet 0413c4b049a8412b9b3b8a36bb5d67fd (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:22:15.772519 21383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0413c4b049a8412b9b3b8a36bb5d67fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.776029 21456 tablet_service.cc:1511] Processing CreateTablet for tablet 0413c4b049a8412b9b3b8a36bb5d67fd (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:22:15.777375 21456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0413c4b049a8412b9b3b8a36bb5d67fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.780405 21574 raft_consensus.cc:515] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.733721 21520 consensus_queue.cc:260] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.791641 21521 tablet_bootstrap.cc:654] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:15.781251 21235 tablet_service.cc:1511] Processing CreateTablet for tablet 0413c4b049a8412b9b3b8a36bb5d67fd (DEFAULT_TABLE table=test-workload [id=80e229d91f7f4c329a408bdb0c349868]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:22:15.793190 21235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0413c4b049a8412b9b3b8a36bb5d67fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:15.800591 21527 raft_consensus.cc:493] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:15.801508 21527 raft_consensus.cc:515] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:15.802634 21520 ts_tablet_manager.cc:1434] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.098s	user 0.004s	sys 0.004s
I20260709 13:22:15.804984 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff163c4709bd4437bd70e20440389d62" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:15.806061 21520 tablet_bootstrap.cc:492] T ff163c4709bd4437bd70e20440389d62 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:15.809978 21400 raft_consensus.cc:3060] T 7772ca3f93fe431e8108b4e7ab98786e P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:15.813498 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff163c4709bd4437bd70e20440389d62" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:15.814441 21248 raft_consensus.cc:2468] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 0.
I20260709 13:22:15.816802 21437 leader_election.cc:304] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 
W20260709 13:22:15.818821 21438 leader_election.cc:343] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:15.824097 21525 raft_consensus.cc:493] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:15.824823 21525 raft_consensus.cc:515] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.825989 21519 tablet_bootstrap.cc:654] T d7c665892b494d9db72239758708b947 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:15.827761 21525 leader_election.cc:290] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:15.829747 21527 leader_election.cc:290] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:15.834254 21581 raft_consensus.cc:2804] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:15.834785 21581 raft_consensus.cc:493] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:15.835144 21581 raft_consensus.cc:3060] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:15.835207 21326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228f7ae94b2848ecb12f93717756ba8b" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:15.835541 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a79b2b6d2c4326ab9e4332e0d7406f" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:15.846007 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a79b2b6d2c4326ab9e4332e0d7406f" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:15.847988 21325 raft_consensus.cc:2468] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
I20260709 13:22:15.849941 21364 leader_election.cc:304] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:15.850929 21521 tablet_bootstrap.cc:492] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:15.851292 21525 raft_consensus.cc:2804] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:15.850811 21148 catalog_manager.cc:5697] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:15.851945 21525 raft_consensus.cc:493] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:15.852185 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228f7ae94b2848ecb12f93717756ba8b" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:15.852531 21525 raft_consensus.cc:3060] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:15.842362 21326 raft_consensus.cc:3060] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:15.853766 21216 leader_election.cc:343] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:15.851552 21521 ts_tablet_manager.cc:1403] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.190s	user 0.011s	sys 0.002s
I20260709 13:22:15.858073 21527 consensus_queue.cc:1048] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:22:15.864140 21518 tablet_bootstrap.cc:492] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:15.864821 21518 ts_tablet_manager.cc:1403] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.202s	user 0.033s	sys 0.045s
I20260709 13:22:15.843158 21574 leader_election.cc:290] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Requested vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:15.866132 21474 raft_consensus.cc:2468] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
I20260709 13:22:15.879657 21520 tablet_bootstrap.cc:654] T ff163c4709bd4437bd70e20440389d62 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:15.880620 21581 raft_consensus.cc:515] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:15.884616 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff163c4709bd4437bd70e20440389d62" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:15.885224 21248 raft_consensus.cc:3060] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:15.892638 21248 raft_consensus.cc:2468] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:15.893837 21326 raft_consensus.cc:2468] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:15.894203 21437 leader_election.cc:304] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:15.895458 21585 raft_consensus.cc:2804] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:15.896044 21585 raft_consensus.cc:697] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:15.897043 21585 consensus_queue.cc:237] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:15.895473 21215 leader_election.cc:304] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:15.899995 21574 raft_consensus.cc:2804] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:15.900496 21574 raft_consensus.cc:697] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:15.901499 21518 raft_consensus.cc:359] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.901288 21574 consensus_queue.cc:237] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.861357 21521 raft_consensus.cc:359] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.907398 21145 catalog_manager.cc:5697] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "74e2ff1a68ac4c849ee6397ebb011375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:15.873253 21525 raft_consensus.cc:515] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.902311 21518 raft_consensus.cc:385] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:15.903637 21521 raft_consensus.cc:385] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:15.916111 21518 raft_consensus.cc:740] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:15.921041 21521 raft_consensus.cc:740] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:15.928707 21326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff163c4709bd4437bd70e20440389d62" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
W20260709 13:22:15.929976 21438 leader_election.cc:343] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:15.923120 21581 leader_election.cc:290] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:15.931702 21530 raft_consensus.cc:493] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:15.936996 21530 raft_consensus.cc:515] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.946175 21521 consensus_queue.cc:260] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.952742 21521 ts_tablet_manager.cc:1434] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.094s	user 0.006s	sys 0.001s
I20260709 13:22:15.962931 21521 tablet_bootstrap.cc:492] T 347c76112a0e430ca7d607278f20c811 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:15.965649 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49acbb28158c4780add00f91662769da" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
W20260709 13:22:15.971236 21291 leader_election.cc:343] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:15.974035 21525 leader_election.cc:290] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:15.963771 21530 leader_election.cc:290] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:15.962564 21519 tablet_bootstrap.cc:492] T d7c665892b494d9db72239758708b947 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:15.975754 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a79b2b6d2c4326ab9e4332e0d7406f" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:15.977718 21326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a79b2b6d2c4326ab9e4332e0d7406f" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:15.979408 21474 raft_consensus.cc:3060] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:15.979501 21326 raft_consensus.cc:3060] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:15.950313 21518 consensus_queue.cc:260] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.981875 21518 ts_tablet_manager.cc:1434] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.117s	user 0.004s	sys 0.005s
I20260709 13:22:15.982867 21518 tablet_bootstrap.cc:492] T 49acbb28158c4780add00f91662769da P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:15.982769 21145 catalog_manager.cc:5697] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 1 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
W20260709 13:22:15.986845 21196 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:15.989954 21474 raft_consensus.cc:2468] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:15.991237 21326 raft_consensus.cc:2468] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:15.992010 21518 tablet_bootstrap.cc:654] T 49acbb28158c4780add00f91662769da P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:15.991629 21363 leader_election.cc:304] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:15.992848 21525 raft_consensus.cc:2804] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:15.993317 21525 raft_consensus.cc:697] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Leader. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:15.994144 21525 consensus_queue.cc:237] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:15.978981 21519 ts_tablet_manager.cc:1403] T d7c665892b494d9db72239758708b947 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.220s	user 0.008s	sys 0.033s
I20260709 13:22:15.996887 21148 catalog_manager.cc:5697] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:16.000402 21520 tablet_bootstrap.cc:492] T ff163c4709bd4437bd70e20440389d62 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:16.002802 21520 ts_tablet_manager.cc:1403] T ff163c4709bd4437bd70e20440389d62 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.197s	user 0.008s	sys 0.009s
I20260709 13:22:16.006323 21574 consensus_queue.cc:1048] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:16.006382 21519 raft_consensus.cc:359] T d7c665892b494d9db72239758708b947 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:16.007620 21519 raft_consensus.cc:385] T d7c665892b494d9db72239758708b947 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.007939 21519 raft_consensus.cc:740] T d7c665892b494d9db72239758708b947 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.008837 21519 consensus_queue.cc:260] T d7c665892b494d9db72239758708b947 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:16.023906 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49acbb28158c4780add00f91662769da" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:16.024516 21251 raft_consensus.cc:2468] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 0.
I20260709 13:22:16.028461 21521 tablet_bootstrap.cc:654] T 347c76112a0e430ca7d607278f20c811 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.029276 21519 ts_tablet_manager.cc:1434] T d7c665892b494d9db72239758708b947 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.002s
I20260709 13:22:16.026278 21289 leader_election.cc:304] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:16.033769 21530 raft_consensus.cc:2804] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:16.034199 21530 raft_consensus.cc:493] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:16.034585 21530 raft_consensus.cc:3060] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.038167 21574 consensus_queue.cc:1048] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:16.041028 21530 raft_consensus.cc:515] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.033344 21145 catalog_manager.cc:5697] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "74e2ff1a68ac4c849ee6397ebb011375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:16.056099 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49acbb28158c4780add00f91662769da" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:16.056958 21251 raft_consensus.cc:3060] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.057857 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49acbb28158c4780add00f91662769da" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:16.060633 21520 raft_consensus.cc:359] T ff163c4709bd4437bd70e20440389d62 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.061669 21520 raft_consensus.cc:385] T ff163c4709bd4437bd70e20440389d62 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.062042 21520 raft_consensus.cc:740] T ff163c4709bd4437bd70e20440389d62 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
W20260709 13:22:16.064596 21291 leader_election.cc:343] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.065052 21520 consensus_queue.cc:260] T ff163c4709bd4437bd70e20440389d62 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.065708 21530 leader_election.cc:290] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:16.066224 21581 raft_consensus.cc:493] T d7c665892b494d9db72239758708b947 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.066838 21581 raft_consensus.cc:515] T d7c665892b494d9db72239758708b947 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:16.069399 21519 tablet_bootstrap.cc:492] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:16.070938 21581 leader_election.cc:290] T d7c665892b494d9db72239758708b947 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:16.072975 21326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7c665892b494d9db72239758708b947" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:16.074030 21326 raft_consensus.cc:2393] T d7c665892b494d9db72239758708b947 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 43cd181efe6b4b2fb79fd3e997a401ea in current term 1: Already voted for candidate 74e2ff1a68ac4c849ee6397ebb011375 in this term.
I20260709 13:22:16.078385 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7c665892b494d9db72239758708b947" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:16.078663 21518 tablet_bootstrap.cc:492] T 49acbb28158c4780add00f91662769da P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:16.079977 21518 ts_tablet_manager.cc:1403] T 49acbb28158c4780add00f91662769da P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.097s	user 0.020s	sys 0.007s
I20260709 13:22:16.081837 21519 tablet_bootstrap.cc:654] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.082391 21439 leader_election.cc:304] T d7c665892b494d9db72239758708b947 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:16.083487 21518 raft_consensus.cc:359] T 49acbb28158c4780add00f91662769da P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.083925 21581 raft_consensus.cc:3060] T d7c665892b494d9db72239758708b947 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.084374 21518 raft_consensus.cc:385] T 49acbb28158c4780add00f91662769da P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.084743 21518 raft_consensus.cc:740] T 49acbb28158c4780add00f91662769da P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.095716 21519 tablet_bootstrap.cc:492] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:16.098699 21518 consensus_queue.cc:260] T 49acbb28158c4780add00f91662769da P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.101127 21519 ts_tablet_manager.cc:1403] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.005s
I20260709 13:22:16.103031 21520 ts_tablet_manager.cc:1434] T ff163c4709bd4437bd70e20440389d62 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.097s	user 0.019s	sys 0.021s
I20260709 13:22:16.104046 21521 tablet_bootstrap.cc:492] T 347c76112a0e430ca7d607278f20c811 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:16.104208 21520 tablet_bootstrap.cc:492] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:16.104607 21521 ts_tablet_manager.cc:1403] T 347c76112a0e430ca7d607278f20c811 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.143s	user 0.009s	sys 0.005s
I20260709 13:22:16.104681 21519 raft_consensus.cc:359] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.105588 21519 raft_consensus.cc:385] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.105890 21519 raft_consensus.cc:740] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.106719 21519 consensus_queue.cc:260] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.107823 21521 raft_consensus.cc:359] T 347c76112a0e430ca7d607278f20c811 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.108603 21521 raft_consensus.cc:385] T 347c76112a0e430ca7d607278f20c811 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.108906 21521 raft_consensus.cc:740] T 347c76112a0e430ca7d607278f20c811 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.109107 21519 ts_tablet_manager.cc:1434] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:22:16.109828 21521 consensus_queue.cc:260] T 347c76112a0e430ca7d607278f20c811 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.112041 21520 tablet_bootstrap.cc:654] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.112246 21525 raft_consensus.cc:493] T 49acbb28158c4780add00f91662769da P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.112826 21521 ts_tablet_manager.cc:1434] T 347c76112a0e430ca7d607278f20c811 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:22:16.112725 21525 raft_consensus.cc:515] T 49acbb28158c4780add00f91662769da P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.113953 21521 tablet_bootstrap.cc:492] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:16.115032 21525 leader_election.cc:290] T 49acbb28158c4780add00f91662769da P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:16.115675 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49acbb28158c4780add00f91662769da" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:16.119443 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49acbb28158c4780add00f91662769da" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:16.120217 21325 raft_consensus.cc:2393] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74e2ff1a68ac4c849ee6397ebb011375 in current term 1: Already voted for candidate 5109d6d6d9404ba39c53508a81fe9f1c in this term.
I20260709 13:22:16.101163 21518 ts_tablet_manager.cc:1434] T 49acbb28158c4780add00f91662769da P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:22:16.121251 21364 leader_election.cc:304] T 49acbb28158c4780add00f91662769da P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:16.121788 21518 tablet_bootstrap.cc:492] T 5957ec3bcdcf40838d52d01a2a65281f P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:16.122540 21525 raft_consensus.cc:3060] T 49acbb28158c4780add00f91662769da P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.124082 21520 tablet_bootstrap.cc:492] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:16.124600 21520 ts_tablet_manager.cc:1403] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.006s
I20260709 13:22:16.128914 21518 tablet_bootstrap.cc:654] T 5957ec3bcdcf40838d52d01a2a65281f P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.128938 21520 raft_consensus.cc:359] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.129891 21520 raft_consensus.cc:385] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.130234 21520 raft_consensus.cc:740] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.131168 21520 consensus_queue.cc:260] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.131723 21525 raft_consensus.cc:2749] T 49acbb28158c4780add00f91662769da P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:16.133922 21520 ts_tablet_manager.cc:1434] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 13:22:16.134305 21519 tablet_bootstrap.cc:492] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:16.136999 21520 tablet_bootstrap.cc:492] T 6b569ec3d17d412c8c9087446e64bb77 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:16.143913 21521 tablet_bootstrap.cc:654] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.144218 21520 tablet_bootstrap.cc:654] T 6b569ec3d17d412c8c9087446e64bb77 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.144412 21519 tablet_bootstrap.cc:654] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.146529 21518 tablet_bootstrap.cc:492] T 5957ec3bcdcf40838d52d01a2a65281f P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:16.147094 21518 ts_tablet_manager.cc:1403] T 5957ec3bcdcf40838d52d01a2a65281f P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.000s
I20260709 13:22:16.151724 21518 raft_consensus.cc:359] T 5957ec3bcdcf40838d52d01a2a65281f P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:16.152644 21518 raft_consensus.cc:385] T 5957ec3bcdcf40838d52d01a2a65281f P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.152958 21518 raft_consensus.cc:740] T 5957ec3bcdcf40838d52d01a2a65281f P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.155522 21520 tablet_bootstrap.cc:492] T 6b569ec3d17d412c8c9087446e64bb77 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:16.156129 21520 ts_tablet_manager.cc:1403] T 6b569ec3d17d412c8c9087446e64bb77 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.003s
I20260709 13:22:16.156122 21518 consensus_queue.cc:260] T 5957ec3bcdcf40838d52d01a2a65281f P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:16.159416 21520 raft_consensus.cc:359] T 6b569ec3d17d412c8c9087446e64bb77 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.159875 21518 ts_tablet_manager.cc:1434] T 5957ec3bcdcf40838d52d01a2a65281f P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 13:22:16.160115 21520 raft_consensus.cc:385] T 6b569ec3d17d412c8c9087446e64bb77 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.160382 21520 raft_consensus.cc:740] T 6b569ec3d17d412c8c9087446e64bb77 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.160841 21518 tablet_bootstrap.cc:492] T 6558bd3928cd424fa13c61523335998a P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:16.161134 21554 consensus_queue.cc:1048] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:16.163237 21520 consensus_queue.cc:260] T 6b569ec3d17d412c8c9087446e64bb77 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.165784 21520 ts_tablet_manager.cc:1434] T 6b569ec3d17d412c8c9087446e64bb77 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:22:16.166755 21520 tablet_bootstrap.cc:492] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:16.168063 21519 tablet_bootstrap.cc:492] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:16.168557 21519 ts_tablet_manager.cc:1403] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.034s	user 0.009s	sys 0.004s
I20260709 13:22:16.171399 21519 raft_consensus.cc:359] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.172143 21519 raft_consensus.cc:385] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.172416 21519 raft_consensus.cc:740] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.172538 21520 tablet_bootstrap.cc:654] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.173368 21519 consensus_queue.cc:260] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.175721 21519 ts_tablet_manager.cc:1434] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:22:16.179082 21518 tablet_bootstrap.cc:654] T 6558bd3928cd424fa13c61523335998a P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.179713 21581 raft_consensus.cc:2749] T d7c665892b494d9db72239758708b947 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:16.181226 21251 raft_consensus.cc:2468] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 1.
I20260709 13:22:16.183732 21519 tablet_bootstrap.cc:492] T 5957ec3bcdcf40838d52d01a2a65281f P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:16.188616 21556 consensus_queue.cc:1048] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:16.217482 21518 tablet_bootstrap.cc:492] T 6558bd3928cd424fa13c61523335998a P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:16.218021 21518 ts_tablet_manager.cc:1403] T 6558bd3928cd424fa13c61523335998a P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.057s	user 0.012s	sys 0.000s
I20260709 13:22:16.222744 21519 tablet_bootstrap.cc:654] T 5957ec3bcdcf40838d52d01a2a65281f P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.228881 21289 leader_election.cc:304] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:16.228819 21518 raft_consensus.cc:359] T 6558bd3928cd424fa13c61523335998a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.229904 21530 raft_consensus.cc:2804] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:16.230165 21518 raft_consensus.cc:385] T 6558bd3928cd424fa13c61523335998a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.230521 21530 raft_consensus.cc:697] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 LEADER]: Becoming Leader. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Running, Role: LEADER
I20260709 13:22:16.231730 21530 consensus_queue.cc:237] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.232700 21520 tablet_bootstrap.cc:492] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:16.233338 21520 ts_tablet_manager.cc:1403] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.067s	user 0.024s	sys 0.031s
I20260709 13:22:16.230557 21518 raft_consensus.cc:740] T 6558bd3928cd424fa13c61523335998a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.236304 21518 consensus_queue.cc:260] T 6558bd3928cd424fa13c61523335998a P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.236925 21520 raft_consensus.cc:359] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:16.237668 21520 raft_consensus.cc:385] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.237947 21520 raft_consensus.cc:740] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.239192 21521 tablet_bootstrap.cc:492] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:16.238576 21520 consensus_queue.cc:260] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:16.239787 21521 ts_tablet_manager.cc:1403] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.126s	user 0.032s	sys 0.033s
I20260709 13:22:16.241408 21520 ts_tablet_manager.cc:1434] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:22:16.242343 21520 tablet_bootstrap.cc:492] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:16.243353 21521 raft_consensus.cc:359] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.244153 21521 raft_consensus.cc:385] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.244506 21521 raft_consensus.cc:740] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.245323 21521 consensus_queue.cc:260] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.246388 21519 tablet_bootstrap.cc:492] T 5957ec3bcdcf40838d52d01a2a65281f P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:16.246878 21519 ts_tablet_manager.cc:1403] T 5957ec3bcdcf40838d52d01a2a65281f P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.063s	user 0.014s	sys 0.005s
I20260709 13:22:16.249342 21519 raft_consensus.cc:359] T 5957ec3bcdcf40838d52d01a2a65281f P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:16.250000 21519 raft_consensus.cc:385] T 5957ec3bcdcf40838d52d01a2a65281f P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.250260 21519 raft_consensus.cc:740] T 5957ec3bcdcf40838d52d01a2a65281f P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.250779 21521 ts_tablet_manager.cc:1434] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:22:16.250923 21519 consensus_queue.cc:260] T 5957ec3bcdcf40838d52d01a2a65281f P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:16.252014 21521 tablet_bootstrap.cc:492] T 6b569ec3d17d412c8c9087446e64bb77 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:16.252923 21519 ts_tablet_manager.cc:1434] T 5957ec3bcdcf40838d52d01a2a65281f P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:22:16.253989 21519 tablet_bootstrap.cc:492] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:16.258445 21518 ts_tablet_manager.cc:1434] T 6558bd3928cd424fa13c61523335998a P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.000s
I20260709 13:22:16.259842 21521 tablet_bootstrap.cc:654] T 6b569ec3d17d412c8c9087446e64bb77 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.261292 21520 tablet_bootstrap.cc:654] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.263943 21519 tablet_bootstrap.cc:654] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.264905 21518 tablet_bootstrap.cc:492] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:16.272306 21145 catalog_manager.cc:5697] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66). New cstate: current_term: 1 leader_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:16.276940 21556 raft_consensus.cc:493] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.283578 21556 raft_consensus.cc:515] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.288645 21518 tablet_bootstrap.cc:654] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.288803 21251 raft_consensus.cc:3060] T 347c76112a0e430ca7d607278f20c811 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.295392 21520 tablet_bootstrap.cc:492] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:16.296098 21520 ts_tablet_manager.cc:1403] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.054s	user 0.029s	sys 0.008s
I20260709 13:22:16.300561 21520 raft_consensus.cc:359] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.301525 21520 raft_consensus.cc:385] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.301898 21520 raft_consensus.cc:740] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.302959 21520 consensus_queue.cc:260] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.305326 21520 ts_tablet_manager.cc:1434] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.003s
I20260709 13:22:16.306313 21520 tablet_bootstrap.cc:492] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:16.306569 21556 leader_election.cc:290] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:16.307224 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228f7ae94b2848ecb12f93717756ba8b" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:16.308525 21326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228f7ae94b2848ecb12f93717756ba8b" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:16.309441 21326 raft_consensus.cc:2393] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74e2ff1a68ac4c849ee6397ebb011375 in current term 1: Already voted for candidate ec879c0528c44465916efc6c33dde636 in this term.
I20260709 13:22:16.311097 21364 leader_election.cc:304] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:16.312424 21556 raft_consensus.cc:3060] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.313191 21520 tablet_bootstrap.cc:654] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.314059 21530 raft_consensus.cc:493] T ff163c4709bd4437bd70e20440389d62 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.314580 21530 raft_consensus.cc:515] T ff163c4709bd4437bd70e20440389d62 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.316676 21530 leader_election.cc:290] T ff163c4709bd4437bd70e20440389d62 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:16.318445 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff163c4709bd4437bd70e20440389d62" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:16.319444 21248 raft_consensus.cc:2393] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5109d6d6d9404ba39c53508a81fe9f1c in current term 1: Already voted for candidate 43cd181efe6b4b2fb79fd3e997a401ea in this term.
I20260709 13:22:16.320901 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff163c4709bd4437bd70e20440389d62" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:16.321887 21581 raft_consensus.cc:493] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.322367 21581 raft_consensus.cc:515] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.323760 21556 raft_consensus.cc:2749] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:16.324010 21585 raft_consensus.cc:493] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.324453 21581 leader_election.cc:290] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:16.324528 21585 raft_consensus.cc:515] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.326900 21326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b569ec3d17d412c8c9087446e64bb77" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:16.327595 21326 raft_consensus.cc:2468] T 6b569ec3d17d412c8c9087446e64bb77 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 0.
I20260709 13:22:16.327837 21289 leader_election.cc:304] T ff163c4709bd4437bd70e20440389d62 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636
I20260709 13:22:16.328656 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6558bd3928cd424fa13c61523335998a" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:16.329066 21530 raft_consensus.cc:3060] T ff163c4709bd4437bd70e20440389d62 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.329159 21438 leader_election.cc:304] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 
I20260709 13:22:16.326939 21585 leader_election.cc:290] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:16.330638 21585 raft_consensus.cc:2804] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:16.331008 21585 raft_consensus.cc:493] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:16.331377 21585 raft_consensus.cc:3060] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.331264 21249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6558bd3928cd424fa13c61523335998a" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:16.331418 21521 tablet_bootstrap.cc:492] T 6b569ec3d17d412c8c9087446e64bb77 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:16.332401 21521 ts_tablet_manager.cc:1403] T 6b569ec3d17d412c8c9087446e64bb77 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.081s	user 0.013s	sys 0.014s
W20260709 13:22:16.333210 21437 leader_election.cc:343] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.334240 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b569ec3d17d412c8c9087446e64bb77" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:16.335315 21521 raft_consensus.cc:359] T 6b569ec3d17d412c8c9087446e64bb77 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
W20260709 13:22:16.335860 21437 leader_election.cc:343] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.329342 21399 raft_consensus.cc:2468] T 6558bd3928cd424fa13c61523335998a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 0.
I20260709 13:22:16.336959 21530 raft_consensus.cc:2749] T ff163c4709bd4437bd70e20440389d62 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:16.338325 21439 leader_election.cc:304] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: ec879c0528c44465916efc6c33dde636
I20260709 13:22:16.339440 21581 raft_consensus.cc:2804] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:16.335986 21521 raft_consensus.cc:385] T 6b569ec3d17d412c8c9087446e64bb77 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.339994 21581 raft_consensus.cc:493] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:16.340423 21581 raft_consensus.cc:3060] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.340354 21521 raft_consensus.cc:740] T 6b569ec3d17d412c8c9087446e64bb77 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.342024 21519 tablet_bootstrap.cc:492] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:16.342569 21519 ts_tablet_manager.cc:1403] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.089s	user 0.019s	sys 0.027s
I20260709 13:22:16.345041 21518 tablet_bootstrap.cc:492] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:16.345350 21519 raft_consensus.cc:359] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.345844 21518 ts_tablet_manager.cc:1403] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.081s	user 0.020s	sys 0.006s
I20260709 13:22:16.346159 21519 raft_consensus.cc:385] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.347609 21519 raft_consensus.cc:740] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.347744 21581 raft_consensus.cc:515] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.348335 21519 consensus_queue.cc:260] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.342059 21521 consensus_queue.cc:260] T 6b569ec3d17d412c8c9087446e64bb77 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.350688 21521 ts_tablet_manager.cc:1434] T 6b569ec3d17d412c8c9087446e64bb77 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:22:16.350999 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6558bd3928cd424fa13c61523335998a" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:16.351696 21521 tablet_bootstrap.cc:492] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:16.351580 21399 raft_consensus.cc:3060] T 6558bd3928cd424fa13c61523335998a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.358312 21521 tablet_bootstrap.cc:654] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.359737 21574 consensus_queue.cc:1048] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:16.360311 21581 leader_election.cc:290] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:16.363821 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6558bd3928cd424fa13c61523335998a" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
W20260709 13:22:16.366341 21437 leader_election.cc:343] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.375829 21519 ts_tablet_manager.cc:1434] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 13:22:16.359766 21399 raft_consensus.cc:2468] T 6558bd3928cd424fa13c61523335998a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:16.384435 21439 leader_election.cc:304] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: ec879c0528c44465916efc6c33dde636
I20260709 13:22:16.381381 21519 tablet_bootstrap.cc:492] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:16.349460 21585 raft_consensus.cc:515] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.385701 21621 raft_consensus.cc:2804] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:16.386389 21621 raft_consensus.cc:697] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:16.387247 21621 consensus_queue.cc:237] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.388825 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b569ec3d17d412c8c9087446e64bb77" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:16.389636 21248 raft_consensus.cc:3060] T 6b569ec3d17d412c8c9087446e64bb77 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.390815 21521 tablet_bootstrap.cc:492] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:16.391302 21521 ts_tablet_manager.cc:1403] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.040s	user 0.014s	sys 0.015s
I20260709 13:22:16.393622 21519 tablet_bootstrap.cc:654] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.394167 21521 raft_consensus.cc:359] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.394897 21521 raft_consensus.cc:385] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.395156 21521 raft_consensus.cc:740] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.395931 21521 consensus_queue.cc:260] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.398569 21521 ts_tablet_manager.cc:1434] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:22:16.399748 21521 tablet_bootstrap.cc:492] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:16.401758 21581 consensus_queue.cc:1048] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:16.404096 21574 raft_consensus.cc:493] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.404649 21574 raft_consensus.cc:515] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.406484 21519 tablet_bootstrap.cc:492] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:16.406839 21574 leader_election.cc:290] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:16.406976 21519 ts_tablet_manager.cc:1403] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.026s	user 0.006s	sys 0.011s
I20260709 13:22:16.409760 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6558bd3928cd424fa13c61523335998a" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:16.410033 21519 raft_consensus.cc:359] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:16.410772 21519 raft_consensus.cc:385] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.411055 21519 raft_consensus.cc:740] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.412143 21521 tablet_bootstrap.cc:654] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.411828 21519 consensus_queue.cc:260] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:16.414139 21519 ts_tablet_manager.cc:1434] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:22:16.415097 21519 tablet_bootstrap.cc:492] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:16.415236 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6558bd3928cd424fa13c61523335998a" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:16.416002 21399 raft_consensus.cc:2393] T 6558bd3928cd424fa13c61523335998a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec879c0528c44465916efc6c33dde636 in current term 1: Already voted for candidate 43cd181efe6b4b2fb79fd3e997a401ea in this term.
I20260709 13:22:16.417126 21216 leader_election.cc:304] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec879c0528c44465916efc6c33dde636; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:16.418291 21599 raft_consensus.cc:3060] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.419466 21530 raft_consensus.cc:493] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.419966 21530 raft_consensus.cc:515] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.422350 21530 leader_election.cc:290] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:16.422364 21519 tablet_bootstrap.cc:654] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.424927 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f301fe3cc72b4e439644a7d1617921aa" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:16.425617 21474 raft_consensus.cc:2468] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 0.
I20260709 13:22:16.426606 21518 raft_consensus.cc:359] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.428632 21289 leader_election.cc:304] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 
I20260709 13:22:16.431728 21518 raft_consensus.cc:385] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.429401 21250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f301fe3cc72b4e439644a7d1617921aa" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:16.435314 21520 tablet_bootstrap.cc:492] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:16.436035 21520 ts_tablet_manager.cc:1403] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.130s	user 0.024s	sys 0.021s
I20260709 13:22:16.437500 21530 raft_consensus.cc:2804] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:16.437685 21250 raft_consensus.cc:2468] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 0.
I20260709 13:22:16.438247 21530 raft_consensus.cc:493] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:16.438792 21530 raft_consensus.cc:3060] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.443418 21520 raft_consensus.cc:359] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.444180 21520 raft_consensus.cc:385] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.444464 21520 raft_consensus.cc:740] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.432202 21599 raft_consensus.cc:2749] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:16.445144 21520 consensus_queue.cc:260] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.447440 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b569ec3d17d412c8c9087446e64bb77" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:16.448294 21325 raft_consensus.cc:3060] T 6b569ec3d17d412c8c9087446e64bb77 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.449347 21520 ts_tablet_manager.cc:1434] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:22:16.450688 21520 tablet_bootstrap.cc:492] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:16.454406 21585 leader_election.cc:290] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:16.434011 21518 raft_consensus.cc:740] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.457027 21518 consensus_queue.cc:260] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.468461 21518 ts_tablet_manager.cc:1434] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.121s	user 0.005s	sys 0.004s
I20260709 13:22:16.466646 21599 consensus_queue.cc:1048] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:22:16.471866 21600 raft_consensus.cc:493] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.472417 21600 raft_consensus.cc:515] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:16.475509 21600 leader_election.cc:290] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:16.477332 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5957ec3bcdcf40838d52d01a2a65281f" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:16.478091 21400 raft_consensus.cc:2468] T 5957ec3bcdcf40838d52d01a2a65281f P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 0.
I20260709 13:22:16.477859 21530 raft_consensus.cc:515] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.478631 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5957ec3bcdcf40838d52d01a2a65281f" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:16.479964 21519 tablet_bootstrap.cc:492] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:16.480216 21518 tablet_bootstrap.cc:492] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:16.480439 21519 ts_tablet_manager.cc:1403] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.066s	user 0.013s	sys 0.001s
I20260709 13:22:16.468583 21325 raft_consensus.cc:2468] T 6b569ec3d17d412c8c9087446e64bb77 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:16.479504 21474 raft_consensus.cc:2468] T 5957ec3bcdcf40838d52d01a2a65281f P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 0.
I20260709 13:22:16.479527 21291 leader_election.cc:304] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:16.483517 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f301fe3cc72b4e439644a7d1617921aa" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:16.483815 21438 leader_election.cc:304] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 
I20260709 13:22:16.484140 21251 raft_consensus.cc:3060] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.488471 21521 tablet_bootstrap.cc:492] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:16.489182 21521 ts_tablet_manager.cc:1403] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.090s	user 0.032s	sys 0.023s
I20260709 13:22:16.492403 21251 raft_consensus.cc:2468] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 1.
I20260709 13:22:16.493425 21556 consensus_queue.cc:1048] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:16.494458 21289 leader_election.cc:304] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:16.490397 21600 raft_consensus.cc:2804] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:16.498293 21520 tablet_bootstrap.cc:654] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.488462 21518 tablet_bootstrap.cc:654] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.495332 21600 raft_consensus.cc:493] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:16.508206 21600 raft_consensus.cc:3060] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.512188 21556 consensus_queue.cc:1048] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:16.513377 21521 raft_consensus.cc:359] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.513722 21527 consensus_queue.cc:1048] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:16.514200 21521 raft_consensus.cc:385] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.498116 21621 raft_consensus.cc:2804] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:16.507469 21525 raft_consensus.cc:493] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.514559 21521 raft_consensus.cc:740] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.519004 21530 leader_election.cc:290] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:16.519670 21621 raft_consensus.cc:697] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:16.519963 21530 raft_consensus.cc:2804] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:16.520474 21525 raft_consensus.cc:515] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.522010 21521 consensus_queue.cc:260] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.520982 21248 raft_consensus.cc:2468] T 6b569ec3d17d412c8c9087446e64bb77 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:16.520507 21530 raft_consensus.cc:697] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 LEADER]: Becoming Leader. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Running, Role: LEADER
I20260709 13:22:16.547338 21530 consensus_queue.cc:237] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.549081 21527 raft_consensus.cc:493] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.549768 21527 raft_consensus.cc:515] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:16.520903 21621 consensus_queue.cc:237] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.553495 21148 catalog_manager.cc:5697] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 1 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:16.556512 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dbdec3b8bc40898dcd64ba19a4ec5d" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:16.529716 21473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f301fe3cc72b4e439644a7d1617921aa" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:16.559811 21473 raft_consensus.cc:3060] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:16.560629 21439 leader_election.cc:343] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.529284 21521 ts_tablet_manager.cc:1434] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.000s
I20260709 13:22:16.541582 21472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8188c780f29f41218f57313c157dae63" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:16.542284 21249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8188c780f29f41218f57313c157dae63" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:16.543843 21519 raft_consensus.cc:359] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.553535 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dbdec3b8bc40898dcd64ba19a4ec5d" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:16.540762 21525 leader_election.cc:290] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:16.563727 21521 tablet_bootstrap.cc:492] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:16.556794 21527 leader_election.cc:290] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:16.563913 21519 raft_consensus.cc:385] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.566083 21519 raft_consensus.cc:740] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.567705 21600 raft_consensus.cc:515] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
W20260709 13:22:16.568786 21363 leader_election.cc:343] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.571810 21519 consensus_queue.cc:260] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
W20260709 13:22:16.573801 21438 leader_election.cc:343] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:16.574139 21363 leader_election.cc:343] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.574326 21438 leader_election.cc:304] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:16.574626 21363 leader_election.cc:304] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636
I20260709 13:22:16.575733 21554 raft_consensus.cc:2749] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:16.576978 21521 tablet_bootstrap.cc:654] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.581718 21581 raft_consensus.cc:2749] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:16.585068 21574 raft_consensus.cc:493] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.585834 21574 raft_consensus.cc:515] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.591821 21574 leader_election.cc:290] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:16.592449 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01967f3f14be4ecaa1cc2469a50fc156" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:16.593891 21323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01967f3f14be4ecaa1cc2469a50fc156" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:16.594652 21323 raft_consensus.cc:2468] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:16.596668 21519 ts_tablet_manager.cc:1434] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.116s	user 0.004s	sys 0.004s
I20260709 13:22:16.597748 21519 tablet_bootstrap.cc:492] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
W20260709 13:22:16.595927 21216 leader_election.cc:343] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.609434 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5957ec3bcdcf40838d52d01a2a65281f" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:16.610337 21400 raft_consensus.cc:3060] T 5957ec3bcdcf40838d52d01a2a65281f P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.616134 21600 leader_election.cc:290] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:16.616880 21215 leader_election.cc:304] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:16.618615 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5957ec3bcdcf40838d52d01a2a65281f" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:16.619278 21474 raft_consensus.cc:3060] T 5957ec3bcdcf40838d52d01a2a65281f P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.627004 21473 raft_consensus.cc:2468] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 1.
I20260709 13:22:16.638738 21520 tablet_bootstrap.cc:492] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:16.639278 21520 ts_tablet_manager.cc:1403] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.189s	user 0.009s	sys 0.025s
I20260709 13:22:16.632130 21532 raft_consensus.cc:2804] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:16.625249 21400 raft_consensus.cc:2468] T 5957ec3bcdcf40838d52d01a2a65281f P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 1.
I20260709 13:22:16.640206 21532 raft_consensus.cc:493] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:16.656675 21532 raft_consensus.cc:3060] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.666973 21532 raft_consensus.cc:515] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.669623 21532 leader_election.cc:290] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Requested vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:16.670639 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01967f3f14be4ecaa1cc2469a50fc156" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:16.671635 21325 raft_consensus.cc:3060] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.671535 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01967f3f14be4ecaa1cc2469a50fc156" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:16.670639 21291 leader_election.cc:304] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
W20260709 13:22:16.678710 21216 leader_election.cc:343] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.680610 21530 raft_consensus.cc:2804] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:16.681142 21530 raft_consensus.cc:697] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 LEADER]: Becoming Leader. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Running, Role: LEADER
I20260709 13:22:16.682055 21530 consensus_queue.cc:237] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:16.683540 21519 tablet_bootstrap.cc:654] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.657315 21474 raft_consensus.cc:2468] T 5957ec3bcdcf40838d52d01a2a65281f P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 1.
I20260709 13:22:16.692394 21149 catalog_manager.cc:5697] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 1 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:16.692319 21325 raft_consensus.cc:2468] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:16.699968 21215 leader_election.cc:304] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:16.701169 21532 raft_consensus.cc:2804] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:16.701952 21532 raft_consensus.cc:697] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:16.701368 21520 raft_consensus.cc:359] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.702703 21520 raft_consensus.cc:385] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.703002 21520 raft_consensus.cc:740] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.707734 21518 tablet_bootstrap.cc:492] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:16.708135 21521 tablet_bootstrap.cc:492] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:16.708274 21518 ts_tablet_manager.cc:1403] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.234s	user 0.033s	sys 0.014s
I20260709 13:22:16.703032 21532 consensus_queue.cc:237] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.708619 21521 ts_tablet_manager.cc:1403] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.145s	user 0.027s	sys 0.013s
I20260709 13:22:16.713112 21519 tablet_bootstrap.cc:492] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:16.713734 21519 ts_tablet_manager.cc:1403] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.116s	user 0.014s	sys 0.000s
I20260709 13:22:16.713410 21521 raft_consensus.cc:359] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.713444 21145 catalog_manager.cc:5697] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66). New cstate: current_term: 1 leader_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:16.714345 21521 raft_consensus.cc:385] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.714169 21518 raft_consensus.cc:359] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.714870 21521 raft_consensus.cc:740] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.714996 21518 raft_consensus.cc:385] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.715363 21518 raft_consensus.cc:740] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.715607 21521 consensus_queue.cc:260] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.716142 21518 consensus_queue.cc:260] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.710685 21520 consensus_queue.cc:260] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.721630 21518 ts_tablet_manager.cc:1434] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.002s
I20260709 13:22:16.724155 21518 tablet_bootstrap.cc:492] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:16.719076 21519 raft_consensus.cc:359] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.723188 21521 ts_tablet_manager.cc:1434] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:22:16.726991 21521 tablet_bootstrap.cc:492] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:16.727550 21519 raft_consensus.cc:385] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.727890 21519 raft_consensus.cc:740] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.728098 21520 ts_tablet_manager.cc:1434] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.089s	user 0.007s	sys 0.001s
I20260709 13:22:16.733981 21518 tablet_bootstrap.cc:654] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.729611 21519 consensus_queue.cc:260] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.737919 21632 consensus_queue.cc:1048] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:16.744709 21519 ts_tablet_manager.cc:1434] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.001s
I20260709 13:22:16.746234 21519 tablet_bootstrap.cc:492] T 200a5b44be554a6bb18a21a29836902e P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:16.735270 21521 tablet_bootstrap.cc:654] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.757527 21519 tablet_bootstrap.cc:654] T 200a5b44be554a6bb18a21a29836902e P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.766882 21145 catalog_manager.cc:5697] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66). New cstate: current_term: 1 leader_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:16.770897 21146 catalog_manager.cc:5697] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:16.772195 21521 tablet_bootstrap.cc:492] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:16.772770 21521 ts_tablet_manager.cc:1403] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.046s	user 0.010s	sys 0.001s
I20260709 13:22:16.759948 21632 consensus_queue.cc:1048] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:22:16.763533 21518 tablet_bootstrap.cc:492] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:16.778653 21520 tablet_bootstrap.cc:492] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:16.775933 21521 raft_consensus.cc:359] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.779791 21518 ts_tablet_manager.cc:1403] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.056s	user 0.014s	sys 0.021s
I20260709 13:22:16.784052 21518 raft_consensus.cc:359] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:16.785156 21518 raft_consensus.cc:385] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.785634 21518 raft_consensus.cc:740] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.788472 21518 consensus_queue.cc:260] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:16.791865 21518 ts_tablet_manager.cc:1434] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:22:16.793411 21518 tablet_bootstrap.cc:492] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:16.779794 21521 raft_consensus.cc:385] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.798156 21521 raft_consensus.cc:740] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.798866 21521 consensus_queue.cc:260] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.800964 21521 ts_tablet_manager.cc:1434] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.028s	user 0.001s	sys 0.004s
I20260709 13:22:16.801923 21521 tablet_bootstrap.cc:492] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:16.813334 21518 tablet_bootstrap.cc:654] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.814204 21521 tablet_bootstrap.cc:654] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.815392 21621 raft_consensus.cc:493] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.815948 21621 raft_consensus.cc:515] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.818087 21520 tablet_bootstrap.cc:654] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.818306 21621 leader_election.cc:290] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:16.819139 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fec8ff4bc5a4d558263e38ae75ece75" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:16.820043 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fec8ff4bc5a4d558263e38ae75ece75" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
W20260709 13:22:16.820662 21437 leader_election.cc:343] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.815534 21519 tablet_bootstrap.cc:492] T 200a5b44be554a6bb18a21a29836902e P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:16.821532 21519 ts_tablet_manager.cc:1403] T 200a5b44be554a6bb18a21a29836902e P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.076s	user 0.017s	sys 0.018s
W20260709 13:22:16.821841 21438 leader_election.cc:343] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.822881 21438 leader_election.cc:304] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:16.825007 21621 raft_consensus.cc:2749] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:16.825785 21519 raft_consensus.cc:359] T 200a5b44be554a6bb18a21a29836902e P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.826552 21519 raft_consensus.cc:385] T 200a5b44be554a6bb18a21a29836902e P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.826828 21519 raft_consensus.cc:740] T 200a5b44be554a6bb18a21a29836902e P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.827498 21519 consensus_queue.cc:260] T 200a5b44be554a6bb18a21a29836902e P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.828994 21519 ts_tablet_manager.cc:1434] T 200a5b44be554a6bb18a21a29836902e P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.002s
I20260709 13:22:16.829792 21519 tablet_bootstrap.cc:492] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:16.842429 21530 raft_consensus.cc:493] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.843219 21530 raft_consensus.cc:515] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.846611 21249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
W20260709 13:22:16.848269 21289 leader_election.cc:343] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.849118 21532 raft_consensus.cc:493] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.849660 21532 raft_consensus.cc:515] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.850189 21530 leader_election.cc:290] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:16.850770 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:16.851823 21532 leader_election.cc:290] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:16.852880 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4eebc08c8f342a69d7a1248e7df69a1" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:16.853533 21474 raft_consensus.cc:2468] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:16.854650 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4eebc08c8f342a69d7a1248e7df69a1" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:16.856119 21214 leader_election.cc:304] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 
W20260709 13:22:16.856971 21215 leader_election.cc:343] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.857748 21532 raft_consensus.cc:2804] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:16.858184 21532 raft_consensus.cc:493] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:16.858582 21532 raft_consensus.cc:3060] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.864656 21519 tablet_bootstrap.cc:654] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
W20260709 13:22:16.866228 21291 leader_election.cc:343] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.866737 21291 leader_election.cc:304] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636
I20260709 13:22:16.867918 21530 raft_consensus.cc:2749] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:16.870977 21518 tablet_bootstrap.cc:492] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:16.871497 21518 ts_tablet_manager.cc:1403] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.078s	user 0.018s	sys 0.014s
I20260709 13:22:16.871551 21521 tablet_bootstrap.cc:492] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:16.872283 21521 ts_tablet_manager.cc:1403] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.071s	user 0.008s	sys 0.004s
I20260709 13:22:16.871520 21532 raft_consensus.cc:515] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.875013 21518 raft_consensus.cc:359] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.849885 21520 tablet_bootstrap.cc:492] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:16.875201 21521 raft_consensus.cc:359] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.875706 21518 raft_consensus.cc:385] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.875941 21521 raft_consensus.cc:385] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.876118 21518 raft_consensus.cc:740] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.876242 21521 raft_consensus.cc:740] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.877075 21532 leader_election.cc:290] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:16.876971 21518 consensus_queue.cc:260] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.877094 21521 consensus_queue.cc:260] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.876210 21520 ts_tablet_manager.cc:1403] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.147s	user 0.013s	sys 0.011s
I20260709 13:22:16.878306 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4eebc08c8f342a69d7a1248e7df69a1" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:16.878947 21474 raft_consensus.cc:3060] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:16.879418 21521 ts_tablet_manager.cc:1434] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:22:16.880394 21521 tablet_bootstrap.cc:492] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:16.881841 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4eebc08c8f342a69d7a1248e7df69a1" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
W20260709 13:22:16.883293 21215 leader_election.cc:343] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.884673 21520 raft_consensus.cc:359] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.885442 21520 raft_consensus.cc:385] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.885811 21520 raft_consensus.cc:740] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.886593 21520 consensus_queue.cc:260] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.888370 21519 tablet_bootstrap.cc:492] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:16.888871 21519 ts_tablet_manager.cc:1403] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.059s	user 0.012s	sys 0.001s
I20260709 13:22:16.888815 21520 ts_tablet_manager.cc:1434] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:22:16.890123 21520 tablet_bootstrap.cc:492] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:16.891806 21519 raft_consensus.cc:359] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.892457 21519 raft_consensus.cc:385] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.892737 21519 raft_consensus.cc:740] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.893944 21556 raft_consensus.cc:493] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.895820 21521 tablet_bootstrap.cc:654] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.895766 21556 raft_consensus.cc:515] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.894199 21519 consensus_queue.cc:260] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.898037 21556 leader_election.cc:290] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:16.900812 21520 tablet_bootstrap.cc:654] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.900573 21473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad13ab5b32d74d80990d10ba376cc2c7" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:16.902232 21249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad13ab5b32d74d80990d10ba376cc2c7" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:16.903961 21474 raft_consensus.cc:2468] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
W20260709 13:22:16.904129 21363 leader_election.cc:343] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:16.905119 21363 leader_election.cc:343] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:16.905702 21363 leader_election.cc:304] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636
I20260709 13:22:16.906100 21518 ts_tablet_manager.cc:1434] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.034s	user 0.003s	sys 0.003s
I20260709 13:22:16.907227 21556 raft_consensus.cc:2749] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:16.909312 21519 ts_tablet_manager.cc:1434] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.020s	user 0.002s	sys 0.005s
I20260709 13:22:16.909729 21214 leader_election.cc:304] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:16.911279 21532 raft_consensus.cc:2804] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:16.912202 21532 raft_consensus.cc:697] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:16.913350 21532 consensus_queue.cc:237] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:16.914631 21519 tablet_bootstrap.cc:492] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:16.921350 21519 tablet_bootstrap.cc:654] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.921803 21518 tablet_bootstrap.cc:492] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:16.931070 21148 catalog_manager.cc:5697] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:16.932235 21621 consensus_queue.cc:1048] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:16.938223 21518 tablet_bootstrap.cc:654] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.941746 21519 tablet_bootstrap.cc:492] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:16.942222 21519 ts_tablet_manager.cc:1403] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.028s	user 0.004s	sys 0.013s
I20260709 13:22:16.942843 21521 tablet_bootstrap.cc:492] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:16.943485 21521 ts_tablet_manager.cc:1403] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.063s	user 0.013s	sys 0.000s
I20260709 13:22:16.944980 21519 raft_consensus.cc:359] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.945691 21519 raft_consensus.cc:385] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.945966 21519 raft_consensus.cc:740] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.946619 21519 consensus_queue.cc:260] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.947093 21520 tablet_bootstrap.cc:492] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:16.947147 21521 raft_consensus.cc:359] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.947862 21520 ts_tablet_manager.cc:1403] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.058s	user 0.022s	sys 0.008s
I20260709 13:22:16.948120 21521 raft_consensus.cc:385] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.948550 21521 raft_consensus.cc:740] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.949587 21621 consensus_queue.cc:1048] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:16.951197 21520 raft_consensus.cc:359] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:16.951967 21520 raft_consensus.cc:385] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:16.952314 21520 raft_consensus.cc:740] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:16.953052 21520 consensus_queue.cc:260] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:16.949234 21521 consensus_queue.cc:260] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.958169 21519 ts_tablet_manager.cc:1434] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.016s	user 0.001s	sys 0.006s
I20260709 13:22:16.958698 21521 ts_tablet_manager.cc:1434] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:22:16.959187 21519 tablet_bootstrap.cc:492] T 5c0c8b12f7474d579a8805d344efb092 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:16.959618 21521 tablet_bootstrap.cc:492] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:16.961684 21520 ts_tablet_manager.cc:1434] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:22:16.962795 21520 tablet_bootstrap.cc:492] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:16.966578 21519 tablet_bootstrap.cc:654] T 5c0c8b12f7474d579a8805d344efb092 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.969338 21520 tablet_bootstrap.cc:654] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:16.985080 21621 raft_consensus.cc:493] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.986253 21621 raft_consensus.cc:515] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:16.989595 21556 raft_consensus.cc:493] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:16.990372 21556 raft_consensus.cc:515] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:16.998062 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01967f3f14be4ecaa1cc2469a50fc156" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:17.002128 21521 tablet_bootstrap.cc:654] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.003166 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01967f3f14be4ecaa1cc2469a50fc156" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:17.004305 21325 raft_consensus.cc:2393] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74e2ff1a68ac4c849ee6397ebb011375 in current term 1: Already voted for candidate ec879c0528c44465916efc6c33dde636 in this term.
I20260709 13:22:17.005411 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf088eed547f42aea083b31277ecece1" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:16.985183 21530 consensus_queue.cc:1048] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 13:22:17.001551 21621 leader_election.cc:290] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:17.013991 21364 leader_election.cc:304] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:17.014983 21525 raft_consensus.cc:3060] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:17.019598 21437 leader_election.cc:343] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.049245 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf088eed547f42aea083b31277ecece1" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
W20260709 13:22:17.051911 21439 leader_election.cc:343] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.052414 21439 leader_election.cc:304] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636
I20260709 13:22:17.002337 21556 leader_election.cc:290] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:17.053525 21621 raft_consensus.cc:2749] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:17.055717 21532 raft_consensus.cc:493] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:17.056207 21196 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:17.056224 21532 raft_consensus.cc:515] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:17.057507 21525 raft_consensus.cc:2749] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:17.058640 21532 leader_election.cc:290] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:17.066219 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d31c45d90b684fb18b03156866654665" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
W20260709 13:22:17.068925 21215 leader_election.cc:343] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.071534 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d31c45d90b684fb18b03156866654665" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
W20260709 13:22:17.075985 21216 leader_election.cc:343] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.076514 21216 leader_election.cc:304] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:17.077682 21532 raft_consensus.cc:2749] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:17.078341 21530 consensus_queue.cc:1048] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:17.087685 21519 tablet_bootstrap.cc:492] T 5c0c8b12f7474d579a8805d344efb092 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:17.088699 21519 ts_tablet_manager.cc:1403] T 5c0c8b12f7474d579a8805d344efb092 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.130s	user 0.044s	sys 0.070s
I20260709 13:22:17.092226 21519 raft_consensus.cc:359] T 5c0c8b12f7474d579a8805d344efb092 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:17.092927 21519 raft_consensus.cc:385] T 5c0c8b12f7474d579a8805d344efb092 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.093206 21519 raft_consensus.cc:740] T 5c0c8b12f7474d579a8805d344efb092 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.094197 21519 consensus_queue.cc:260] T 5c0c8b12f7474d579a8805d344efb092 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:17.101055 21518 tablet_bootstrap.cc:492] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:17.101624 21518 ts_tablet_manager.cc:1403] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.181s	user 0.020s	sys 0.020s
I20260709 13:22:17.107298 21519 ts_tablet_manager.cc:1434] T 5c0c8b12f7474d579a8805d344efb092 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.000s
I20260709 13:22:17.108781 21519 tablet_bootstrap.cc:492] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:17.114171 21518 raft_consensus.cc:359] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:17.114854 21518 raft_consensus.cc:385] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.115137 21518 raft_consensus.cc:740] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.115684 21519 tablet_bootstrap.cc:654] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.115940 21518 consensus_queue.cc:260] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:17.118755 21518 ts_tablet_manager.cc:1434] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:22:17.120071 21518 tablet_bootstrap.cc:492] T a85e6742ffca4d9c962b8613986c3896 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:17.115221 21621 raft_consensus.cc:493] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.129431 21527 raft_consensus.cc:493] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.135882 21527 raft_consensus.cc:515] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:17.138219 21527 leader_election.cc:290] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:17.140326 21632 raft_consensus.cc:493] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.140909 21632 raft_consensus.cc:515] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:17.129803 21621 raft_consensus.cc:515] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:17.143200 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8188c780f29f41218f57313c157dae63" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:17.143893 21400 raft_consensus.cc:2468] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 0.
I20260709 13:22:17.144945 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8188c780f29f41218f57313c157dae63" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:17.145651 21251 raft_consensus.cc:2468] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 0.
I20260709 13:22:17.147797 21521 tablet_bootstrap.cc:492] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:17.148406 21521 ts_tablet_manager.cc:1403] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.189s	user 0.021s	sys 0.021s
I20260709 13:22:17.151715 21437 leader_election.cc:304] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:17.151834 21326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b503b6f79842efbb8a7ecd5c5851e0" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
W20260709 13:22:17.154373 21438 leader_election.cc:343] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.159607 21632 leader_election.cc:290] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:17.160585 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b503b6f79842efbb8a7ecd5c5851e0" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:17.167943 21631 consensus_queue.cc:1048] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:17.170221 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d275cc2417214c6f80ff88e7fc3916b7" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:17.170722 21532 consensus_queue.cc:1048] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:17.144979 21621 leader_election.cc:290] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
W20260709 13:22:17.176789 21291 leader_election.cc:343] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.177485 21621 raft_consensus.cc:2804] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.178032 21621 raft_consensus.cc:493] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.178246 21631 raft_consensus.cc:493] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.182194 21520 tablet_bootstrap.cc:492] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:17.182724 21520 ts_tablet_manager.cc:1403] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.220s	user 0.066s	sys 0.058s
I20260709 13:22:17.179020 21631 raft_consensus.cc:515] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:17.182335 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d275cc2417214c6f80ff88e7fc3916b7" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:17.188668 21556 raft_consensus.cc:493] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.185509 21520 raft_consensus.cc:359] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:17.202412 21520 raft_consensus.cc:385] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.202927 21520 raft_consensus.cc:740] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.178352 21621 raft_consensus.cc:3060] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.205773 21519 tablet_bootstrap.cc:492] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:17.207027 21519 ts_tablet_manager.cc:1403] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.099s	user 0.017s	sys 0.042s
I20260709 13:22:17.187995 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "200a5b44be554a6bb18a21a29836902e" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:17.185814 21631 leader_election.cc:290] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:17.220585 21520 consensus_queue.cc:260] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:17.203536 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "200a5b44be554a6bb18a21a29836902e" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:17.224113 21474 raft_consensus.cc:2468] T 200a5b44be554a6bb18a21a29836902e P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 0.
I20260709 13:22:17.210299 21251 raft_consensus.cc:2468] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 0.
I20260709 13:22:17.226194 21519 raft_consensus.cc:359] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:17.227330 21519 raft_consensus.cc:385] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.227663 21519 raft_consensus.cc:740] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.228515 21519 consensus_queue.cc:260] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
W20260709 13:22:17.229377 21289 leader_election.cc:343] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.230381 21289 leader_election.cc:304] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:17.231449 21668 consensus_queue.cc:1048] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:17.205984 21556 raft_consensus.cc:515] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:17.236140 21556 leader_election.cc:290] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:17.241441 21520 ts_tablet_manager.cc:1434] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.058s	user 0.005s	sys 0.004s
I20260709 13:22:17.243839 21520 tablet_bootstrap.cc:492] T 5fec8ff4bc5a4d558263e38ae75ece75 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:17.246694 21289 leader_election.cc:304] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: ec879c0528c44465916efc6c33dde636
I20260709 13:22:17.247925 21600 raft_consensus.cc:2804] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.248425 21600 raft_consensus.cc:493] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.248860 21600 raft_consensus.cc:3060] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.253014 21521 raft_consensus.cc:359] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:17.253991 21521 raft_consensus.cc:385] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.254320 21521 raft_consensus.cc:740] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.256831 21519 ts_tablet_manager.cc:1434] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.000s
I20260709 13:22:17.258193 21519 tablet_bootstrap.cc:492] T 799c8e732d4d46e3b26dcd77f458dcbe P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:17.262557 21631 raft_consensus.cc:2804] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.262905 21518 tablet_bootstrap.cc:654] T a85e6742ffca4d9c962b8613986c3896 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.263155 21631 raft_consensus.cc:493] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.263897 21631 raft_consensus.cc:3060] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.266467 21521 consensus_queue.cc:260] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:17.269651 21521 ts_tablet_manager.cc:1434] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.121s	user 0.009s	sys 0.000s
I20260709 13:22:17.270843 21530 consensus_queue.cc:1048] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:17.276387 21530 raft_consensus.cc:493] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.277105 21530 raft_consensus.cc:515] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:17.279448 21532 raft_consensus.cc:493] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.280319 21532 raft_consensus.cc:515] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:17.272032 21323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dbdec3b8bc40898dcd64ba19a4ec5d" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:17.281968 21323 raft_consensus.cc:2468] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
I20260709 13:22:17.284161 21364 leader_election.cc:304] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:17.286485 21525 raft_consensus.cc:2804] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.286973 21525 raft_consensus.cc:493] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.287376 21525 raft_consensus.cc:3060] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.291977 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dbdec3b8bc40898dcd64ba19a4ec5d" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:17.306252 21669 raft_consensus.cc:493] T 5c0c8b12f7474d579a8805d344efb092 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.307109 21669 raft_consensus.cc:515] T 5c0c8b12f7474d579a8805d344efb092 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:17.316064 21621 raft_consensus.cc:515] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
W20260709 13:22:17.322710 21289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36863, user_credentials={real_user=slave}} blocked reactor thread for 76138us
I20260709 13:22:17.324349 21519 tablet_bootstrap.cc:654] T 799c8e732d4d46e3b26dcd77f458dcbe P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.327041 21398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c0c8b12f7474d579a8805d344efb092" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
W20260709 13:22:17.328747 21439 leader_election.cc:343] T 5c0c8b12f7474d579a8805d344efb092 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.328975 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dbdec3b8bc40898dcd64ba19a4ec5d" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:17.329862 21474 raft_consensus.cc:2468] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 0.
I20260709 13:22:17.330224 21521 tablet_bootstrap.cc:492] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:17.332366 21289 leader_election.cc:304] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 
I20260709 13:22:17.327069 21530 leader_election.cc:290] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:17.335947 21532 leader_election.cc:290] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:17.334210 21669 leader_election.cc:290] T 5c0c8b12f7474d579a8805d344efb092 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
W20260709 13:22:17.349517 21437 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:35963, user_credentials={real_user=slave}} blocked reactor thread for 197922us
I20260709 13:22:17.358564 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c0c8b12f7474d579a8805d344efb092" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:17.365453 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fec8ff4bc5a4d558263e38ae75ece75" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:17.366225 21474 raft_consensus.cc:2468] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:17.377914 21398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8188c780f29f41218f57313c157dae63" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:17.378600 21398 raft_consensus.cc:3060] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.380571 21520 tablet_bootstrap.cc:654] T 5fec8ff4bc5a4d558263e38ae75ece75 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.383320 21621 leader_election.cc:290] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:17.398643 21521 tablet_bootstrap.cc:654] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.377314 21525 raft_consensus.cc:515] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:17.399928 21530 raft_consensus.cc:2804] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.415374 21530 raft_consensus.cc:493] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.415805 21530 raft_consensus.cc:3060] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.393404 21631 raft_consensus.cc:515] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:17.430001 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dbdec3b8bc40898dcd64ba19a4ec5d" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:17.430974 21474 raft_consensus.cc:2468] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
I20260709 13:22:17.432669 21399 raft_consensus.cc:2393] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5109d6d6d9404ba39c53508a81fe9f1c in current term 1: Already voted for candidate 74e2ff1a68ac4c849ee6397ebb011375 in this term.
I20260709 13:22:17.421535 21621 raft_consensus.cc:493] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.439920 21621 raft_consensus.cc:515] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:17.441491 21518 tablet_bootstrap.cc:492] T a85e6742ffca4d9c962b8613986c3896 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:17.442042 21518 ts_tablet_manager.cc:1403] T a85e6742ffca4d9c962b8613986c3896 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.322s	user 0.021s	sys 0.003s
I20260709 13:22:17.442073 21621 leader_election.cc:290] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:17.447237 21525 leader_election.cc:290] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Requested vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
W20260709 13:22:17.449092 21437 leader_election.cc:343] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.449718 21437 leader_election.cc:304] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:17.451200 21667 raft_consensus.cc:2749] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:22:17.451296 21437 leader_election.cc:343] T 5c0c8b12f7474d579a8805d344efb092 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.451980 21437 leader_election.cc:304] T 5c0c8b12f7474d579a8805d344efb092 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636
I20260709 13:22:17.451992 21667 consensus_queue.cc:1048] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:17.452482 21631 leader_election.cc:290] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:17.455135 21398 raft_consensus.cc:2468] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:17.453274 21632 raft_consensus.cc:493] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.456678 21632 raft_consensus.cc:515] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:17.457635 21530 raft_consensus.cc:515] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:17.458658 21439 leader_election.cc:304] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:17.460547 21530 leader_election.cc:290] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:17.453326 21600 raft_consensus.cc:515] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:17.461402 21669 raft_consensus.cc:2804] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:17.462486 21473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "799c8e732d4d46e3b26dcd77f458dcbe" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:17.463099 21214 leader_election.cc:304] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:17.445024 21518 raft_consensus.cc:359] T a85e6742ffca4d9c962b8613986c3896 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:17.458868 21632 leader_election.cc:290] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:17.473269 21249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc40f11dce5540d1a056e9bacba075e2" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:17.464691 21667 raft_consensus.cc:2749] T 5c0c8b12f7474d579a8805d344efb092 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:17.471448 21472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dbdec3b8bc40898dcd64ba19a4ec5d" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:17.499429 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "200a5b44be554a6bb18a21a29836902e" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:17.474157 21669 raft_consensus.cc:697] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:17.507679 21532 raft_consensus.cc:2804] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.509406 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8188c780f29f41218f57313c157dae63" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:17.475797 21518 raft_consensus.cc:385] T a85e6742ffca4d9c962b8613986c3896 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.473933 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc40f11dce5540d1a056e9bacba075e2" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:17.521246 21518 raft_consensus.cc:740] T a85e6742ffca4d9c962b8613986c3896 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.517189 21472 raft_consensus.cc:3060] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.515720 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dbdec3b8bc40898dcd64ba19a4ec5d" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:17.526872 21518 consensus_queue.cc:260] T a85e6742ffca4d9c962b8613986c3896 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:17.518111 21532 raft_consensus.cc:493] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.519405 21474 raft_consensus.cc:3060] T 200a5b44be554a6bb18a21a29836902e P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.522360 21250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "200a5b44be554a6bb18a21a29836902e" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:17.529270 21251 raft_consensus.cc:3060] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.550345 21518 ts_tablet_manager.cc:1434] T a85e6742ffca4d9c962b8613986c3896 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.108s	user 0.005s	sys 0.009s
W20260709 13:22:17.528288 21289 leader_election.cc:343] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.539215 21532 raft_consensus.cc:3060] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:17.555580 21437 leader_election.cc:343] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.556043 21519 tablet_bootstrap.cc:492] T 799c8e732d4d46e3b26dcd77f458dcbe P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:17.556577 21519 ts_tablet_manager.cc:1403] T 799c8e732d4d46e3b26dcd77f458dcbe P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.299s	user 0.035s	sys 0.007s
I20260709 13:22:17.560580 21519 raft_consensus.cc:359] T 799c8e732d4d46e3b26dcd77f458dcbe P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:17.561681 21519 raft_consensus.cc:385] T 799c8e732d4d46e3b26dcd77f458dcbe P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.562302 21519 raft_consensus.cc:740] T 799c8e732d4d46e3b26dcd77f458dcbe P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.562976 21519 consensus_queue.cc:260] T 799c8e732d4d46e3b26dcd77f458dcbe P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:17.565232 21519 ts_tablet_manager.cc:1434] T 799c8e732d4d46e3b26dcd77f458dcbe P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:22:17.543691 21556 raft_consensus.cc:493] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.566174 21519 tablet_bootstrap.cc:492] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:17.566061 21556 raft_consensus.cc:515] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:17.568238 21556 leader_election.cc:290] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:17.569041 21470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dbdec3b8bc40898dcd64ba19a4ec5d" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:17.569926 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97dbdec3b8bc40898dcd64ba19a4ec5d" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:17.573674 21399 raft_consensus.cc:2393] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5109d6d6d9404ba39c53508a81fe9f1c in current term 1: Already voted for candidate 74e2ff1a68ac4c849ee6397ebb011375 in this term.
I20260709 13:22:17.574887 21635 consensus_queue.cc:1048] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:17.571316 21532 raft_consensus.cc:515] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:17.576366 21397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "799c8e732d4d46e3b26dcd77f458dcbe" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
W20260709 13:22:17.572506 21439 leader_election.cc:343] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.577505 21520 tablet_bootstrap.cc:492] T 5fec8ff4bc5a4d558263e38ae75ece75 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:17.577739 21439 leader_election.cc:304] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636
I20260709 13:22:17.578047 21520 ts_tablet_manager.cc:1403] T 5fec8ff4bc5a4d558263e38ae75ece75 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.335s	user 0.010s	sys 0.012s
I20260709 13:22:17.584314 21532 leader_election.cc:290] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:17.585727 21621 raft_consensus.cc:2749] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:17.586666 21472 raft_consensus.cc:2468] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 1.
I20260709 13:22:17.587131 21600 leader_election.cc:290] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:17.588560 21289 leader_election.cc:304] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 
I20260709 13:22:17.572774 21325 raft_consensus.cc:2393] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 74e2ff1a68ac4c849ee6397ebb011375 in current term 1: Already voted for candidate 5109d6d6d9404ba39c53508a81fe9f1c in this term.
I20260709 13:22:17.590855 21397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d275cc2417214c6f80ff88e7fc3916b7" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
W20260709 13:22:17.591390 21291 leader_election.cc:343] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.570658 21669 consensus_queue.cc:237] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:17.592003 21291 leader_election.cc:304] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:17.594481 21474 raft_consensus.cc:2468] T 200a5b44be554a6bb18a21a29836902e P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 1.
I20260709 13:22:17.575623 21323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d31c45d90b684fb18b03156866654665" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:17.599684 21518 tablet_bootstrap.cc:492] T d275cc2417214c6f80ff88e7fc3916b7 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:17.596282 21600 raft_consensus.cc:2804] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:22:17.604861 21291 leader_election.cc:343] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.609344 21364 leader_election.cc:304] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:17.610599 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fec8ff4bc5a4d558263e38ae75ece75" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:17.610859 21520 raft_consensus.cc:359] T 5fec8ff4bc5a4d558263e38ae75ece75 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:17.611951 21520 raft_consensus.cc:385] T 5fec8ff4bc5a4d558263e38ae75ece75 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.612421 21520 raft_consensus.cc:740] T 5fec8ff4bc5a4d558263e38ae75ece75 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.615235 21521 tablet_bootstrap.cc:492] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:17.614405 21520 consensus_queue.cc:260] T 5fec8ff4bc5a4d558263e38ae75ece75 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:17.602430 21250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d31c45d90b684fb18b03156866654665" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:17.610126 21322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fec8ff4bc5a4d558263e38ae75ece75" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:17.603765 21289 leader_election.cc:304] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 
I20260709 13:22:17.622895 21520 ts_tablet_manager.cc:1434] T 5fec8ff4bc5a4d558263e38ae75ece75 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.045s	user 0.009s	sys 0.000s
I20260709 13:22:17.615341 21518 tablet_bootstrap.cc:654] T d275cc2417214c6f80ff88e7fc3916b7 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.624176 21250 raft_consensus.cc:2468] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
I20260709 13:22:17.613385 21600 raft_consensus.cc:697] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 LEADER]: Becoming Leader. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Running, Role: LEADER
I20260709 13:22:17.625669 21325 raft_consensus.cc:2468] T 5fec8ff4bc5a4d558263e38ae75ece75 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:17.614265 21556 raft_consensus.cc:2749] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:22:17.626724 21681 raft_consensus.cc:2749] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:17.616685 21521 ts_tablet_manager.cc:1403] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.287s	user 0.030s	sys 0.015s
W20260709 13:22:17.625833 21364 leader_election.cc:343] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:17.640578 21215 consensus_peers.cc:597] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:17.645603 21363 leader_election.cc:304] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:17.646449 21556 raft_consensus.cc:2804] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.646828 21556 raft_consensus.cc:493] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.647183 21556 raft_consensus.cc:3060] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.648545 21251 raft_consensus.cc:2468] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:17.651945 21683 raft_consensus.cc:2804] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:17.652531 21683 raft_consensus.cc:697] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 LEADER]: Becoming Leader. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Running, Role: LEADER
I20260709 13:22:17.653396 21683 consensus_queue.cc:237] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:17.653867 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fec8ff4bc5a4d558263e38ae75ece75" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
W20260709 13:22:17.642975 21289 leader_election.cc:343] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.654708 21474 raft_consensus.cc:3060] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.656440 21520 tablet_bootstrap.cc:492] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:17.654047 21600 consensus_queue.cc:237] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:17.669472 21519 tablet_bootstrap.cc:654] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.676288 21520 tablet_bootstrap.cc:654] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.680254 21521 raft_consensus.cc:359] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:17.680997 21521 raft_consensus.cc:385] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.681303 21521 raft_consensus.cc:740] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.681874 21525 raft_consensus.cc:493] T a85e6742ffca4d9c962b8613986c3896 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.681998 21521 consensus_queue.cc:260] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:17.682386 21525 raft_consensus.cc:515] T a85e6742ffca4d9c962b8613986c3896 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:17.685236 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a85e6742ffca4d9c962b8613986c3896" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:17.686640 21525 leader_election.cc:290] T a85e6742ffca4d9c962b8613986c3896 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
W20260709 13:22:17.688042 21364 leader_election.cc:343] T a85e6742ffca4d9c962b8613986c3896 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.703260 21469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a85e6742ffca4d9c962b8613986c3896" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
W20260709 13:22:17.705370 21363 leader_election.cc:343] T a85e6742ffca4d9c962b8613986c3896 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.706079 21363 leader_election.cc:304] T a85e6742ffca4d9c962b8613986c3896 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:17.707870 21676 raft_consensus.cc:2749] T a85e6742ffca4d9c962b8613986c3896 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:17.701377 21145 catalog_manager.cc:5697] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 1 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:17.715366 21521 ts_tablet_manager.cc:1434] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.079s	user 0.005s	sys 0.001s
I20260709 13:22:17.716161 21521 tablet_bootstrap.cc:492] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:17.725446 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d275cc2417214c6f80ff88e7fc3916b7" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:17.726393 21248 raft_consensus.cc:3060] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.737308 21474 raft_consensus.cc:2468] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:17.748658 21521 tablet_bootstrap.cc:654] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.750339 21518 tablet_bootstrap.cc:492] T d275cc2417214c6f80ff88e7fc3916b7 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:17.751067 21518 ts_tablet_manager.cc:1403] T d275cc2417214c6f80ff88e7fc3916b7 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.152s	user 0.016s	sys 0.005s
I20260709 13:22:17.758404 21518 raft_consensus.cc:359] T d275cc2417214c6f80ff88e7fc3916b7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:17.759472 21518 raft_consensus.cc:385] T d275cc2417214c6f80ff88e7fc3916b7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.759780 21518 raft_consensus.cc:740] T d275cc2417214c6f80ff88e7fc3916b7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.760306 21556 raft_consensus.cc:515] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:17.760696 21518 consensus_queue.cc:260] T d275cc2417214c6f80ff88e7fc3916b7 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:17.754719 21635 consensus_queue.cc:1048] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:22:17.764024 21518 ts_tablet_manager.cc:1434] T d275cc2417214c6f80ff88e7fc3916b7 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:22:17.764993 21518 tablet_bootstrap.cc:492] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:17.780678 21214 leader_election.cc:304] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:17.785063 21247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d31c45d90b684fb18b03156866654665" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:17.785497 21556 leader_election.cc:290] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:17.786068 21247 raft_consensus.cc:3060] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.786943 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d31c45d90b684fb18b03156866654665" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:17.787932 21635 raft_consensus.cc:2804] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:17.788481 21635 raft_consensus.cc:697] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
W20260709 13:22:17.789032 21364 leader_election.cc:343] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.789350 21635 consensus_queue.cc:237] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:17.807474 21247 raft_consensus.cc:2468] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:17.810468 21519 tablet_bootstrap.cc:492] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:17.817327 21519 ts_tablet_manager.cc:1403] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.251s	user 0.025s	sys 0.078s
I20260709 13:22:17.817989 21363 leader_election.cc:304] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:17.819203 21556 raft_consensus.cc:2804] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:17.819849 21556 raft_consensus.cc:697] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Leader. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:17.820106 21519 raft_consensus.cc:359] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:17.812820 21248 raft_consensus.cc:2468] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 1.
I20260709 13:22:17.814796 21518 tablet_bootstrap.cc:654] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.820802 21556 consensus_queue.cc:237] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:17.820807 21519 raft_consensus.cc:385] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.821837 21289 leader_election.cc:304] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:17.822109 21519 raft_consensus.cc:740] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.822875 21520 tablet_bootstrap.cc:492] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:17.823077 21683 raft_consensus.cc:2804] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:17.823360 21520 ts_tablet_manager.cc:1403] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.167s	user 0.008s	sys 0.012s
I20260709 13:22:17.816592 21521 tablet_bootstrap.cc:492] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:17.824666 21521 ts_tablet_manager.cc:1403] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.109s	user 0.009s	sys 0.004s
I20260709 13:22:17.823793 21519 consensus_queue.cc:260] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:17.823628 21683 raft_consensus.cc:697] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 LEADER]: Becoming Leader. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Running, Role: LEADER
I20260709 13:22:17.828933 21683 consensus_queue.cc:237] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:17.835111 21519 ts_tablet_manager.cc:1434] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:22:17.836424 21519 tablet_bootstrap.cc:492] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:17.838474 21525 raft_consensus.cc:493] T d275cc2417214c6f80ff88e7fc3916b7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.839015 21525 raft_consensus.cc:515] T d275cc2417214c6f80ff88e7fc3916b7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:17.841228 21525 leader_election.cc:290] T d275cc2417214c6f80ff88e7fc3916b7 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:17.842900 21247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d275cc2417214c6f80ff88e7fc3916b7" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:17.833886 21635 raft_consensus.cc:493] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.844491 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d275cc2417214c6f80ff88e7fc3916b7" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:17.847572 21635 raft_consensus.cc:515] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:17.847605 21247 raft_consensus.cc:2393] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74e2ff1a68ac4c849ee6397ebb011375 in current term 1: Already voted for candidate 5109d6d6d9404ba39c53508a81fe9f1c in this term.
I20260709 13:22:17.850242 21363 leader_election.cc:304] T d275cc2417214c6f80ff88e7fc3916b7 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:17.851141 21676 raft_consensus.cc:3060] T d275cc2417214c6f80ff88e7fc3916b7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.852790 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "200a5b44be554a6bb18a21a29836902e" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:17.853996 21474 raft_consensus.cc:2393] T 200a5b44be554a6bb18a21a29836902e P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec879c0528c44465916efc6c33dde636 in current term 1: Already voted for candidate 5109d6d6d9404ba39c53508a81fe9f1c in this term.
I20260709 13:22:17.855201 21519 tablet_bootstrap.cc:654] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.866196 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "200a5b44be554a6bb18a21a29836902e" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:17.874285 21518 tablet_bootstrap.cc:492] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:17.874833 21518 ts_tablet_manager.cc:1403] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.110s	user 0.013s	sys 0.002s
I20260709 13:22:17.870651 21635 leader_election.cc:290] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:17.878322 21676 raft_consensus.cc:2749] T d275cc2417214c6f80ff88e7fc3916b7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:17.880024 21521 raft_consensus.cc:359] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:17.880765 21521 raft_consensus.cc:385] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.882176 21521 raft_consensus.cc:740] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.873517 21214 leader_election.cc:304] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec879c0528c44465916efc6c33dde636; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:17.886351 21532 raft_consensus.cc:3060] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.888552 21520 raft_consensus.cc:359] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:17.893222 21518 raft_consensus.cc:359] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:17.894068 21518 raft_consensus.cc:385] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.894347 21518 raft_consensus.cc:740] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.895035 21518 consensus_queue.cc:260] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:17.895879 21520 raft_consensus.cc:385] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.896379 21520 raft_consensus.cc:740] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.897326 21518 ts_tablet_manager.cc:1434] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:22:17.898248 21532 raft_consensus.cc:2749] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:17.898389 21518 tablet_bootstrap.cc:492] T 37a3c02685464cf1af805559b46c4356 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:17.897435 21520 consensus_queue.cc:260] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:17.901942 21521 consensus_queue.cc:260] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:17.899245 21146 catalog_manager.cc:5697] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:17.906638 21520 ts_tablet_manager.cc:1434] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.082s	user 0.002s	sys 0.005s
I20260709 13:22:17.908344 21520 tablet_bootstrap.cc:492] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:17.908609 21521 ts_tablet_manager.cc:1434] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.084s	user 0.005s	sys 0.001s
I20260709 13:22:17.909670 21521 tablet_bootstrap.cc:492] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:17.911492 21518 tablet_bootstrap.cc:654] T 37a3c02685464cf1af805559b46c4356 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.916385 21521 tablet_bootstrap.cc:654] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.917047 21600 raft_consensus.cc:493] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.917470 21520 tablet_bootstrap.cc:654] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.917675 21600 raft_consensus.cc:515] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:17.922511 21600 leader_election.cc:290] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:17.925283 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a85e6742ffca4d9c962b8613986c3896" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:17.931656 21520 tablet_bootstrap.cc:492] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:17.932394 21520 ts_tablet_manager.cc:1403] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.000s
W20260709 13:22:17.929948 21289 leader_election.cc:343] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.936322 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a85e6742ffca4d9c962b8613986c3896" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:17.937078 21400 raft_consensus.cc:2468] T a85e6742ffca4d9c962b8613986c3896 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 0.
I20260709 13:22:17.941236 21520 raft_consensus.cc:359] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:17.942406 21520 raft_consensus.cc:385] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.942612 21291 leader_election.cc:304] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 43cd181efe6b4b2fb79fd3e997a401ea
I20260709 13:22:17.943822 21683 raft_consensus.cc:2804] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.944211 21683 raft_consensus.cc:493] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.944495 21683 raft_consensus.cc:3060] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.946713 21532 raft_consensus.cc:493] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.947284 21532 raft_consensus.cc:515] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:17.949448 21532 leader_election.cc:290] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:17.950331 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:17.951013 21325 raft_consensus.cc:2468] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:17.951849 21556 raft_consensus.cc:493] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:17.952348 21556 raft_consensus.cc:515] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:17.952690 21683 raft_consensus.cc:515] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:17.954639 21556 leader_election.cc:290] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:17.956113 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "799c8e732d4d46e3b26dcd77f458dcbe" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:17.956372 21215 leader_election.cc:304] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:17.956723 21469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a85e6742ffca4d9c962b8613986c3896" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:17.958546 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "799c8e732d4d46e3b26dcd77f458dcbe" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:17.958606 21397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:17.959385 21474 raft_consensus.cc:2468] T 799c8e732d4d46e3b26dcd77f458dcbe P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
I20260709 13:22:17.956825 21325 raft_consensus.cc:2468] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
W20260709 13:22:17.961961 21289 leader_election.cc:343] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.962872 21364 leader_election.cc:304] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:17.960592 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a85e6742ffca4d9c962b8613986c3896" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:17.963689 21400 raft_consensus.cc:3060] T a85e6742ffca4d9c962b8613986c3896 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.964645 21683 leader_election.cc:290] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
W20260709 13:22:17.960335 21216 leader_election.cc:343] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:17.962000 21521 tablet_bootstrap.cc:492] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:17.965750 21556 raft_consensus.cc:2804] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.968107 21556 raft_consensus.cc:493] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.968612 21556 raft_consensus.cc:3060] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.969751 21149 catalog_manager.cc:5697] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66). New cstate: current_term: 1 leader_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:17.942688 21520 raft_consensus.cc:740] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.972092 21149 catalog_manager.cc:5697] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66). New cstate: current_term: 1 leader_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:17.966465 21521 ts_tablet_manager.cc:1403] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.057s	user 0.017s	sys 0.012s
I20260709 13:22:17.976701 21521 raft_consensus.cc:359] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:17.972410 21520 consensus_queue.cc:260] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:17.978073 21521 raft_consensus.cc:385] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:17.978444 21521 raft_consensus.cc:740] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.971136 21532 raft_consensus.cc:2804] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:17.979168 21521 consensus_queue.cc:260] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:17.979777 21520 ts_tablet_manager.cc:1434] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.047s	user 0.010s	sys 0.000s
I20260709 13:22:17.979667 21532 raft_consensus.cc:493] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:17.980767 21532 raft_consensus.cc:3060] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:17.980856 21520 tablet_bootstrap.cc:492] T d31c45d90b684fb18b03156866654665 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:17.981820 21521 ts_tablet_manager.cc:1434] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:22:17.982846 21521 tablet_bootstrap.cc:492] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:17.987435 21520 tablet_bootstrap.cc:654] T d31c45d90b684fb18b03156866654665 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.988870 21521 tablet_bootstrap.cc:654] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:17.994434 21519 tablet_bootstrap.cc:492] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:17.994940 21519 ts_tablet_manager.cc:1403] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.159s	user 0.017s	sys 0.004s
I20260709 13:22:18.004823 21520 tablet_bootstrap.cc:492] T d31c45d90b684fb18b03156866654665 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:18.006078 21520 ts_tablet_manager.cc:1403] T d31c45d90b684fb18b03156866654665 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.016s
I20260709 13:22:18.006733 21518 tablet_bootstrap.cc:492] T 37a3c02685464cf1af805559b46c4356 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:18.007244 21518 ts_tablet_manager.cc:1403] T 37a3c02685464cf1af805559b46c4356 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.109s	user 0.019s	sys 0.006s
I20260709 13:22:18.009039 21146 catalog_manager.cc:5697] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66). New cstate: current_term: 1 leader_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:18.009646 21520 raft_consensus.cc:359] T d31c45d90b684fb18b03156866654665 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:18.010064 21518 raft_consensus.cc:359] T 37a3c02685464cf1af805559b46c4356 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:18.010553 21520 raft_consensus.cc:385] T d31c45d90b684fb18b03156866654665 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:18.010776 21518 raft_consensus.cc:385] T 37a3c02685464cf1af805559b46c4356 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:18.010974 21520 raft_consensus.cc:740] T d31c45d90b684fb18b03156866654665 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:17.993131 21400 raft_consensus.cc:2468] T a85e6742ffca4d9c962b8613986c3896 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 1.
I20260709 13:22:18.012084 21520 consensus_queue.cc:260] T d31c45d90b684fb18b03156866654665 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:18.011137 21518 raft_consensus.cc:740] T 37a3c02685464cf1af805559b46c4356 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:18.013898 21518 consensus_queue.cc:260] T 37a3c02685464cf1af805559b46c4356 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:18.014644 21291 leader_election.cc:304] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 43cd181efe6b4b2fb79fd3e997a401ea
I20260709 13:22:18.016536 21518 ts_tablet_manager.cc:1434] T 37a3c02685464cf1af805559b46c4356 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
I20260709 13:22:18.018092 21518 tablet_bootstrap.cc:492] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:18.020062 21621 consensus_queue.cc:1048] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.021894 21600 raft_consensus.cc:2804] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:18.022490 21600 raft_consensus.cc:697] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 LEADER]: Becoming Leader. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Running, Role: LEADER
I20260709 13:22:18.023360 21600 consensus_queue.cc:237] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:18.024391 21521 tablet_bootstrap.cc:492] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:18.024926 21521 ts_tablet_manager.cc:1403] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.042s	user 0.012s	sys 0.025s
I20260709 13:22:18.025442 21518 tablet_bootstrap.cc:654] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:18.027993 21521 raft_consensus.cc:359] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:18.028890 21521 raft_consensus.cc:385] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:18.029325 21521 raft_consensus.cc:740] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:18.024677 21519 raft_consensus.cc:359] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:18.030189 21521 consensus_queue.cc:260] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:18.030515 21520 ts_tablet_manager.cc:1434] T d31c45d90b684fb18b03156866654665 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 13:22:18.030676 21519 raft_consensus.cc:385] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:18.039036 21519 raft_consensus.cc:740] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:18.040251 21520 tablet_bootstrap.cc:492] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:18.049880 21519 consensus_queue.cc:260] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:18.065017 21521 ts_tablet_manager.cc:1434] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.040s	user 0.012s	sys 0.000s
I20260709 13:22:18.067771 21521 tablet_bootstrap.cc:492] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:18.062335 21686 raft_consensus.cc:493] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.073501 21686 raft_consensus.cc:515] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:18.075937 21683 consensus_queue.cc:1048] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:18.076532 21686 leader_election.cc:290] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:18.079432 21398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf088eed547f42aea083b31277ecece1" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
W20260709 13:22:18.081036 21216 leader_election.cc:343] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:18.081893 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf088eed547f42aea083b31277ecece1" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:18.082580 21474 raft_consensus.cc:2468] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:18.082211 21532 raft_consensus.cc:515] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:18.085436 21683 raft_consensus.cc:493] T d31c45d90b684fb18b03156866654665 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.085971 21683 raft_consensus.cc:515] T d31c45d90b684fb18b03156866654665 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:18.086306 21322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:18.086477 21681 consensus_queue.cc:1048] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.086930 21322 raft_consensus.cc:3060] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.087076 21214 leader_election.cc:304] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:18.089018 21687 raft_consensus.cc:2804] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:18.089394 21687 raft_consensus.cc:493] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:18.089736 21687 raft_consensus.cc:3060] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.089993 21247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d31c45d90b684fb18b03156866654665" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:18.090777 21247 raft_consensus.cc:2393] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5109d6d6d9404ba39c53508a81fe9f1c in current term 1: Already voted for candidate 74e2ff1a68ac4c849ee6397ebb011375 in this term.
I20260709 13:22:18.094599 21683 leader_election.cc:290] T d31c45d90b684fb18b03156866654665 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:18.094795 21669 consensus_queue.cc:1048] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.096851 21705 raft_consensus.cc:493] T 37a3c02685464cf1af805559b46c4356 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.097373 21705 raft_consensus.cc:515] T 37a3c02685464cf1af805559b46c4356 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:18.098138 21532 leader_election.cc:290] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Requested vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:18.099829 21705 leader_election.cc:290] T 37a3c02685464cf1af805559b46c4356 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:18.100544 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a3c02685464cf1af805559b46c4356" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:18.100675 21398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d31c45d90b684fb18b03156866654665" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:18.100942 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a3c02685464cf1af805559b46c4356" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
W20260709 13:22:18.101838 21364 leader_election.cc:343] T 37a3c02685464cf1af805559b46c4356 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:18.102536 21363 leader_election.cc:343] T 37a3c02685464cf1af805559b46c4356 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:18.102857 21291 leader_election.cc:304] T d31c45d90b684fb18b03156866654665 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636
I20260709 13:22:18.103047 21363 leader_election.cc:304] T 37a3c02685464cf1af805559b46c4356 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:18.103396 21398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:18.104063 21683 raft_consensus.cc:3060] T d31c45d90b684fb18b03156866654665 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.104211 21705 raft_consensus.cc:2749] T 37a3c02685464cf1af805559b46c4356 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:22:18.104959 21216 leader_election.cc:343] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:18.105115 21521 tablet_bootstrap.cc:654] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:18.114382 21519 ts_tablet_manager.cc:1434] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.119s	user 0.009s	sys 0.001s
I20260709 13:22:18.115816 21519 tablet_bootstrap.cc:492] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:18.122720 21519 tablet_bootstrap.cc:654] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:18.123921 21681 consensus_queue.cc:1048] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.131529 21520 tablet_bootstrap.cc:654] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:18.133078 21322 raft_consensus.cc:2468] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:18.053426 21556 raft_consensus.cc:515] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:18.145207 21149 catalog_manager.cc:5697] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66). New cstate: current_term: 1 leader_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:18.135913 21683 raft_consensus.cc:2749] T d31c45d90b684fb18b03156866654665 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:18.145676 21600 raft_consensus.cc:493] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.136898 21687 raft_consensus.cc:515] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:18.166541 21215 leader_election.cc:304] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:18.179471 21686 raft_consensus.cc:2804] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:18.180047 21686 raft_consensus.cc:697] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:18.180897 21686 consensus_queue.cc:237] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:18.185938 21518 tablet_bootstrap.cc:492] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:18.186589 21518 ts_tablet_manager.cc:1403] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.169s	user 0.032s	sys 0.053s
I20260709 13:22:18.160369 21600 raft_consensus.cc:515] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:18.185860 21556 leader_election.cc:290] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Requested vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:18.190963 21469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "799c8e732d4d46e3b26dcd77f458dcbe" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:18.191669 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "799c8e732d4d46e3b26dcd77f458dcbe" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:18.191680 21469 raft_consensus.cc:3060] T 799c8e732d4d46e3b26dcd77f458dcbe P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.192353 21325 raft_consensus.cc:3060] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.194840 21687 leader_election.cc:290] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:18.196900 21472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf088eed547f42aea083b31277ecece1" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:18.197551 21472 raft_consensus.cc:3060] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.214671 21518 raft_consensus.cc:359] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:18.217873 21518 raft_consensus.cc:385] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:18.218295 21518 raft_consensus.cc:740] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:18.233878 21518 consensus_queue.cc:260] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:18.239157 21149 catalog_manager.cc:5697] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "74e2ff1a68ac4c849ee6397ebb011375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:18.212397 21398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf088eed547f42aea083b31277ecece1" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
W20260709 13:22:18.224239 21196 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:18.224933 21473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4eebc08c8f342a69d7a1248e7df69a1" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:18.219270 21532 raft_consensus.cc:493] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.243918 21472 raft_consensus.cc:2468] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:18.244333 21532 raft_consensus.cc:515] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:18.246457 21532 leader_election.cc:290] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:18.248066 21322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b503b6f79842efbb8a7ecd5c5851e0" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
W20260709 13:22:18.249549 21215 leader_election.cc:343] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:18.250526 21216 leader_election.cc:343] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:18.251055 21521 tablet_bootstrap.cc:492] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:18.251592 21521 ts_tablet_manager.cc:1403] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.184s	user 0.016s	sys 0.012s
I20260709 13:22:18.252040 21214 leader_election.cc:304] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:18.252918 21687 raft_consensus.cc:2804] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:18.263665 21518 ts_tablet_manager.cc:1434] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.077s	user 0.011s	sys 0.000s
I20260709 13:22:18.241250 21469 raft_consensus.cc:2468] T 799c8e732d4d46e3b26dcd77f458dcbe P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:18.267385 21520 tablet_bootstrap.cc:492] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:18.267889 21520 ts_tablet_manager.cc:1403] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.229s	user 0.016s	sys 0.009s
I20260709 13:22:18.269649 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4eebc08c8f342a69d7a1248e7df69a1" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:18.259389 21687 raft_consensus.cc:697] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:18.270233 21521 raft_consensus.cc:359] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:18.262482 21473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b503b6f79842efbb8a7ecd5c5851e0" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:18.271189 21521 raft_consensus.cc:385] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:18.271593 21521 raft_consensus.cc:740] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:18.271988 21519 tablet_bootstrap.cc:492] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:18.271303 21687 consensus_queue.cc:237] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:18.272589 21521 consensus_queue.cc:260] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:18.274739 21518 tablet_bootstrap.cc:492] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:18.279896 21519 ts_tablet_manager.cc:1403] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.164s	user 0.018s	sys 0.026s
I20260709 13:22:18.277395 21325 raft_consensus.cc:2468] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:18.274858 21521 ts_tablet_manager.cc:1434] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.023s	user 0.004s	sys 0.002s
I20260709 13:22:18.276854 21363 leader_election.cc:304] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:18.271392 21473 raft_consensus.cc:2468] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:18.283941 21556 raft_consensus.cc:2804] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:18.277096 21669 raft_consensus.cc:493] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.285992 21556 raft_consensus.cc:697] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Leader. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:18.285676 21521 tablet_bootstrap.cc:492] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:18.286104 21669 raft_consensus.cc:515] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:18.286872 21556 consensus_queue.cc:237] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:18.292347 21600 leader_election.cc:290] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:18.292737 21214 leader_election.cc:304] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:18.282871 21519 raft_consensus.cc:359] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:18.294515 21519 raft_consensus.cc:385] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:18.295465 21519 raft_consensus.cc:740] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:18.300998 21519 consensus_queue.cc:260] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:18.305178 21519 ts_tablet_manager.cc:1434] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:22:18.306491 21519 tablet_bootstrap.cc:492] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:18.308964 21686 raft_consensus.cc:2804] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:18.309423 21686 raft_consensus.cc:493] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:18.310027 21686 raft_consensus.cc:3060] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.310158 21520 raft_consensus.cc:359] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:18.311065 21520 raft_consensus.cc:385] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:18.313294 21520 raft_consensus.cc:740] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:18.311579 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad13ab5b32d74d80990d10ba376cc2c7" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:18.314589 21400 raft_consensus.cc:2468] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 0.
I20260709 13:22:18.315776 21439 leader_election.cc:304] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:18.316721 21527 raft_consensus.cc:2804] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:18.317098 21527 raft_consensus.cc:493] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:18.317427 21527 raft_consensus.cc:3060] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.322213 21669 leader_election.cc:290] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:18.315786 21519 tablet_bootstrap.cc:654] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:18.324129 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad13ab5b32d74d80990d10ba376cc2c7" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:18.324906 21248 raft_consensus.cc:2468] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 0.
I20260709 13:22:18.330137 21518 tablet_bootstrap.cc:654] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:18.329682 21520 consensus_queue.cc:260] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:18.330916 21532 consensus_queue.cc:1048] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:18.346457 21520 ts_tablet_manager.cc:1434] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.078s	user 0.009s	sys 0.000s
I20260709 13:22:18.349210 21520 tablet_bootstrap.cc:492] T 51766d5a86a6498b8e30249efa0ac990 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:18.356678 21520 tablet_bootstrap.cc:654] T 51766d5a86a6498b8e30249efa0ac990 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:18.361451 21325 raft_consensus.cc:3060] T 5fec8ff4bc5a4d558263e38ae75ece75 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.338773 21521 tablet_bootstrap.cc:654] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
W20260709 13:22:18.341430 21289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:35963, user_credentials={real_user=slave}} blocked reactor thread for 71964.8us
I20260709 13:22:18.337718 21686 raft_consensus.cc:515] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:18.337026 21527 raft_consensus.cc:515] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:18.375859 21686 leader_election.cc:290] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:18.378717 21725 consensus_queue.cc:1048] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.379027 21322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b503b6f79842efbb8a7ecd5c5851e0" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:18.379623 21322 raft_consensus.cc:3060] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.384202 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b503b6f79842efbb8a7ecd5c5851e0" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:18.384860 21474 raft_consensus.cc:3060] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.381407 21289 leader_election.cc:304] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636
I20260709 13:22:18.389281 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad13ab5b32d74d80990d10ba376cc2c7" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:18.391184 21725 raft_consensus.cc:2749] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:18.391224 21400 raft_consensus.cc:3060] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.396384 21556 consensus_queue.cc:1048] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.406546 21686 consensus_queue.cc:1048] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.408828 21527 leader_election.cc:290] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:18.414021 21247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad13ab5b32d74d80990d10ba376cc2c7" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:18.414693 21247 raft_consensus.cc:3060] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.418401 21323 raft_consensus.cc:3060] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.418211 21145 catalog_manager.cc:5697] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "74e2ff1a68ac4c849ee6397ebb011375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:18.437083 21527 raft_consensus.cc:493] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.437676 21527 raft_consensus.cc:515] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:18.441160 21520 tablet_bootstrap.cc:492] T 51766d5a86a6498b8e30249efa0ac990 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:18.441931 21520 ts_tablet_manager.cc:1403] T 51766d5a86a6498b8e30249efa0ac990 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.093s	user 0.022s	sys 0.031s
I20260709 13:22:18.445183 21527 leader_election.cc:290] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:18.446871 21518 tablet_bootstrap.cc:492] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:18.447445 21518 ts_tablet_manager.cc:1403] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.183s	user 0.014s	sys 0.009s
I20260709 13:22:18.447734 21521 tablet_bootstrap.cc:492] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:18.448418 21521 ts_tablet_manager.cc:1403] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.164s	user 0.011s	sys 0.007s
I20260709 13:22:18.432802 21322 raft_consensus.cc:2468] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:18.435250 21400 raft_consensus.cc:2468] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:18.467466 21215 leader_election.cc:304] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:18.476441 21686 raft_consensus.cc:2804] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:18.476970 21686 raft_consensus.cc:697] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:18.477834 21686 consensus_queue.cc:237] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:18.479040 21725 consensus_queue.cc:1048] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.483757 21398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a85e6742ffca4d9c962b8613986c3896" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:18.484197 21439 leader_election.cc:304] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:18.484649 21398 raft_consensus.cc:2393] T a85e6742ffca4d9c962b8613986c3896 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 43cd181efe6b4b2fb79fd3e997a401ea in current term 1: Already voted for candidate 5109d6d6d9404ba39c53508a81fe9f1c in this term.
I20260709 13:22:18.485263 21667 raft_consensus.cc:2804] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:18.490936 21146 catalog_manager.cc:5697] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:18.492982 21146 catalog_manager.cc:5697] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:18.496018 21518 raft_consensus.cc:359] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:18.496678 21518 raft_consensus.cc:385] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:18.496973 21518 raft_consensus.cc:740] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:18.498883 21518 consensus_queue.cc:260] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:18.508737 21518 ts_tablet_manager.cc:1434] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.061s	user 0.006s	sys 0.001s
I20260709 13:22:18.510226 21519 tablet_bootstrap.cc:492] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:18.510959 21519 ts_tablet_manager.cc:1403] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.205s	user 0.021s	sys 0.023s
I20260709 13:22:18.512266 21518 tablet_bootstrap.cc:492] T 7a5e0169610b4059877434f37effb058 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:18.545992 21676 raft_consensus.cc:493] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.546614 21676 raft_consensus.cc:515] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:18.548611 21676 leader_election.cc:290] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:18.549671 21724 consensus_queue.cc:1048] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.550944 21473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c0c8b12f7474d579a8805d344efb092" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:18.551988 21473 raft_consensus.cc:2468] T 5c0c8b12f7474d579a8805d344efb092 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
I20260709 13:22:18.553592 21363 leader_election.cc:304] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:18.554890 21469 raft_consensus.cc:3060] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.554971 21247 raft_consensus.cc:2468] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:18.556236 21676 raft_consensus.cc:2804] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:18.556722 21676 raft_consensus.cc:493] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:18.557126 21676 raft_consensus.cc:3060] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.545473 21520 raft_consensus.cc:359] T 51766d5a86a6498b8e30249efa0ac990 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:18.563508 21520 raft_consensus.cc:385] T 51766d5a86a6498b8e30249efa0ac990 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:18.563938 21520 raft_consensus.cc:740] T 51766d5a86a6498b8e30249efa0ac990 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:18.576411 21676 raft_consensus.cc:515] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:18.579738 21472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c0c8b12f7474d579a8805d344efb092" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:18.580387 21472 raft_consensus.cc:3060] T 5c0c8b12f7474d579a8805d344efb092 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.582031 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c0c8b12f7474d579a8805d344efb092" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:18.582886 21520 consensus_queue.cc:260] T 51766d5a86a6498b8e30249efa0ac990 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:18.576005 21519 raft_consensus.cc:359] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:18.585948 21519 raft_consensus.cc:385] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:18.557284 21474 raft_consensus.cc:2468] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:18.586469 21519 raft_consensus.cc:740] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:18.591511 21519 consensus_queue.cc:260] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
W20260709 13:22:18.585539 21739 log.cc:927] Time spent T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636: Append to log took a long time: real 0.060s	user 0.003s	sys 0.000s
I20260709 13:22:18.598904 21521 raft_consensus.cc:359] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:18.599738 21521 raft_consensus.cc:385] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:18.599997 21521 raft_consensus.cc:740] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:18.600625 21521 consensus_queue.cc:260] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:18.602756 21527 raft_consensus.cc:493] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.591260 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a85e6742ffca4d9c962b8613986c3896" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:18.604254 21519 ts_tablet_manager.cc:1434] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.093s	user 0.007s	sys 0.002s
I20260709 13:22:18.591768 21667 raft_consensus.cc:697] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:18.606782 21519 tablet_bootstrap.cc:492] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:18.604668 21527 raft_consensus.cc:515] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:18.612262 21521 ts_tablet_manager.cc:1434] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.163s	user 0.008s	sys 0.001s
W20260709 13:22:18.594274 21363 leader_election.cc:343] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:18.611963 21705 consensus_queue.cc:1048] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:22:18.616789 21527 leader_election.cc:290] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:18.620820 21518 tablet_bootstrap.cc:654] T 7a5e0169610b4059877434f37effb058 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:18.627143 21250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1bf171a09b745ec963d790f1eefe432" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:18.627879 21250 raft_consensus.cc:2468] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 0.
I20260709 13:22:18.629854 21520 ts_tablet_manager.cc:1434] T 51766d5a86a6498b8e30249efa0ac990 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.188s	user 0.010s	sys 0.000s
I20260709 13:22:18.636198 21520 tablet_bootstrap.cc:492] T 7a5e0169610b4059877434f37effb058 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:18.634977 21705 raft_consensus.cc:493] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.638442 21705 raft_consensus.cc:515] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:18.640595 21705 leader_election.cc:290] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:18.647261 21438 leader_election.cc:304] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:18.649698 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:18.650604 21324 raft_consensus.cc:2393] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74e2ff1a68ac4c849ee6397ebb011375 in current term 1: Already voted for candidate ec879c0528c44465916efc6c33dde636 in this term.
I20260709 13:22:18.649780 21437 leader_election.cc:304] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:18.651291 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1bf171a09b745ec963d790f1eefe432" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:18.651566 21669 raft_consensus.cc:2804] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:18.652010 21669 raft_consensus.cc:493] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:18.652405 21669 raft_consensus.cc:3060] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.656067 21472 raft_consensus.cc:2468] T 5c0c8b12f7474d579a8805d344efb092 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:18.657910 21250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:18.658607 21363 leader_election.cc:304] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:18.659960 21556 raft_consensus.cc:2804] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:18.659744 21725 consensus_queue.cc:1048] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:18.660465 21556 raft_consensus.cc:697] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Leader. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:18.660554 21399 raft_consensus.cc:3060] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.661420 21556 consensus_queue.cc:237] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
W20260709 13:22:18.661707 21438 leader_election.cc:343] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:18.612874 21676 leader_election.cc:290] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:18.669934 21363 leader_election.cc:304] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:18.670584 21527 raft_consensus.cc:2749] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:18.677376 21676 raft_consensus.cc:2749] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:18.679566 21725 consensus_queue.cc:1048] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.684782 21519 tablet_bootstrap.cc:654] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:18.692497 21518 tablet_bootstrap.cc:492] T 7a5e0169610b4059877434f37effb058 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:18.692921 21740 raft_consensus.cc:493] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.693580 21740 raft_consensus.cc:515] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:18.696960 21740 leader_election.cc:290] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:18.693051 21518 ts_tablet_manager.cc:1403] T 7a5e0169610b4059877434f37effb058 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.181s	user 0.011s	sys 0.028s
I20260709 13:22:18.698613 21397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a3c02685464cf1af805559b46c4356" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:18.699448 21397 raft_consensus.cc:2468] T 37a3c02685464cf1af805559b46c4356 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 0.
I20260709 13:22:18.700737 21439 leader_election.cc:304] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:18.701818 21518 raft_consensus.cc:359] T 7a5e0169610b4059877434f37effb058 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:18.702117 21740 raft_consensus.cc:2804] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:18.702491 21518 raft_consensus.cc:385] T 7a5e0169610b4059877434f37effb058 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:18.702749 21740 raft_consensus.cc:493] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:18.702984 21518 raft_consensus.cc:740] T 7a5e0169610b4059877434f37effb058 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:18.703235 21740 raft_consensus.cc:3060] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.704037 21518 consensus_queue.cc:260] T 7a5e0169610b4059877434f37effb058 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:18.704424 21519 tablet_bootstrap.cc:492] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:18.705996 21519 ts_tablet_manager.cc:1403] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.099s	user 0.012s	sys 0.006s
I20260709 13:22:18.706024 21518 ts_tablet_manager.cc:1434] T 7a5e0169610b4059877434f37effb058 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:22:18.707254 21518 tablet_bootstrap.cc:492] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:18.708998 21519 raft_consensus.cc:359] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:18.711915 21519 raft_consensus.cc:385] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:18.712438 21519 raft_consensus.cc:740] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:18.719529 21521 tablet_bootstrap.cc:492] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:18.723263 21519 consensus_queue.cc:260] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:18.723812 21148 catalog_manager.cc:5697] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:18.724541 21742 consensus_queue.cc:1048] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:18.726200 21521 tablet_bootstrap.cc:654] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:18.689008 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c0c8b12f7474d579a8805d344efb092" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:18.732592 21519 ts_tablet_manager.cc:1434] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.003s
I20260709 13:22:18.713805 21667 consensus_queue.cc:237] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:18.736482 21519 tablet_bootstrap.cc:492] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
W20260709 13:22:18.737891 21363 leader_election.cc:343] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:18.740310 21669 raft_consensus.cc:515] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:18.741612 21520 tablet_bootstrap.cc:654] T 7a5e0169610b4059877434f37effb058 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:18.734288 21322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a3c02685464cf1af805559b46c4356" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:18.744057 21740 raft_consensus.cc:515] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:18.774690 21669 leader_election.cc:290] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:18.776553 21518 tablet_bootstrap.cc:654] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
W20260709 13:22:18.766781 21438 leader_election.cc:343] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:18.786406 21250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1bf171a09b745ec963d790f1eefe432" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:18.791086 21520 tablet_bootstrap.cc:492] T 7a5e0169610b4059877434f37effb058 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:18.791533 21250 raft_consensus.cc:3060] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.791699 21520 ts_tablet_manager.cc:1403] T 7a5e0169610b4059877434f37effb058 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.161s	user 0.019s	sys 0.009s
I20260709 13:22:18.794727 21520 raft_consensus.cc:359] T 7a5e0169610b4059877434f37effb058 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:18.795369 21520 raft_consensus.cc:385] T 7a5e0169610b4059877434f37effb058 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:18.795634 21520 raft_consensus.cc:740] T 7a5e0169610b4059877434f37effb058 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:18.796213 21740 leader_election.cc:290] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Requested vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:18.796260 21520 consensus_queue.cc:260] T 7a5e0169610b4059877434f37effb058 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:18.798130 21397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a3c02685464cf1af805559b46c4356" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:18.798698 21520 ts_tablet_manager.cc:1434] T 7a5e0169610b4059877434f37effb058 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:22:18.799697 21520 tablet_bootstrap.cc:492] T a1bf171a09b745ec963d790f1eefe432 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:18.782873 21687 consensus_queue.cc:1048] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 13:22:18.807521 21322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1bf171a09b745ec963d790f1eefe432" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:18.808706 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a3c02685464cf1af805559b46c4356" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
W20260709 13:22:18.809263 21438 leader_election.cc:343] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:18.810421 21519 tablet_bootstrap.cc:654] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:18.815533 21520 tablet_bootstrap.cc:654] T a1bf171a09b745ec963d790f1eefe432 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:18.813678 21250 raft_consensus.cc:2468] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:18.817792 21748 consensus_queue.cc:1048] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:18.798738 21397 raft_consensus.cc:3060] T 37a3c02685464cf1af805559b46c4356 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.819583 21437 leader_election.cc:304] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:18.821496 21740 raft_consensus.cc:2804] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:18.822528 21740 raft_consensus.cc:697] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:18.823819 21740 consensus_queue.cc:237] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:18.835856 21145 catalog_manager.cc:5697] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "74e2ff1a68ac4c849ee6397ebb011375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:18.844250 21397 raft_consensus.cc:2468] T 37a3c02685464cf1af805559b46c4356 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:18.839237 21683 raft_consensus.cc:493] T 51766d5a86a6498b8e30249efa0ac990 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.864876 21683 raft_consensus.cc:515] T 51766d5a86a6498b8e30249efa0ac990 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
W20260709 13:22:18.851346 21438 leader_election.cc:343] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:18.869057 21439 leader_election.cc:304] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:18.873349 21667 raft_consensus.cc:2804] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:18.873894 21667 raft_consensus.cc:697] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:18.874738 21667 consensus_queue.cc:237] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:18.883749 21683 leader_election.cc:290] T 51766d5a86a6498b8e30249efa0ac990 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:18.885511 21687 consensus_queue.cc:1048] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:18.892656 21527 raft_consensus.cc:493] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:18.893136 21250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51766d5a86a6498b8e30249efa0ac990" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:18.893383 21527 raft_consensus.cc:515] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:18.906250 21146 catalog_manager.cc:5697] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 1 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:18.923218 21398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51766d5a86a6498b8e30249efa0ac990" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:18.925990 21687 consensus_queue.cc:1048] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:22:18.919781 21289 leader_election.cc:343] T 51766d5a86a6498b8e30249efa0ac990 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:18.931960 21216 consensus_peers.cc:597] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 -> Peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Couldn't send request to peer 74e2ff1a68ac4c849ee6397ebb011375. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:18.934387 21519 tablet_bootstrap.cc:492] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:18.935088 21519 ts_tablet_manager.cc:1403] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.199s	user 0.019s	sys 0.004s
I20260709 13:22:18.941834 21748 consensus_queue.cc:1048] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:18.947978 21519 raft_consensus.cc:359] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:18.949024 21519 raft_consensus.cc:385] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:18.949373 21519 raft_consensus.cc:740] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:18.950299 21519 consensus_queue.cc:260] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:18.955989 21527 leader_election.cc:290] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:18.963171 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5e0169610b4059877434f37effb058" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:18.963994 21399 raft_consensus.cc:2468] T 7a5e0169610b4059877434f37effb058 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 0.
I20260709 13:22:18.965770 21439 leader_election.cc:304] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:18.979792 21519 ts_tablet_manager.cc:1434] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.044s	user 0.009s	sys 0.000s
I20260709 13:22:18.981791 21519 tablet_bootstrap.cc:492] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:18.968820 21740 raft_consensus.cc:2804] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:18.984999 21740 raft_consensus.cc:493] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:18.991153 21740 raft_consensus.cc:3060] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:18.992059 21521 tablet_bootstrap.cc:492] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:18.992590 21521 ts_tablet_manager.cc:1403] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.273s	user 0.015s	sys 0.006s
W20260709 13:22:18.992944 21291 leader_election.cc:343] T 51766d5a86a6498b8e30249efa0ac990 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:18.993470 21291 leader_election.cc:304] T 51766d5a86a6498b8e30249efa0ac990 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636
I20260709 13:22:18.995411 21518 tablet_bootstrap.cc:492] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:18.995927 21518 ts_tablet_manager.cc:1403] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.289s	user 0.046s	sys 0.058s
I20260709 13:22:18.999647 21687 consensus_queue.cc:1048] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:19.000630 21518 raft_consensus.cc:359] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:19.001300 21518 raft_consensus.cc:385] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:19.001588 21518 raft_consensus.cc:740] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:19.002337 21725 raft_consensus.cc:2749] T 51766d5a86a6498b8e30249efa0ac990 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:18.999590 21521 raft_consensus.cc:359] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:19.003211 21521 raft_consensus.cc:385] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:19.003509 21521 raft_consensus.cc:740] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:19.005056 21521 consensus_queue.cc:260] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:19.005064 21518 consensus_queue.cc:260] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:19.014430 21521 ts_tablet_manager.cc:1434] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 13:22:19.015384 21521 tablet_bootstrap.cc:492] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:19.019783 21518 ts_tablet_manager.cc:1434] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.002s
I20260709 13:22:19.021059 21518 tablet_bootstrap.cc:492] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:19.024094 21519 tablet_bootstrap.cc:654] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:19.029912 21322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5e0169610b4059877434f37effb058" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:19.030684 21322 raft_consensus.cc:2468] T 7a5e0169610b4059877434f37effb058 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 0.
W20260709 13:22:19.033273 21758 log.cc:927] Time spent T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:22:19.034981 21518 tablet_bootstrap.cc:654] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:19.037356 21520 tablet_bootstrap.cc:492] T a1bf171a09b745ec963d790f1eefe432 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:19.037874 21520 ts_tablet_manager.cc:1403] T a1bf171a09b745ec963d790f1eefe432 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.238s	user 0.014s	sys 0.004s
I20260709 13:22:19.040050 21532 raft_consensus.cc:493] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:19.042459 21532 raft_consensus.cc:515] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:19.044152 21740 raft_consensus.cc:515] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:19.045207 21532 leader_election.cc:290] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:19.047909 21472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c0c8b12f7474d579a8805d344efb092" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:19.048743 21472 raft_consensus.cc:2393] T 5c0c8b12f7474d579a8805d344efb092 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec879c0528c44465916efc6c33dde636 in current term 1: Already voted for candidate 74e2ff1a68ac4c849ee6397ebb011375 in this term.
I20260709 13:22:19.048997 21520 raft_consensus.cc:359] T a1bf171a09b745ec963d790f1eefe432 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:19.049736 21520 raft_consensus.cc:385] T a1bf171a09b745ec963d790f1eefe432 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:19.050001 21520 raft_consensus.cc:740] T a1bf171a09b745ec963d790f1eefe432 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:19.050052 21740 leader_election.cc:290] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Requested vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:19.050441 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5e0169610b4059877434f37effb058" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:19.051070 21396 raft_consensus.cc:3060] T 7a5e0169610b4059877434f37effb058 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:19.050714 21520 consensus_queue.cc:260] T a1bf171a09b745ec963d790f1eefe432 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:19.058719 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a5e0169610b4059877434f37effb058" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:19.059361 21324 raft_consensus.cc:3060] T 7a5e0169610b4059877434f37effb058 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:19.062750 21398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c0c8b12f7474d579a8805d344efb092" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:19.065234 21216 leader_election.cc:304] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec879c0528c44465916efc6c33dde636; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:19.067234 21687 raft_consensus.cc:3060] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:19.067438 21324 raft_consensus.cc:2468] T 7a5e0169610b4059877434f37effb058 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:19.069931 21438 leader_election.cc:304] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 
I20260709 13:22:19.070775 21667 raft_consensus.cc:2804] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:19.071277 21667 raft_consensus.cc:697] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:19.072108 21667 consensus_queue.cc:237] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:19.076326 21687 raft_consensus.cc:2749] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:19.079568 21520 ts_tablet_manager.cc:1434] T a1bf171a09b745ec963d790f1eefe432 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.041s	user 0.005s	sys 0.003s
I20260709 13:22:19.080921 21520 tablet_bootstrap.cc:492] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:19.132964 21521 tablet_bootstrap.cc:654] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:19.142448 21520 tablet_bootstrap.cc:654] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:19.149147 21518 tablet_bootstrap.cc:492] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:19.149655 21518 ts_tablet_manager.cc:1403] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.129s	user 0.024s	sys 0.014s
I20260709 13:22:19.130520 21396 raft_consensus.cc:2468] T 7a5e0169610b4059877434f37effb058 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:19.152437 21518 raft_consensus.cc:359] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:19.157802 21518 raft_consensus.cc:385] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:19.158169 21518 raft_consensus.cc:740] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:19.161021 21518 consensus_queue.cc:260] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:19.178381 21518 ts_tablet_manager.cc:1434] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
I20260709 13:22:19.183661 21518 tablet_bootstrap.cc:492] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:19.185024 21676 raft_consensus.cc:493] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:19.185990 21676 raft_consensus.cc:515] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:19.188657 21520 tablet_bootstrap.cc:492] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:19.189157 21520 ts_tablet_manager.cc:1403] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.109s	user 0.010s	sys 0.006s
I20260709 13:22:19.189548 21676 leader_election.cc:290] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:19.190915 21469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc40f11dce5540d1a056e9bacba075e2" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:19.191965 21250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc40f11dce5540d1a056e9bacba075e2" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:19.191910 21469 raft_consensus.cc:2468] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
W20260709 13:22:19.193740 21363 leader_election.cc:343] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:19.194537 21363 leader_election.cc:304] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: ec879c0528c44465916efc6c33dde636
I20260709 13:22:19.196241 21676 raft_consensus.cc:2804] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:19.196647 21676 raft_consensus.cc:493] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:19.196879 21518 tablet_bootstrap.cc:654] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:19.197145 21676 raft_consensus.cc:3060] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:19.197402 21527 consensus_queue.cc:1048] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:19.198320 21520 raft_consensus.cc:359] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:19.199091 21520 raft_consensus.cc:385] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:19.199401 21520 raft_consensus.cc:740] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:19.203729 21521 tablet_bootstrap.cc:492] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:19.210177 21521 ts_tablet_manager.cc:1403] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.195s	user 0.023s	sys 0.016s
I20260709 13:22:19.211089 21520 consensus_queue.cc:260] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:19.213626 21521 raft_consensus.cc:359] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:19.214524 21521 raft_consensus.cc:385] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:19.214826 21521 raft_consensus.cc:740] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:19.215744 21521 consensus_queue.cc:260] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:19.216256 21520 ts_tablet_manager.cc:1434] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 13:22:19.221297 21521 ts_tablet_manager.cc:1434] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:22:19.205987 21145 catalog_manager.cc:5697] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 1 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:19.222448 21521 tablet_bootstrap.cc:492] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:19.223093 21145 catalog_manager.cc:5697] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 1 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:19.210126 21519 tablet_bootstrap.cc:492] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:19.226985 21519 ts_tablet_manager.cc:1403] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.246s	user 0.022s	sys 0.011s
I20260709 13:22:19.229530 21521 tablet_bootstrap.cc:654] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:19.208595 21676 raft_consensus.cc:515] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:19.231271 21519 raft_consensus.cc:359] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:19.232343 21519 raft_consensus.cc:385] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:19.232694 21519 raft_consensus.cc:740] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:19.235055 21469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc40f11dce5540d1a056e9bacba075e2" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:19.235764 21469 raft_consensus.cc:3060] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:19.225697 21520 tablet_bootstrap.cc:492] T 37a3c02685464cf1af805559b46c4356 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:19.234460 21519 consensus_queue.cc:260] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:19.239166 21676 leader_election.cc:290] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:19.239869 21250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc40f11dce5540d1a056e9bacba075e2" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:19.242429 21748 raft_consensus.cc:493] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:19.251426 21748 raft_consensus.cc:515] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:19.252456 21519 ts_tablet_manager.cc:1434] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
W20260709 13:22:19.252535 21363 leader_election.cc:343] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:19.252777 21469 raft_consensus.cc:2468] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:19.253539 21519 tablet_bootstrap.cc:492] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:19.247470 21667 consensus_queue.cc:1048] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:19.238277 21145 catalog_manager.cc:5697] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 1 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:19.260855 21363 leader_election.cc:304] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: ec879c0528c44465916efc6c33dde636
I20260709 13:22:19.262699 21747 raft_consensus.cc:2804] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:19.263259 21747 raft_consensus.cc:697] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Leader. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:19.264127 21747 consensus_queue.cc:237] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:19.267752 21740 consensus_queue.cc:1048] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:19.270453 21250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf088eed547f42aea083b31277ecece1" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:19.253901 21748 leader_election.cc:290] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:19.305979 21748 consensus_queue.cc:1048] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:22:19.315855 21519 tablet_bootstrap.cc:654] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:19.322135 21520 tablet_bootstrap.cc:654] T 37a3c02685464cf1af805559b46c4356 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:19.330721 21676 consensus_queue.cc:1048] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:19.335955 21518 tablet_bootstrap.cc:492] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:19.336448 21518 ts_tablet_manager.cc:1403] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.157s	user 0.031s	sys 0.026s
I20260709 13:22:19.339428 21469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf088eed547f42aea083b31277ecece1" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:19.345063 21145 catalog_manager.cc:5697] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "74e2ff1a68ac4c849ee6397ebb011375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:19.352082 21363 leader_election.cc:304] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636
I20260709 13:22:19.357182 21676 raft_consensus.cc:2749] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:19.358824 21323 raft_consensus.cc:3060] T a1bf171a09b745ec963d790f1eefe432 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:19.356158 21518 raft_consensus.cc:359] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:19.363992 21518 raft_consensus.cc:385] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:19.364262 21518 raft_consensus.cc:740] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:19.364934 21518 consensus_queue.cc:260] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
W20260709 13:22:19.366324 21438 consensus_peers.cc:597] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:19.371771 21518 ts_tablet_manager.cc:1434] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.035s	user 0.000s	sys 0.006s
I20260709 13:22:19.372825 21518 tablet_bootstrap.cc:492] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:19.372597 21676 raft_consensus.cc:493] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:19.373543 21676 raft_consensus.cc:515] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:19.376123 21521 tablet_bootstrap.cc:492] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:19.376685 21521 ts_tablet_manager.cc:1403] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.154s	user 0.019s	sys 0.037s
I20260709 13:22:19.377216 21676 leader_election.cc:290] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:19.377915 21247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51766d5a86a6498b8e30249efa0ac990" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:19.378674 21247 raft_consensus.cc:2468] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
I20260709 13:22:19.379842 21521 raft_consensus.cc:359] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:19.380589 21521 raft_consensus.cc:385] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:19.380897 21521 raft_consensus.cc:740] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:19.380703 21363 leader_election.cc:304] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:19.381714 21521 consensus_queue.cc:260] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:19.382783 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51766d5a86a6498b8e30249efa0ac990" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:19.383574 21324 raft_consensus.cc:2468] T 51766d5a86a6498b8e30249efa0ac990 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
I20260709 13:22:19.383934 21521 ts_tablet_manager.cc:1434] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:22:19.383193 21676 raft_consensus.cc:2804] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:19.388773 21669 raft_consensus.cc:493] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:19.389611 21669 raft_consensus.cc:515] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:19.392554 21669 leader_election.cc:290] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:19.393659 21521 tablet_bootstrap.cc:492] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:19.385046 21676 raft_consensus.cc:493] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:19.400910 21250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e1cba0e20343aeb61c31c917dd1f50" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:19.396365 21669 consensus_queue.cc:1048] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:22:19.407693 21398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e1cba0e20343aeb61c31c917dd1f50" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:19.409822 21724 raft_consensus.cc:493] T a1bf171a09b745ec963d790f1eefe432 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:19.410451 21724 raft_consensus.cc:515] T a1bf171a09b745ec963d790f1eefe432 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:19.411504 21518 tablet_bootstrap.cc:654] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:19.399585 21676 raft_consensus.cc:3060] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:19.417186 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1bf171a09b745ec963d790f1eefe432" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:19.419747 21520 tablet_bootstrap.cc:492] T 37a3c02685464cf1af805559b46c4356 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:19.420261 21520 ts_tablet_manager.cc:1403] T 37a3c02685464cf1af805559b46c4356 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.195s	user 0.020s	sys 0.000s
I20260709 13:22:19.422803 21667 raft_consensus.cc:493] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:19.423502 21519 tablet_bootstrap.cc:492] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:19.423444 21667 raft_consensus.cc:515] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:19.424033 21519 ts_tablet_manager.cc:1403] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.171s	user 0.012s	sys 0.002s
I20260709 13:22:19.425508 21520 raft_consensus.cc:359] T 37a3c02685464cf1af805559b46c4356 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:19.425818 21521 tablet_bootstrap.cc:654] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:19.426271 21520 raft_consensus.cc:385] T 37a3c02685464cf1af805559b46c4356 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:19.426574 21520 raft_consensus.cc:740] T 37a3c02685464cf1af805559b46c4356 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:19.427287 21520 consensus_queue.cc:260] T 37a3c02685464cf1af805559b46c4356 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:19.431505 21724 leader_election.cc:290] T a1bf171a09b745ec963d790f1eefe432 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:19.432726 21247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1bf171a09b745ec963d790f1eefe432" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:19.427652 21519 raft_consensus.cc:359] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:19.434440 21519 raft_consensus.cc:385] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:19.434772 21519 raft_consensus.cc:740] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:19.435714 21520 ts_tablet_manager.cc:1434] T 37a3c02685464cf1af805559b46c4356 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:22:19.436990 21520 tablet_bootstrap.cc:492] T e4905b7b4db2444484e9403fed337486 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:19.438014 21519 consensus_queue.cc:260] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:19.451894 21519 ts_tablet_manager.cc:1434] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.001s
W20260709 13:22:19.414644 21437 leader_election.cc:343] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:19.453945 21519 tablet_bootstrap.cc:492] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
W20260709 13:22:19.455659 21439 leader_election.cc:343] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:19.459321 21196 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:19.453696 21289 leader_election.cc:304] T a1bf171a09b745ec963d790f1eefe432 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636
W20260709 13:22:19.461686 21437 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:35963, user_credentials={real_user=slave}} blocked reactor thread for 49228.8us
I20260709 13:22:19.462786 21762 raft_consensus.cc:2749] T a1bf171a09b745ec963d790f1eefe432 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:22:19.463181 21439 leader_election.cc:304] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636
I20260709 13:22:19.468278 21667 leader_election.cc:290] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:19.470515 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4905b7b4db2444484e9403fed337486" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:19.472260 21667 raft_consensus.cc:2749] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:19.474253 21635 raft_consensus.cc:493] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:19.474787 21635 raft_consensus.cc:515] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
W20260709 13:22:19.477872 21438 leader_election.cc:343] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:19.478344 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c6754d9a8e41ceb1fb416dcbbf0234" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:19.480561 21398 raft_consensus.cc:3060] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:19.480729 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4905b7b4db2444484e9403fed337486" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
W20260709 13:22:19.485915 21439 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:43181, user_credentials={real_user=slave}} blocked reactor thread for 69138.9us
I20260709 13:22:19.486266 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c6754d9a8e41ceb1fb416dcbbf0234" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:19.479028 21324 raft_consensus.cc:2468] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:19.487845 21669 consensus_queue.cc:1048] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:19.488248 21215 leader_election.cc:304] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:19.489205 21532 raft_consensus.cc:2804] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:19.489607 21532 raft_consensus.cc:493] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:19.489961 21532 raft_consensus.cc:3060] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:19.496641 21635 leader_election.cc:290] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:19.512768 21521 tablet_bootstrap.cc:492] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
W20260709 13:22:19.502796 21216 leader_election.cc:343] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:19.514138 21667 consensus_queue.cc:1048] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:19.517925 21521 ts_tablet_manager.cc:1403] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.120s	user 0.011s	sys 0.022s
I20260709 13:22:19.539852 21676 raft_consensus.cc:515] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:19.543223 21519 tablet_bootstrap.cc:654] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:19.544898 21521 raft_consensus.cc:359] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:19.545622 21521 raft_consensus.cc:385] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:19.545909 21521 raft_consensus.cc:740] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:19.546728 21521 consensus_queue.cc:260] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:19.550603 21676 leader_election.cc:290] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:19.552403 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51766d5a86a6498b8e30249efa0ac990" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:19.564631 21325 raft_consensus.cc:3060] T 51766d5a86a6498b8e30249efa0ac990 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:19.551863 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51766d5a86a6498b8e30249efa0ac990" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:19.569705 21251 raft_consensus.cc:3060] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:19.559912 21437 leader_election.cc:343] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:19.560563 21518 tablet_bootstrap.cc:492] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:19.565768 21521 ts_tablet_manager.cc:1434] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.001s
I20260709 13:22:19.594841 21518 ts_tablet_manager.cc:1403] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.222s	user 0.016s	sys 0.017s
I20260709 13:22:19.595341 21521 tablet_bootstrap.cc:492] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:19.601135 21667 consensus_queue.cc:1048] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:19.598805 21437 leader_election.cc:304] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
W20260709 13:22:19.604604 21437 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:35963, user_credentials={real_user=slave}} blocked reactor thread for 44738.6us
I20260709 13:22:19.605839 21669 raft_consensus.cc:2749] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:19.608552 21519 tablet_bootstrap.cc:492] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:19.609357 21519 ts_tablet_manager.cc:1403] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.156s	user 0.017s	sys 0.025s
I20260709 13:22:19.610532 21251 raft_consensus.cc:2468] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:19.615480 21521 tablet_bootstrap.cc:654] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:19.615741 21363 leader_election.cc:304] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:19.616600 21676 raft_consensus.cc:2804] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:19.613039 21520 tablet_bootstrap.cc:654] T e4905b7b4db2444484e9403fed337486 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:19.613137 21519 raft_consensus.cc:359] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:19.617456 21518 raft_consensus.cc:359] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:19.618026 21519 raft_consensus.cc:385] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:19.618599 21519 raft_consensus.cc:740] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:19.619817 21676 raft_consensus.cc:697] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Leader. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:19.619675 21519 consensus_queue.cc:260] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:19.620749 21676 consensus_queue.cc:237] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:19.622853 21519 ts_tablet_manager.cc:1434] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:22:19.618222 21518 raft_consensus.cc:385] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:19.625450 21518 raft_consensus.cc:740] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:19.626351 21518 consensus_queue.cc:260] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:19.628942 21518 ts_tablet_manager.cc:1434] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 13:22:19.629916 21518 tablet_bootstrap.cc:492] T cdc82236ff024407ae854c96dd8c89ec P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:19.637027 21667 raft_consensus.cc:493] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:19.637809 21667 raft_consensus.cc:515] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:19.636250 21518 tablet_bootstrap.cc:654] T cdc82236ff024407ae854c96dd8c89ec P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:19.640862 21667 leader_election.cc:290] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:19.642185 21519 tablet_bootstrap.cc:492] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:19.598978 21532 raft_consensus.cc:515] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:19.649600 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450b7111fe014a51b6e801869913ec52" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
W20260709 13:22:19.651774 21438 leader_election.cc:343] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:19.658725 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450b7111fe014a51b6e801869913ec52" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:19.661731 21532 leader_election.cc:290] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Requested vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:19.665738 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c6754d9a8e41ceb1fb416dcbbf0234" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:19.666662 21399 raft_consensus.cc:3060] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:19.672945 21437 leader_election.cc:343] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:19.679329 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c6754d9a8e41ceb1fb416dcbbf0234" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:19.679997 21324 raft_consensus.cc:3060] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:19.669328 21325 raft_consensus.cc:2468] T 51766d5a86a6498b8e30249efa0ac990 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:19.690205 21519 tablet_bootstrap.cc:654] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:19.691900 21437 leader_election.cc:304] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:19.705929 21324 raft_consensus.cc:2468] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:19.707913 21521 tablet_bootstrap.cc:492] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:19.708420 21521 ts_tablet_manager.cc:1403] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.113s	user 0.013s	sys 0.009s
I20260709 13:22:19.708050 21667 raft_consensus.cc:2749] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:19.718924 21399 raft_consensus.cc:2468] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:19.719648 21521 raft_consensus.cc:359] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:19.720371 21521 raft_consensus.cc:385] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:19.720629 21521 raft_consensus.cc:740] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:19.721268 21521 consensus_queue.cc:260] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:19.725458 21521 ts_tablet_manager.cc:1434] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.001s
I20260709 13:22:19.726085 21215 leader_election.cc:304] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:19.727293 21532 raft_consensus.cc:2804] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:19.727581 21520 tablet_bootstrap.cc:492] T e4905b7b4db2444484e9403fed337486 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:19.727730 21532 raft_consensus.cc:697] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:19.728108 21520 ts_tablet_manager.cc:1403] T e4905b7b4db2444484e9403fed337486 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.291s	user 0.017s	sys 0.010s
I20260709 13:22:19.728614 21532 consensus_queue.cc:237] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:19.730273 21519 tablet_bootstrap.cc:492] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:19.730932 21519 ts_tablet_manager.cc:1403] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.107s	user 0.014s	sys 0.005s
I20260709 13:22:19.731048 21520 raft_consensus.cc:359] T e4905b7b4db2444484e9403fed337486 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:19.731823 21518 tablet_bootstrap.cc:492] T cdc82236ff024407ae854c96dd8c89ec P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:19.731778 21520 raft_consensus.cc:385] T e4905b7b4db2444484e9403fed337486 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:19.733078 21520 raft_consensus.cc:740] T e4905b7b4db2444484e9403fed337486 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:19.733754 21520 consensus_queue.cc:260] T e4905b7b4db2444484e9403fed337486 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:19.733992 21519 raft_consensus.cc:359] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:19.734719 21519 raft_consensus.cc:385] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:19.734988 21519 raft_consensus.cc:740] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:19.735641 21519 consensus_queue.cc:260] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:19.737751 21519 ts_tablet_manager.cc:1434] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:22:19.738620 21519 tablet_bootstrap.cc:492] T 9805db96a0ab4ac3a686eb8ec904fb24 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:19.740481 21518 ts_tablet_manager.cc:1403] T cdc82236ff024407ae854c96dd8c89ec P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.111s	user 0.019s	sys 0.000s
I20260709 13:22:19.741189 21520 ts_tablet_manager.cc:1434] T e4905b7b4db2444484e9403fed337486 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:22:19.742170 21520 tablet_bootstrap.cc:492] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:19.746258 21251 raft_consensus.cc:3060] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:19.746533 21521 tablet_bootstrap.cc:492] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:19.750455 21520 tablet_bootstrap.cc:654] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:19.753347 21519 tablet_bootstrap.cc:654] T 9805db96a0ab4ac3a686eb8ec904fb24 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:19.755977 21518 raft_consensus.cc:359] T cdc82236ff024407ae854c96dd8c89ec P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:19.756690 21518 raft_consensus.cc:385] T cdc82236ff024407ae854c96dd8c89ec P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:19.756986 21521 tablet_bootstrap.cc:654] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:19.756964 21518 raft_consensus.cc:740] T cdc82236ff024407ae854c96dd8c89ec P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:19.758641 21518 consensus_queue.cc:260] T cdc82236ff024407ae854c96dd8c89ec P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:19.761333 21748 consensus_queue.cc:1048] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:22:19.765821 21724 raft_consensus.cc:493] T 37a3c02685464cf1af805559b46c4356 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:19.766378 21724 raft_consensus.cc:515] T 37a3c02685464cf1af805559b46c4356 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:19.768663 21724 leader_election.cc:290] T 37a3c02685464cf1af805559b46c4356 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:19.771171 21472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a3c02685464cf1af805559b46c4356" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:19.774658 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a3c02685464cf1af805559b46c4356" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:19.776777 21291 leader_election.cc:304] T 37a3c02685464cf1af805559b46c4356 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:19.752750 21146 catalog_manager.cc:5697] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "74e2ff1a68ac4c849ee6397ebb011375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:19.778438 21724 raft_consensus.cc:2749] T 37a3c02685464cf1af805559b46c4356 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:19.799868 21519 tablet_bootstrap.cc:492] T 9805db96a0ab4ac3a686eb8ec904fb24 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:19.800499 21519 ts_tablet_manager.cc:1403] T 9805db96a0ab4ac3a686eb8ec904fb24 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.062s	user 0.011s	sys 0.011s
I20260709 13:22:19.805359 21518 ts_tablet_manager.cc:1434] T cdc82236ff024407ae854c96dd8c89ec P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.065s	user 0.013s	sys 0.004s
I20260709 13:22:19.806447 21518 tablet_bootstrap.cc:492] T f4e1cba0e20343aeb61c31c917dd1f50 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:19.810473 21521 tablet_bootstrap.cc:492] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:19.811118 21521 ts_tablet_manager.cc:1403] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.065s	user 0.012s	sys 0.004s
I20260709 13:22:19.813081 21518 tablet_bootstrap.cc:654] T f4e1cba0e20343aeb61c31c917dd1f50 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:19.814188 21519 raft_consensus.cc:359] T 9805db96a0ab4ac3a686eb8ec904fb24 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:19.814070 21521 raft_consensus.cc:359] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:19.814877 21519 raft_consensus.cc:385] T 9805db96a0ab4ac3a686eb8ec904fb24 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:19.814992 21521 raft_consensus.cc:385] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:19.815259 21519 raft_consensus.cc:740] T 9805db96a0ab4ac3a686eb8ec904fb24 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:19.815407 21521 raft_consensus.cc:740] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:19.816166 21521 consensus_queue.cc:260] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:19.816001 21519 consensus_queue.cc:260] T 9805db96a0ab4ac3a686eb8ec904fb24 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:19.818805 21521 ts_tablet_manager.cc:1434] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:22:19.819857 21521 tablet_bootstrap.cc:492] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:19.821208 21519 ts_tablet_manager.cc:1434] T 9805db96a0ab4ac3a686eb8ec904fb24 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.002s
I20260709 13:22:19.822177 21519 tablet_bootstrap.cc:492] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:19.825989 21521 tablet_bootstrap.cc:654] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:19.828269 21519 tablet_bootstrap.cc:654] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:19.840268 21521 tablet_bootstrap.cc:492] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:19.840802 21521 ts_tablet_manager.cc:1403] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.000s
I20260709 13:22:19.843568 21521 raft_consensus.cc:359] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:19.844231 21521 raft_consensus.cc:385] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:19.844496 21521 raft_consensus.cc:740] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:19.844774 21324 raft_consensus.cc:3060] T 37a3c02685464cf1af805559b46c4356 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:19.845110 21521 consensus_queue.cc:260] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:19.847278 21521 ts_tablet_manager.cc:1434] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:22:19.848403 21521 tablet_bootstrap.cc:492] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:19.856395 21145 catalog_manager.cc:5697] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:19.857923 21521 tablet_bootstrap.cc:654] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:19.865908 21667 raft_consensus.cc:493] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:19.866711 21667 raft_consensus.cc:515] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:19.871040 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fe3abd2b2647d384d0e147d1932c0a" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
W20260709 13:22:19.874662 21439 leader_election.cc:343] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:19.880798 21667 leader_election.cc:290] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:19.884811 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fe3abd2b2647d384d0e147d1932c0a" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:19.888571 21532 raft_consensus.cc:493] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:19.915331 21532 raft_consensus.cc:515] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:19.918615 21532 leader_election.cc:290] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:19.923344 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450b7111fe014a51b6e801869913ec52" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
W20260709 13:22:19.925529 21215 leader_election.cc:343] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:19.943790 21521 tablet_bootstrap.cc:492] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:19.944416 21521 ts_tablet_manager.cc:1403] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.096s	user 0.019s	sys 0.026s
I20260709 13:22:19.947183 21521 raft_consensus.cc:359] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:19.947849 21521 raft_consensus.cc:385] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:19.948132 21521 raft_consensus.cc:740] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:19.948931 21521 consensus_queue.cc:260] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
W20260709 13:22:19.949419 21437 leader_election.cc:343] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:19.950011 21437 leader_election.cc:304] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636
I20260709 13:22:19.951015 21667 raft_consensus.cc:2749] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:19.952392 21472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450b7111fe014a51b6e801869913ec52" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:19.953372 21472 raft_consensus.cc:2468] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:19.956629 21521 ts_tablet_manager.cc:1434] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:22:19.959645 21521 tablet_bootstrap.cc:492] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:19.970316 21521 tablet_bootstrap.cc:654] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:19.979336 21808 raft_consensus.cc:493] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:19.981730 21808 raft_consensus.cc:515] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:19.993273 21532 raft_consensus.cc:493] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:19.993978 21532 raft_consensus.cc:515] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:19.997668 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc40f11dce5540d1a056e9bacba075e2" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
W20260709 13:22:20.013012 21437 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:35963, user_credentials={real_user=slave}} blocked reactor thread for 61131.8us
I20260709 13:22:20.014341 21214 leader_election.cc:304] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:20.017238 21667 raft_consensus.cc:493] T 9805db96a0ab4ac3a686eb8ec904fb24 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.018072 21667 raft_consensus.cc:515] T 9805db96a0ab4ac3a686eb8ec904fb24 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:20.021685 21518 tablet_bootstrap.cc:492] T f4e1cba0e20343aeb61c31c917dd1f50 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:20.022300 21527 raft_consensus.cc:493] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.022997 21532 leader_election.cc:290] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:20.023597 21527 raft_consensus.cc:515] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:20.024650 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9805db96a0ab4ac3a686eb8ec904fb24" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:20.026253 21527 leader_election.cc:290] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:20.022326 21808 leader_election.cc:290] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
W20260709 13:22:20.026996 21439 leader_election.cc:343] T 9805db96a0ab4ac3a686eb8ec904fb24 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:20.022241 21518 ts_tablet_manager.cc:1403] T f4e1cba0e20343aeb61c31c917dd1f50 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.216s	user 0.016s	sys 0.023s
I20260709 13:22:20.028625 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc82236ff024407ae854c96dd8c89ec" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:20.023734 21532 raft_consensus.cc:2804] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:20.029457 21399 raft_consensus.cc:2468] T cdc82236ff024407ae854c96dd8c89ec P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 0.
I20260709 13:22:20.029757 21532 raft_consensus.cc:493] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:20.030356 21532 raft_consensus.cc:3060] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:20.031611 21439 leader_election.cc:304] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:20.034415 21740 raft_consensus.cc:2804] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:20.034821 21740 raft_consensus.cc:493] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:20.035192 21740 raft_consensus.cc:3060] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:20.036001 21518 raft_consensus.cc:359] T f4e1cba0e20343aeb61c31c917dd1f50 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:20.040724 21521 tablet_bootstrap.cc:492] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:20.041653 21521 ts_tablet_manager.cc:1403] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.084s	user 0.015s	sys 0.004s
I20260709 13:22:20.046856 21521 raft_consensus.cc:359] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:20.047653 21521 raft_consensus.cc:385] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:20.047397 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9805db96a0ab4ac3a686eb8ec904fb24" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:20.048003 21521 raft_consensus.cc:740] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:20.048839 21521 consensus_queue.cc:260] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:20.061908 21521 ts_tablet_manager.cc:1434] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:22:20.049214 21740 raft_consensus.cc:515] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:20.023659 21667 leader_election.cc:290] T 9805db96a0ab4ac3a686eb8ec904fb24 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:20.049918 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4905b7b4db2444484e9403fed337486" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:20.053737 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc82236ff024407ae854c96dd8c89ec" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
W20260709 13:22:20.057861 21438 leader_election.cc:343] T 9805db96a0ab4ac3a686eb8ec904fb24 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:20.037249 21518 raft_consensus.cc:385] T f4e1cba0e20343aeb61c31c917dd1f50 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:20.072578 21518 raft_consensus.cc:740] T f4e1cba0e20343aeb61c31c917dd1f50 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:20.073859 21532 raft_consensus.cc:515] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:20.075261 21521 tablet_bootstrap.cc:492] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:20.073544 21518 consensus_queue.cc:260] T f4e1cba0e20343aeb61c31c917dd1f50 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:20.075399 21438 leader_election.cc:304] T 9805db96a0ab4ac3a686eb8ec904fb24 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:20.084138 21667 raft_consensus.cc:2749] T 9805db96a0ab4ac3a686eb8ec904fb24 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:20.068325 21324 raft_consensus.cc:2468] T e4905b7b4db2444484e9403fed337486 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:20.085023 21687 raft_consensus.cc:493] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:22:20.062134 21363 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36863, user_credentials={real_user=slave}} blocked reactor thread for 302971us
I20260709 13:22:20.085673 21687 raft_consensus.cc:515] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:20.088233 21687 leader_election.cc:290] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:20.089099 21215 leader_election.cc:304] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:20.090127 21687 raft_consensus.cc:2804] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:20.090569 21687 raft_consensus.cc:493] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:20.090747 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fe3abd2b2647d384d0e147d1932c0a" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:20.090984 21687 raft_consensus.cc:3060] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:20.092386 21216 leader_election.cc:343] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:20.094533 21438 leader_election.cc:343] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:20.095288 21519 tablet_bootstrap.cc:492] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:20.100787 21519 ts_tablet_manager.cc:1403] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.279s	user 0.047s	sys 0.099s
I20260709 13:22:20.097895 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc82236ff024407ae854c96dd8c89ec" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
W20260709 13:22:20.103211 21438 leader_election.cc:343] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:20.096200 21740 leader_election.cc:290] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Requested vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:20.101800 21518 ts_tablet_manager.cc:1434] T f4e1cba0e20343aeb61c31c917dd1f50 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.074s	user 0.004s	sys 0.004s
I20260709 13:22:20.101505 21800 consensus_queue.cc:1048] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:20.105202 21518 tablet_bootstrap.cc:492] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:20.113513 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc82236ff024407ae854c96dd8c89ec" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:20.114375 21396 raft_consensus.cc:3060] T cdc82236ff024407ae854c96dd8c89ec P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:20.117519 21635 consensus_queue.cc:1048] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:20.121341 21635 raft_consensus.cc:493] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.122432 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc40f11dce5540d1a056e9bacba075e2" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:20.122865 21469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fe3abd2b2647d384d0e147d1932c0a" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:20.123190 21474 raft_consensus.cc:2468] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:20.123624 21469 raft_consensus.cc:2468] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:20.123232 21472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4905b7b4db2444484e9403fed337486" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:20.124605 21472 raft_consensus.cc:2468] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:20.125371 21470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450b7111fe014a51b6e801869913ec52" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:20.125773 21521 tablet_bootstrap.cc:654] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:20.121941 21635 raft_consensus.cc:515] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:20.126065 21470 raft_consensus.cc:3060] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:20.126986 21532 leader_election.cc:290] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:20.127902 21635 leader_election.cc:290] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:20.129943 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a6063e86146e8ba5f721d593edf0a" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
W20260709 13:22:20.131672 21216 leader_election.cc:343] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:20.134780 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450b7111fe014a51b6e801869913ec52" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:20.150794 21520 tablet_bootstrap.cc:492] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:20.151621 21520 ts_tablet_manager.cc:1403] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.410s	user 0.102s	sys 0.113s
I20260709 13:22:20.153967 21214 leader_election.cc:304] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:20.155990 21214 leader_election.cc:304] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:20.166942 21470 raft_consensus.cc:2468] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:20.141220 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a6063e86146e8ba5f721d593edf0a" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:20.161720 21635 raft_consensus.cc:2804] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:22:20.160524 21215 leader_election.cc:343] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:20.169736 21800 consensus_queue.cc:1048] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 13:22:20.167917 21687 raft_consensus.cc:515] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:20.186627 21635 raft_consensus.cc:493] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:20.188591 21635 raft_consensus.cc:3060] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:20.192432 21215 leader_election.cc:343] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:20.193327 21215 leader_election.cc:304] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:20.194903 21808 raft_consensus.cc:2749] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:20.176465 21520 raft_consensus.cc:359] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:20.197156 21520 raft_consensus.cc:385] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:20.197588 21520 raft_consensus.cc:740] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:20.169099 21519 raft_consensus.cc:359] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:20.198582 21532 raft_consensus.cc:2804] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:20.198941 21519 raft_consensus.cc:385] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:20.199363 21519 raft_consensus.cc:740] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:20.182271 21396 raft_consensus.cc:2468] T cdc82236ff024407ae854c96dd8c89ec P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:20.200074 21519 consensus_queue.cc:260] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:20.201122 21532 raft_consensus.cc:493] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 74e2ff1a68ac4c849ee6397ebb011375)
I20260709 13:22:20.201764 21532 raft_consensus.cc:3060] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:20.201661 21439 leader_election.cc:304] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:20.203388 21520 consensus_queue.cc:260] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:20.217468 21519 ts_tablet_manager.cc:1434] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.113s	user 0.011s	sys 0.000s
I20260709 13:22:20.219048 21519 tablet_bootstrap.cc:492] T 033f24b2a37f421cbea89f5c5c61d0a3 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:20.219527 21518 tablet_bootstrap.cc:654] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:20.212289 21740 raft_consensus.cc:2804] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:20.221323 21740 raft_consensus.cc:697] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:20.222332 21740 consensus_queue.cc:237] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:20.225136 21520 ts_tablet_manager.cc:1434] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.073s	user 0.012s	sys 0.000s
I20260709 13:22:20.204174 21687 leader_election.cc:290] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:20.190346 21214 leader_election.cc:304] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:20.241041 21635 raft_consensus.cc:515] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:20.274264 21521 tablet_bootstrap.cc:492] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:20.274814 21521 ts_tablet_manager.cc:1403] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.202s	user 0.016s	sys 0.033s
I20260709 13:22:20.275755 21532 raft_consensus.cc:515] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:20.282502 21532 leader_election.cc:290] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 2 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:20.253659 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4905b7b4db2444484e9403fed337486" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:20.270998 21520 tablet_bootstrap.cc:492] T 6898b14b98d944278cfd7cb32d964074 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:20.286794 21247 raft_consensus.cc:1240] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Rejecting Update request from peer 74e2ff1a68ac4c849ee6397ebb011375 for earlier term 1. Current term is 2. Ops: []
W20260709 13:22:20.278786 21214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36863, user_credentials={real_user=slave}} blocked reactor thread for 105893us
I20260709 13:22:20.290843 21324 raft_consensus.cc:3060] T e4905b7b4db2444484e9403fed337486 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:20.301697 21519 tablet_bootstrap.cc:654] T 033f24b2a37f421cbea89f5c5c61d0a3 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:20.312282 21521 raft_consensus.cc:359] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:20.313053 21521 raft_consensus.cc:385] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:20.313333 21521 raft_consensus.cc:740] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:20.331636 21521 consensus_queue.cc:260] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:20.309348 21808 raft_consensus.cc:2804] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:20.331070 21748 consensus_queue.cc:1048] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:22:20.341722 21635 leader_election.cc:290] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:20.343981 21521 ts_tablet_manager.cc:1434] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.069s	user 0.007s	sys 0.000s
I20260709 13:22:20.346030 21518 tablet_bootstrap.cc:492] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:20.346551 21518 ts_tablet_manager.cc:1403] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.242s	user 0.012s	sys 0.009s
I20260709 13:22:20.349036 21808 raft_consensus.cc:697] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:20.350339 21808 consensus_queue.cc:237] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:20.400966 21518 raft_consensus.cc:359] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:20.403272 21518 raft_consensus.cc:385] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:20.403693 21518 raft_consensus.cc:740] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:20.414907 21520 tablet_bootstrap.cc:654] T 6898b14b98d944278cfd7cb32d964074 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:20.424455 21521 tablet_bootstrap.cc:492] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:20.424710 21469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fe3abd2b2647d384d0e147d1932c0a" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:20.425910 21469 raft_consensus.cc:3060] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:20.404772 21518 consensus_queue.cc:260] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:20.469475 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4905b7b4db2444484e9403fed337486" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:20.470785 21518 ts_tablet_manager.cc:1434] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.124s	user 0.002s	sys 0.011s
W20260709 13:22:20.472549 21216 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:43181, user_credentials={real_user=slave}} blocked reactor thread for 229313us
I20260709 13:22:20.470745 21474 raft_consensus.cc:3060] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:20.473336 21518 tablet_bootstrap.cc:492] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:20.468302 21800 raft_consensus.cc:493] T f4e1cba0e20343aeb61c31c917dd1f50 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.513535 21472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc40f11dce5540d1a056e9bacba075e2" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:20.520898 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fe3abd2b2647d384d0e147d1932c0a" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:20.521756 21396 raft_consensus.cc:3060] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:20.527860 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc40f11dce5540d1a056e9bacba075e2" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:20.536273 21396 raft_consensus.cc:2468] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:20.544204 21823 consensus_queue.cc:1059] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:22:20.547199 21821 raft_consensus.cc:493] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.548276 21821 raft_consensus.cc:515] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:20.551571 21216 leader_election.cc:304] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:20.557111 21822 raft_consensus.cc:2804] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:20.557804 21822 raft_consensus.cc:697] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:20.559104 21822 consensus_queue.cc:237] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:20.564702 21521 tablet_bootstrap.cc:654] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:20.568845 21800 raft_consensus.cc:515] T f4e1cba0e20343aeb61c31c917dd1f50 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:20.571749 21800 leader_election.cc:290] T f4e1cba0e20343aeb61c31c917dd1f50 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:20.572616 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6898b14b98d944278cfd7cb32d964074" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:20.545472 21519 tablet_bootstrap.cc:492] T 033f24b2a37f421cbea89f5c5c61d0a3 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:20.568301 21821 leader_election.cc:290] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:20.602473 21519 ts_tablet_manager.cc:1403] T 033f24b2a37f421cbea89f5c5c61d0a3 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.384s	user 0.014s	sys 0.006s
I20260709 13:22:20.608767 21829 raft_consensus.cc:3055] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:20.609604 21829 raft_consensus.cc:740] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:20.618309 21519 raft_consensus.cc:359] T 033f24b2a37f421cbea89f5c5c61d0a3 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:20.619433 21519 raft_consensus.cc:385] T 033f24b2a37f421cbea89f5c5c61d0a3 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:20.619872 21519 raft_consensus.cc:740] T 033f24b2a37f421cbea89f5c5c61d0a3 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
W20260709 13:22:20.624923 21437 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:35963, user_credentials={real_user=slave}} blocked reactor thread for 46875.5us
I20260709 13:22:20.624425 21145 catalog_manager.cc:5697] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 1 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:20.640583 21518 tablet_bootstrap.cc:654] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
W20260709 13:22:20.642714 21216 leader_election.cc:343] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:20.634635 21829 consensus_queue.cc:260] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:20.637182 21214 leader_election.cc:304] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec879c0528c44465916efc6c33dde636; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:20.645794 21824 raft_consensus.cc:2749] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:22:20.654620 21469 raft_consensus.cc:2468] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:20.662611 21687 raft_consensus.cc:493] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.663925 21687 raft_consensus.cc:515] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:20.667774 21687 leader_election.cc:290] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:20.672127 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e1cba0e20343aeb61c31c917dd1f50" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:20.673164 21399 raft_consensus.cc:2468] T f4e1cba0e20343aeb61c31c917dd1f50 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:20.680431 21216 leader_election.cc:304] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:20.680469 21323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6898b14b98d944278cfd7cb32d964074" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:20.681967 21827 raft_consensus.cc:2804] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:20.682514 21827 raft_consensus.cc:493] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:20.682958 21827 raft_consensus.cc:3060] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:20.687690 21474 raft_consensus.cc:2468] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:20.613073 21324 raft_consensus.cc:2468] T e4905b7b4db2444484e9403fed337486 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:20.620898 21519 consensus_queue.cc:260] T 033f24b2a37f421cbea89f5c5c61d0a3 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:20.721819 21827 raft_consensus.cc:515] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:20.750245 21519 ts_tablet_manager.cc:1434] T 033f24b2a37f421cbea89f5c5c61d0a3 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.147s	user 0.005s	sys 0.007s
I20260709 13:22:20.752010 21519 tablet_bootstrap.cc:492] T d2bf69be9e9240b5ac99adc49258432e P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:20.761814 21829 raft_consensus.cc:3060] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:22:20.739715 21363 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:35963, user_credentials={real_user=slave}} blocked reactor thread for 274476us
I20260709 13:22:20.772575 21687 consensus_queue.cc:1048] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:22:20.778475 21827 leader_election.cc:290] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:20.786831 21470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e1cba0e20343aeb61c31c917dd1f50" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:20.787712 21470 raft_consensus.cc:3060] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:20.788045 21521 tablet_bootstrap.cc:492] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:20.788555 21521 ts_tablet_manager.cc:1403] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.365s	user 0.017s	sys 0.007s
I20260709 13:22:20.788909 21469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e1cba0e20343aeb61c31c917dd1f50" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:20.791419 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e1cba0e20343aeb61c31c917dd1f50" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:20.792102 21399 raft_consensus.cc:3060] T f4e1cba0e20343aeb61c31c917dd1f50 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:20.792423 21214 leader_election.cc:304] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:20.795480 21687 raft_consensus.cc:2804] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:20.796031 21687 raft_consensus.cc:697] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:20.797264 21687 consensus_queue.cc:237] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:20.798907 21767 raft_consensus.cc:493] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.799641 21767 raft_consensus.cc:515] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:20.804589 21767 leader_election.cc:290] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:20.807700 21740 raft_consensus.cc:493] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.808353 21740 raft_consensus.cc:515] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:20.811286 21740 leader_election.cc:290] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
W20260709 13:22:20.807783 21215 leader_election.cc:343] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:20.812198 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e1cba0e20343aeb61c31c917dd1f50" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:20.812520 21215 leader_election.cc:304] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:20.813074 21251 raft_consensus.cc:2393] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74e2ff1a68ac4c849ee6397ebb011375 in current term 1: Already voted for candidate ec879c0528c44465916efc6c33dde636 in this term.
I20260709 13:22:20.813125 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ddeabcec9b64fc68b40c70df90b80ef" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:20.814502 21808 raft_consensus.cc:2749] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:20.820102 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e1cba0e20343aeb61c31c917dd1f50" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:20.828698 21399 raft_consensus.cc:2468] T f4e1cba0e20343aeb61c31c917dd1f50 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
W20260709 13:22:20.829993 21438 consensus_peers.cc:597] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:20.832646 21520 tablet_bootstrap.cc:492] T 6898b14b98d944278cfd7cb32d964074 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:20.818584 21521 raft_consensus.cc:359] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:20.835078 21521 raft_consensus.cc:385] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:20.835537 21521 raft_consensus.cc:740] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:20.834014 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9805db96a0ab4ac3a686eb8ec904fb24" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:20.847991 21518 tablet_bootstrap.cc:492] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:20.848739 21518 ts_tablet_manager.cc:1403] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.376s	user 0.024s	sys 0.017s
I20260709 13:22:20.865330 21520 ts_tablet_manager.cc:1403] T 6898b14b98d944278cfd7cb32d964074 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.595s	user 0.049s	sys 0.057s
I20260709 13:22:20.838876 21474 raft_consensus.cc:2468] T 9805db96a0ab4ac3a686eb8ec904fb24 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 0.
I20260709 13:22:20.871109 21289 leader_election.cc:304] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 
I20260709 13:22:20.872722 21832 raft_consensus.cc:2804] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:20.873143 21832 raft_consensus.cc:493] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:20.873510 21832 raft_consensus.cc:3060] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:20.875265 21437 leader_election.cc:343] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:20.887688 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9805db96a0ab4ac3a686eb8ec904fb24" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:20.890405 21521 consensus_queue.cc:260] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:20.891198 21740 consensus_queue.cc:1048] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:20.894363 21521 ts_tablet_manager.cc:1434] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.106s	user 0.009s	sys 0.000s
I20260709 13:22:20.894536 21363 leader_election.cc:304] T f4e1cba0e20343aeb61c31c917dd1f50 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636
I20260709 13:22:20.895619 21834 raft_consensus.cc:2749] T f4e1cba0e20343aeb61c31c917dd1f50 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:20.853019 21518 raft_consensus.cc:359] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:20.897465 21518 raft_consensus.cc:385] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:20.897791 21518 raft_consensus.cc:740] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:20.898664 21518 consensus_queue.cc:260] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:20.898931 21832 raft_consensus.cc:515] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
W20260709 13:22:20.899802 21291 leader_election.cc:343] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:20.901782 21322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ddeabcec9b64fc68b40c70df90b80ef" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:20.902293 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9805db96a0ab4ac3a686eb8ec904fb24" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:20.902925 21470 raft_consensus.cc:2468] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:20.903113 21474 raft_consensus.cc:3060] T 9805db96a0ab4ac3a686eb8ec904fb24 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:20.915763 21832 leader_election.cc:290] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:20.917264 21827 raft_consensus.cc:493] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:20.929828 21518 ts_tablet_manager.cc:1434] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.081s	user 0.007s	sys 0.004s
I20260709 13:22:20.932721 21520 raft_consensus.cc:359] T 6898b14b98d944278cfd7cb32d964074 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:20.939913 21520 raft_consensus.cc:385] T 6898b14b98d944278cfd7cb32d964074 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:20.940215 21520 raft_consensus.cc:740] T 6898b14b98d944278cfd7cb32d964074 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:20.937044 21214 leader_election.cc:304] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:20.943070 21831 raft_consensus.cc:2804] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:20.943553 21831 raft_consensus.cc:697] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
W20260709 13:22:20.937168 21438 leader_election.cc:343] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:20.944597 21438 leader_election.cc:304] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:20.944738 21831 consensus_queue.cc:237] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:20.945604 21740 raft_consensus.cc:2749] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:20.943383 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9805db96a0ab4ac3a686eb8ec904fb24" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:20.960513 21521 tablet_bootstrap.cc:492] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:20.933792 21827 raft_consensus.cc:515] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
W20260709 13:22:20.966645 21291 leader_election.cc:343] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:20.975669 21827 leader_election.cc:290] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:20.977332 21470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ddeabcec9b64fc68b40c70df90b80ef" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:20.978176 21470 raft_consensus.cc:2468] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:20.967504 21520 consensus_queue.cc:260] T 6898b14b98d944278cfd7cb32d964074 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:20.981087 21323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ddeabcec9b64fc68b40c70df90b80ef" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:20.981585 21520 ts_tablet_manager.cc:1434] T 6898b14b98d944278cfd7cb32d964074 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.116s	user 0.006s	sys 0.002s
I20260709 13:22:20.983126 21518 tablet_bootstrap.cc:492] T 9805db96a0ab4ac3a686eb8ec904fb24 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:20.983340 21519 tablet_bootstrap.cc:654] T d2bf69be9e9240b5ac99adc49258432e P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
W20260709 13:22:20.983794 21215 leader_election.cc:343] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:20.983175 21520 tablet_bootstrap.cc:492] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:20.991379 21474 raft_consensus.cc:2468] T 9805db96a0ab4ac3a686eb8ec904fb24 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 1.
I20260709 13:22:20.998823 21289 leader_election.cc:304] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:20.997283 21214 leader_election.cc:304] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:21.013831 21833 raft_consensus.cc:2804] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:21.014550 21833 raft_consensus.cc:697] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 LEADER]: Becoming Leader. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Running, Role: LEADER
I20260709 13:22:21.017765 21831 raft_consensus.cc:2804] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:21.022640 21831 raft_consensus.cc:493] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:21.023101 21831 raft_consensus.cc:3060] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:21.028734 21520 tablet_bootstrap.cc:654] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:21.015633 21833 consensus_queue.cc:237] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:21.049911 21831 raft_consensus.cc:515] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:21.062593 21519 tablet_bootstrap.cc:492] T d2bf69be9e9240b5ac99adc49258432e P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:21.063192 21519 ts_tablet_manager.cc:1403] T d2bf69be9e9240b5ac99adc49258432e P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.312s	user 0.014s	sys 0.006s
I20260709 13:22:21.082921 21519 raft_consensus.cc:359] T d2bf69be9e9240b5ac99adc49258432e P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:21.083670 21519 raft_consensus.cc:385] T d2bf69be9e9240b5ac99adc49258432e P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:21.083930 21519 raft_consensus.cc:740] T d2bf69be9e9240b5ac99adc49258432e P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:21.095171 21521 tablet_bootstrap.cc:654] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:21.089419 21822 consensus_queue.cc:1048] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:22:21.108295 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ddeabcec9b64fc68b40c70df90b80ef" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:21.108958 21474 raft_consensus.cc:3060] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:21.098888 21519 consensus_queue.cc:260] T d2bf69be9e9240b5ac99adc49258432e P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:21.112929 21518 tablet_bootstrap.cc:654] T 9805db96a0ab4ac3a686eb8ec904fb24 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:21.111475 21147 catalog_manager.cc:5697] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:21.114228 21830 raft_consensus.cc:493] T d2bf69be9e9240b5ac99adc49258432e P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:21.115458 21830 raft_consensus.cc:515] T d2bf69be9e9240b5ac99adc49258432e P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:21.114850 21667 raft_consensus.cc:493] T 033f24b2a37f421cbea89f5c5c61d0a3 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:21.116271 21147 catalog_manager.cc:5697] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:21.110059 21827 consensus_queue.cc:1048] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:21.119318 21834 raft_consensus.cc:493] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:21.126605 21834 raft_consensus.cc:515] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:21.117794 21667 raft_consensus.cc:515] T 033f24b2a37f421cbea89f5c5c61d0a3 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:21.130920 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6898b14b98d944278cfd7cb32d964074" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:21.131693 21251 raft_consensus.cc:2468] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
I20260709 13:22:21.132879 21667 leader_election.cc:290] T 033f24b2a37f421cbea89f5c5c61d0a3 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:21.133160 21323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033f24b2a37f421cbea89f5c5c61d0a3" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:21.133574 21363 leader_election.cc:304] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:21.136833 21830 leader_election.cc:290] T d2bf69be9e9240b5ac99adc49258432e P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
W20260709 13:22:21.137055 21438 leader_election.cc:343] T 033f24b2a37f421cbea89f5c5c61d0a3 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:21.139015 21823 raft_consensus.cc:2804] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:21.142825 21823 raft_consensus.cc:493] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:21.143299 21823 raft_consensus.cc:3060] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:21.150295 21834 leader_election.cc:290] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:21.151629 21323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bf69be9e9240b5ac99adc49258432e" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:21.152684 21519 ts_tablet_manager.cc:1434] T d2bf69be9e9240b5ac99adc49258432e P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.089s	user 0.005s	sys 0.004s
I20260709 13:22:21.160192 21519 tablet_bootstrap.cc:492] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:21.169339 21521 tablet_bootstrap.cc:492] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:21.170116 21521 ts_tablet_manager.cc:1403] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.231s	user 0.017s	sys 0.001s
I20260709 13:22:21.180261 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ddeabcec9b64fc68b40c70df90b80ef" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:21.157074 21823 raft_consensus.cc:515] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:21.185284 21823 leader_election.cc:290] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:21.180338 21822 consensus_queue.cc:1048] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:22:21.186651 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6898b14b98d944278cfd7cb32d964074" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
W20260709 13:22:21.187245 21215 leader_election.cc:343] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:21.187323 21251 raft_consensus.cc:3060] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:21.189128 21215 consensus_peers.cc:597] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:21.194962 21519 tablet_bootstrap.cc:654] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:21.196403 21251 raft_consensus.cc:2468] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:21.183553 21147 catalog_manager.cc:5697] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:21.197883 21363 leader_election.cc:304] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:21.198868 21834 raft_consensus.cc:2804] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:21.198261 21147 catalog_manager.cc:5697] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:21.199364 21834 raft_consensus.cc:697] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Leader. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:21.200209 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033f24b2a37f421cbea89f5c5c61d0a3" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:21.167922 21831 leader_election.cc:290] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:21.207088 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6898b14b98d944278cfd7cb32d964074" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:21.200222 21834 consensus_queue.cc:237] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
W20260709 13:22:21.183080 21438 leader_election.cc:343] T d2bf69be9e9240b5ac99adc49258432e P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:21.207975 21325 raft_consensus.cc:3060] T 6898b14b98d944278cfd7cb32d964074 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:21.210278 21439 leader_election.cc:343] T 033f24b2a37f421cbea89f5c5c61d0a3 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:21.210829 21439 leader_election.cc:304] T 033f24b2a37f421cbea89f5c5c61d0a3 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:21.181610 21323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6898b14b98d944278cfd7cb32d964074" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:21.201550 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bf69be9e9240b5ac99adc49258432e" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:21.207219 21474 raft_consensus.cc:2468] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:21.222535 21214 leader_election.cc:304] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:21.224311 21822 raft_consensus.cc:2804] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:21.224792 21822 raft_consensus.cc:697] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:21.225795 21822 consensus_queue.cc:237] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:21.226961 21325 raft_consensus.cc:2468] T 6898b14b98d944278cfd7cb32d964074 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
W20260709 13:22:21.227504 21439 leader_election.cc:343] T d2bf69be9e9240b5ac99adc49258432e P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:21.228405 21439 leader_election.cc:304] T d2bf69be9e9240b5ac99adc49258432e P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:21.232318 21740 raft_consensus.cc:2749] T d2bf69be9e9240b5ac99adc49258432e P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:21.236490 21521 raft_consensus.cc:359] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:21.240005 21520 tablet_bootstrap.cc:492] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:21.240604 21520 ts_tablet_manager.cc:1403] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.258s	user 0.013s	sys 0.013s
I20260709 13:22:21.243862 21520 raft_consensus.cc:359] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:21.244752 21520 raft_consensus.cc:385] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:21.245118 21520 raft_consensus.cc:740] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:21.237422 21521 raft_consensus.cc:385] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:21.215834 21667 raft_consensus.cc:2749] T 033f24b2a37f421cbea89f5c5c61d0a3 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:21.252272 21146 catalog_manager.cc:5697] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66). New cstate: current_term: 1 leader_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:21.253490 21520 consensus_queue.cc:260] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:21.249531 21521 raft_consensus.cc:740] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:21.273063 21808 consensus_queue.cc:1048] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:21.267033 21521 consensus_queue.cc:260] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:21.309012 21520 ts_tablet_manager.cc:1434] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.068s	user 0.006s	sys 0.002s
I20260709 13:22:21.305578 21831 raft_consensus.cc:493] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:21.320614 21521 ts_tablet_manager.cc:1434] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.149s	user 0.005s	sys 0.006s
I20260709 13:22:21.321863 21831 raft_consensus.cc:515] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:21.322554 21520 tablet_bootstrap.cc:492] T cdc82236ff024407ae854c96dd8c89ec P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:21.324183 21831 leader_election.cc:290] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:21.326237 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcf55c037e6c48bc9d73f87a54b26f92" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:21.326622 21521 tablet_bootstrap.cc:492] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
W20260709 13:22:21.339692 21216 leader_election.cc:343] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:21.336392 21836 raft_consensus.cc:493] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:21.349686 21836 raft_consensus.cc:515] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:21.348346 21469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcf55c037e6c48bc9d73f87a54b26f92" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:21.357357 21472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450b7111fe014a51b6e801869913ec52" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:21.359500 21808 consensus_queue.cc:1048] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:22:21.361627 21214 leader_election.cc:343] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:21.362105 21214 leader_election.cc:304] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec879c0528c44465916efc6c33dde636; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:21.363191 21808 raft_consensus.cc:2749] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:21.374354 21836 leader_election.cc:290] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:21.376706 21519 tablet_bootstrap.cc:492] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:21.380916 21519 ts_tablet_manager.cc:1403] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.221s	user 0.015s	sys 0.011s
I20260709 13:22:21.383900 21519 raft_consensus.cc:359] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:21.384851 21519 raft_consensus.cc:385] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:21.385226 21519 raft_consensus.cc:740] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:21.387991 21519 consensus_queue.cc:260] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:21.390529 21519 ts_tablet_manager.cc:1434] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:22:21.391383 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450b7111fe014a51b6e801869913ec52" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:21.394093 21808 consensus_queue.cc:1048] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:21.398694 21289 leader_election.cc:304] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636
I20260709 13:22:21.399550 21832 raft_consensus.cc:2749] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:21.399816 21520 tablet_bootstrap.cc:654] T cdc82236ff024407ae854c96dd8c89ec P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
W20260709 13:22:21.399964 21196 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:21.404088 21518 tablet_bootstrap.cc:492] T 9805db96a0ab4ac3a686eb8ec904fb24 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:21.404702 21518 ts_tablet_manager.cc:1403] T 9805db96a0ab4ac3a686eb8ec904fb24 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.441s	user 0.026s	sys 0.020s
I20260709 13:22:21.422092 21521 tablet_bootstrap.cc:654] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:21.422359 21519 tablet_bootstrap.cc:492] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:21.424922 21518 raft_consensus.cc:359] T 9805db96a0ab4ac3a686eb8ec904fb24 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:21.425830 21518 raft_consensus.cc:385] T 9805db96a0ab4ac3a686eb8ec904fb24 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:21.426146 21518 raft_consensus.cc:740] T 9805db96a0ab4ac3a686eb8ec904fb24 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:21.427449 21518 consensus_queue.cc:260] T 9805db96a0ab4ac3a686eb8ec904fb24 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:21.434842 21519 tablet_bootstrap.cc:654] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:21.432797 21148 catalog_manager.cc:5697] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:21.434099 21146 catalog_manager.cc:5697] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "74e2ff1a68ac4c849ee6397ebb011375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:21.443627 21518 ts_tablet_manager.cc:1434] T 9805db96a0ab4ac3a686eb8ec904fb24 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.001s
I20260709 13:22:21.445015 21518 tablet_bootstrap.cc:492] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:21.457428 21519 tablet_bootstrap.cc:492] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:21.457904 21518 tablet_bootstrap.cc:654] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:21.457994 21519 ts_tablet_manager.cc:1403] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.037s	user 0.017s	sys 0.003s
I20260709 13:22:21.461166 21519 raft_consensus.cc:359] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:21.461884 21519 raft_consensus.cc:385] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:21.462149 21519 raft_consensus.cc:740] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:21.462882 21519 consensus_queue.cc:260] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:21.465262 21519 ts_tablet_manager.cc:1434] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:22:21.466358 21519 tablet_bootstrap.cc:492] T 99478ce52e24419ea31804ba62f7a46a P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:21.477138 21519 tablet_bootstrap.cc:654] T 99478ce52e24419ea31804ba62f7a46a P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:21.481272 21518 tablet_bootstrap.cc:492] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:21.481887 21518 ts_tablet_manager.cc:1403] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.037s	user 0.014s	sys 0.003s
I20260709 13:22:21.485596 21518 raft_consensus.cc:359] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:21.486549 21518 raft_consensus.cc:385] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:21.486872 21518 raft_consensus.cc:740] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:21.487761 21518 consensus_queue.cc:260] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:21.490993 21519 tablet_bootstrap.cc:492] T 99478ce52e24419ea31804ba62f7a46a P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:21.491660 21519 ts_tablet_manager.cc:1403] T 99478ce52e24419ea31804ba62f7a46a P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.006s
I20260709 13:22:21.493000 21832 consensus_queue.cc:1048] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:21.493957 21834 raft_consensus.cc:493] T 9805db96a0ab4ac3a686eb8ec904fb24 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:21.494490 21834 raft_consensus.cc:515] T 9805db96a0ab4ac3a686eb8ec904fb24 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:21.496781 21834 leader_election.cc:290] T 9805db96a0ab4ac3a686eb8ec904fb24 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:21.499405 21470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9805db96a0ab4ac3a686eb8ec904fb24" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:21.502585 21519 raft_consensus.cc:359] T 99478ce52e24419ea31804ba62f7a46a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:21.503743 21519 raft_consensus.cc:385] T 99478ce52e24419ea31804ba62f7a46a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:21.503369 21323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9805db96a0ab4ac3a686eb8ec904fb24" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:21.504132 21519 raft_consensus.cc:740] T 99478ce52e24419ea31804ba62f7a46a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:21.505834 21364 leader_election.cc:304] T 9805db96a0ab4ac3a686eb8ec904fb24 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:21.506325 21518 ts_tablet_manager.cc:1434] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:22:21.507007 21834 raft_consensus.cc:2749] T 9805db96a0ab4ac3a686eb8ec904fb24 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:21.507511 21518 tablet_bootstrap.cc:492] T dcf55c037e6c48bc9d73f87a54b26f92 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:21.507933 21519 consensus_queue.cc:260] T 99478ce52e24419ea31804ba62f7a46a P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:21.510588 21519 ts_tablet_manager.cc:1434] T 99478ce52e24419ea31804ba62f7a46a P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.001s
I20260709 13:22:21.511845 21519 tablet_bootstrap.cc:492] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:21.514386 21518 tablet_bootstrap.cc:654] T dcf55c037e6c48bc9d73f87a54b26f92 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:21.518070 21519 tablet_bootstrap.cc:654] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:21.523535 21400 raft_consensus.cc:3060] T 9805db96a0ab4ac3a686eb8ec904fb24 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:21.530674 21808 consensus_queue.cc:1048] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:21.523671 21521 tablet_bootstrap.cc:492] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:21.539726 21520 tablet_bootstrap.cc:492] T cdc82236ff024407ae854c96dd8c89ec P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:21.540586 21520 ts_tablet_manager.cc:1403] T cdc82236ff024407ae854c96dd8c89ec P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.218s	user 0.016s	sys 0.019s
I20260709 13:22:21.544977 21832 consensus_queue.cc:1048] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:21.550097 21518 tablet_bootstrap.cc:492] T dcf55c037e6c48bc9d73f87a54b26f92 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:21.550660 21518 ts_tablet_manager.cc:1403] T dcf55c037e6c48bc9d73f87a54b26f92 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.043s	user 0.011s	sys 0.002s
I20260709 13:22:21.554425 21518 raft_consensus.cc:359] T dcf55c037e6c48bc9d73f87a54b26f92 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:21.555126 21518 raft_consensus.cc:385] T dcf55c037e6c48bc9d73f87a54b26f92 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:21.555393 21518 raft_consensus.cc:740] T dcf55c037e6c48bc9d73f87a54b26f92 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:21.561492 21520 raft_consensus.cc:359] T cdc82236ff024407ae854c96dd8c89ec P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:21.562366 21520 raft_consensus.cc:385] T cdc82236ff024407ae854c96dd8c89ec P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:21.562685 21520 raft_consensus.cc:740] T cdc82236ff024407ae854c96dd8c89ec P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:21.538317 21521 ts_tablet_manager.cc:1403] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.217s	user 0.010s	sys 0.003s
I20260709 13:22:21.565148 21518 consensus_queue.cc:260] T dcf55c037e6c48bc9d73f87a54b26f92 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:21.563475 21520 consensus_queue.cc:260] T cdc82236ff024407ae854c96dd8c89ec P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:21.569067 21518 ts_tablet_manager.cc:1434] T dcf55c037e6c48bc9d73f87a54b26f92 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 13:22:21.586854 21518 tablet_bootstrap.cc:492] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:21.595418 21520 ts_tablet_manager.cc:1434] T cdc82236ff024407ae854c96dd8c89ec P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.054s	user 0.004s	sys 0.004s
I20260709 13:22:21.596680 21520 tablet_bootstrap.cc:492] T 8f2a6063e86146e8ba5f721d593edf0a P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:21.610052 21519 tablet_bootstrap.cc:492] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:21.610654 21519 ts_tablet_manager.cc:1403] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.099s	user 0.016s	sys 0.001s
I20260709 13:22:21.596300 21521 raft_consensus.cc:359] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:21.613413 21521 raft_consensus.cc:385] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:21.613727 21521 raft_consensus.cc:740] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:21.614392 21521 consensus_queue.cc:260] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:21.615208 21519 raft_consensus.cc:359] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:21.615973 21519 raft_consensus.cc:385] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:21.616250 21519 raft_consensus.cc:740] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:21.616889 21521 ts_tablet_manager.cc:1434] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.000s
I20260709 13:22:21.617906 21520 tablet_bootstrap.cc:654] T 8f2a6063e86146e8ba5f721d593edf0a P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:21.618018 21521 tablet_bootstrap.cc:492] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:21.621703 21519 consensus_queue.cc:260] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:21.624316 21519 ts_tablet_manager.cc:1434] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:22:21.625394 21519 tablet_bootstrap.cc:492] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:21.631793 21519 tablet_bootstrap.cc:654] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:21.638069 21521 tablet_bootstrap.cc:654] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:21.648911 21518 tablet_bootstrap.cc:654] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:21.649875 21323 raft_consensus.cc:3060] T cdc82236ff024407ae854c96dd8c89ec P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:21.656034 21520 tablet_bootstrap.cc:492] T 8f2a6063e86146e8ba5f721d593edf0a P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:21.656914 21520 ts_tablet_manager.cc:1403] T 8f2a6063e86146e8ba5f721d593edf0a P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.060s	user 0.016s	sys 0.008s
I20260709 13:22:21.658495 21667 raft_consensus.cc:493] T 99478ce52e24419ea31804ba62f7a46a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:21.662248 21667 raft_consensus.cc:515] T 99478ce52e24419ea31804ba62f7a46a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:21.664531 21667 leader_election.cc:290] T 99478ce52e24419ea31804ba62f7a46a P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:21.664361 21823 consensus_queue.cc:1048] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:21.671752 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99478ce52e24419ea31804ba62f7a46a" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:21.679563 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99478ce52e24419ea31804ba62f7a46a" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:21.681859 21669 raft_consensus.cc:493] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:21.682421 21669 raft_consensus.cc:515] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
W20260709 13:22:21.681258 21437 leader_election.cc:343] T 99478ce52e24419ea31804ba62f7a46a P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:21.686653 21247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7acd3d8dbc204f82b3f0f75ea8b68c57" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:21.687647 21247 raft_consensus.cc:2468] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 0.
W20260709 13:22:21.688890 21439 leader_election.cc:343] T 99478ce52e24419ea31804ba62f7a46a P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:21.689417 21439 leader_election.cc:304] T 99478ce52e24419ea31804ba62f7a46a P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636
W20260709 13:22:21.690451 21215 consensus_peers.cc:597] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:21.699064 21521 tablet_bootstrap.cc:492] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:21.699625 21521 ts_tablet_manager.cc:1403] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.082s	user 0.018s	sys 0.007s
I20260709 13:22:21.700560 21669 leader_election.cc:290] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:21.701010 21808 consensus_queue.cc:1048] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:21.703295 21437 leader_election.cc:304] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:21.691496 21740 raft_consensus.cc:2749] T 99478ce52e24419ea31804ba62f7a46a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:21.704986 21740 raft_consensus.cc:2804] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:21.705335 21740 raft_consensus.cc:493] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:21.705400 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7acd3d8dbc204f82b3f0f75ea8b68c57" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:21.705781 21740 raft_consensus.cc:3060] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:21.706907 21519 tablet_bootstrap.cc:492] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:21.707437 21519 ts_tablet_manager.cc:1403] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.082s	user 0.012s	sys 0.024s
W20260709 13:22:21.711215 21438 leader_election.cc:343] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:21.712160 21520 raft_consensus.cc:359] T 8f2a6063e86146e8ba5f721d593edf0a P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:21.712951 21520 raft_consensus.cc:385] T 8f2a6063e86146e8ba5f721d593edf0a P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:21.714468 21520 raft_consensus.cc:740] T 8f2a6063e86146e8ba5f721d593edf0a P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:21.716204 21325 raft_consensus.cc:3060] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:21.716689 21669 raft_consensus.cc:493] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:21.717181 21669 raft_consensus.cc:515] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:21.720661 21669 leader_election.cc:290] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:21.722325 21247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcf55c037e6c48bc9d73f87a54b26f92" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:21.727191 21247 raft_consensus.cc:2468] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 0.
I20260709 13:22:21.725433 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcf55c037e6c48bc9d73f87a54b26f92" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:21.728240 21400 raft_consensus.cc:2468] T dcf55c037e6c48bc9d73f87a54b26f92 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 0.
I20260709 13:22:21.728770 21437 leader_election.cc:304] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:21.730360 21669 raft_consensus.cc:2804] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:21.730769 21669 raft_consensus.cc:493] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:21.731165 21669 raft_consensus.cc:3060] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:21.726153 21518 tablet_bootstrap.cc:492] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:21.738557 21518 ts_tablet_manager.cc:1403] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.152s	user 0.018s	sys 0.035s
I20260709 13:22:21.748716 21521 raft_consensus.cc:359] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:21.749913 21521 raft_consensus.cc:385] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:21.752388 21667 raft_consensus.cc:493] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 74e2ff1a68ac4c849ee6397ebb011375)
I20260709 13:22:21.751026 21521 raft_consensus.cc:740] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:21.752997 21667 raft_consensus.cc:515] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:21.757449 21667 leader_election.cc:290] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:21.746053 21520 consensus_queue.cc:260] T 8f2a6063e86146e8ba5f721d593edf0a P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:21.763820 21397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc40f11dce5540d1a056e9bacba075e2" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:21.759583 21247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc40f11dce5540d1a056e9bacba075e2" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:21.785588 21518 raft_consensus.cc:359] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:21.787775 21518 raft_consensus.cc:385] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:21.788259 21518 raft_consensus.cc:740] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:21.786353 21397 raft_consensus.cc:2468] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 2.
I20260709 13:22:21.776124 21521 consensus_queue.cc:260] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:21.794327 21247 raft_consensus.cc:2393] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 43cd181efe6b4b2fb79fd3e997a401ea in current term 2: Already voted for candidate ec879c0528c44465916efc6c33dde636 in this term.
I20260709 13:22:21.799144 21518 consensus_queue.cc:260] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:21.805234 21520 ts_tablet_manager.cc:1434] T 8f2a6063e86146e8ba5f721d593edf0a P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.148s	user 0.006s	sys 0.002s
I20260709 13:22:21.810083 21518 ts_tablet_manager.cc:1434] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.071s	user 0.009s	sys 0.000s
I20260709 13:22:21.804636 21519 raft_consensus.cc:359] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:21.813292 21519 raft_consensus.cc:385] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:21.814497 21740 raft_consensus.cc:515] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:21.816921 21740 leader_election.cc:290] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:21.818257 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7acd3d8dbc204f82b3f0f75ea8b68c57" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:21.818487 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7acd3d8dbc204f82b3f0f75ea8b68c57" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:21.819007 21248 raft_consensus.cc:3060] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:21.819856 21438 leader_election.cc:343] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:21.824750 21669 raft_consensus.cc:515] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:21.826490 21521 ts_tablet_manager.cc:1434] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.127s	user 0.009s	sys 0.001s
I20260709 13:22:21.827201 21669 leader_election.cc:290] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:21.827683 21521 tablet_bootstrap.cc:492] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:21.829700 21518 tablet_bootstrap.cc:492] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:21.813663 21519 raft_consensus.cc:740] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:21.834048 21247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcf55c037e6c48bc9d73f87a54b26f92" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:21.834946 21247 raft_consensus.cc:3060] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:21.832829 21519 consensus_queue.cc:260] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:21.837042 21518 tablet_bootstrap.cc:654] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:21.837782 21520 tablet_bootstrap.cc:492] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:21.849754 21823 raft_consensus.cc:493] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:21.863101 21858 raft_consensus.cc:493] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:21.863747 21858 raft_consensus.cc:515] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:21.866997 21858 leader_election.cc:290] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:21.867300 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fff6b36d1984d38b6f791e0ac15f56b" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
W20260709 13:22:21.868969 21437 leader_election.cc:343] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:21.872155 21823 raft_consensus.cc:515] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:21.874701 21247 raft_consensus.cc:2468] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:21.876174 21437 leader_election.cc:304] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:21.876926 21823 leader_election.cc:290] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:21.877676 21669 raft_consensus.cc:2804] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:21.877982 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fff6b36d1984d38b6f791e0ac15f56b" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:21.879177 21669 raft_consensus.cc:697] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:21.880296 21669 consensus_queue.cc:237] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:21.881502 21247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a6063e86146e8ba5f721d593edf0a" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
W20260709 13:22:21.886426 21438 leader_election.cc:343] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:21.888011 21438 leader_election.cc:304] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:21.889616 21740 raft_consensus.cc:2749] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:21.896588 21439 leader_election.cc:304] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: ec879c0528c44465916efc6c33dde636
I20260709 13:22:21.898023 21667 raft_consensus.cc:2804] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:21.898312 21520 tablet_bootstrap.cc:654] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:21.898463 21667 raft_consensus.cc:493] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 74e2ff1a68ac4c849ee6397ebb011375)
I20260709 13:22:21.898844 21667 raft_consensus.cc:3060] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:21.907974 21831 raft_consensus.cc:493] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:21.908639 21831 raft_consensus.cc:515] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:21.911621 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08649134c0c94e7caa55fcc7513eec5d" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:21.912367 21521 tablet_bootstrap.cc:654] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:21.913118 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08649134c0c94e7caa55fcc7513eec5d" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
W20260709 13:22:21.913395 21215 leader_election.cc:343] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:21.913949 21247 raft_consensus.cc:2468] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
I20260709 13:22:21.915894 21363 leader_election.cc:304] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:21.916895 21322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a6063e86146e8ba5f721d593edf0a" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:21.917671 21322 raft_consensus.cc:2468] T 8f2a6063e86146e8ba5f721d593edf0a P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
W20260709 13:22:21.916982 21216 leader_election.cc:343] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:21.911746 21831 leader_election.cc:290] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:21.907841 21248 raft_consensus.cc:2468] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:21.924856 21519 ts_tablet_manager.cc:1434] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.217s	user 0.005s	sys 0.005s
I20260709 13:22:21.935034 21667 raft_consensus.cc:515] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:21.922848 21834 raft_consensus.cc:2804] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:21.922623 21216 leader_election.cc:304] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:21.949935 21834 raft_consensus.cc:493] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:21.957021 21831 raft_consensus.cc:2749] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:21.959450 21437 leader_election.cc:304] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:21.962507 21858 raft_consensus.cc:2804] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:21.963219 21858 raft_consensus.cc:697] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:21.964322 21858 consensus_queue.cc:237] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:21.965342 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcf55c037e6c48bc9d73f87a54b26f92" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:21.966199 21396 raft_consensus.cc:3060] T dcf55c037e6c48bc9d73f87a54b26f92 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:21.969180 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc40f11dce5540d1a056e9bacba075e2" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:21.970022 21251 raft_consensus.cc:2393] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 43cd181efe6b4b2fb79fd3e997a401ea in current term 2: Already voted for candidate ec879c0528c44465916efc6c33dde636 in this term.
I20260709 13:22:21.984664 21887 raft_consensus.cc:493] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:21.961227 21834 raft_consensus.cc:3060] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:21.985334 21887 raft_consensus.cc:515] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:21.955343 21667 leader_election.cc:290] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:21.989617 21887 leader_election.cc:290] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:21.991710 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033f24b2a37f421cbea89f5c5c61d0a3" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:21.992663 21474 raft_consensus.cc:2468] T 033f24b2a37f421cbea89f5c5c61d0a3 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
I20260709 13:22:21.994445 21363 leader_election.cc:304] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:21.995712 21886 raft_consensus.cc:2804] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:21.996196 21886 raft_consensus.cc:493] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:21.996628 21886 raft_consensus.cc:3060] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:22.002991 21519 tablet_bootstrap.cc:492] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:22.014389 21834 raft_consensus.cc:515] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:22.116218 21396 raft_consensus.cc:2468] T dcf55c037e6c48bc9d73f87a54b26f92 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:22.135582 21886 raft_consensus.cc:515] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:22.147147 21834 leader_election.cc:290] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:22.158336 21520 tablet_bootstrap.cc:492] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:22.158923 21520 ts_tablet_manager.cc:1403] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.353s	user 0.027s	sys 0.040s
I20260709 13:22:22.159063 21521 tablet_bootstrap.cc:492] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:22.159664 21521 ts_tablet_manager.cc:1403] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.332s	user 0.053s	sys 0.055s
I20260709 13:22:22.161841 21520 raft_consensus.cc:359] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:22.162539 21520 raft_consensus.cc:385] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:22.162801 21520 raft_consensus.cc:740] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:22.154208 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a6063e86146e8ba5f721d593edf0a" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:22.158412 21518 tablet_bootstrap.cc:492] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:22.163718 21251 raft_consensus.cc:3060] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:22.164109 21518 ts_tablet_manager.cc:1403] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.335s	user 0.039s	sys 0.015s
I20260709 13:22:22.166829 21518 raft_consensus.cc:359] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:22.167431 21518 raft_consensus.cc:385] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:22.167665 21518 raft_consensus.cc:740] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:22.168365 21520 consensus_queue.cc:260] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:22.162657 21521 raft_consensus.cc:359] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:22.170132 21521 raft_consensus.cc:385] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:22.172362 21521 raft_consensus.cc:740] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:22.173242 21520 ts_tablet_manager.cc:1434] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 13:22:22.173233 21521 consensus_queue.cc:260] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:22.176247 21521 ts_tablet_manager.cc:1434] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:22:22.177119 21251 raft_consensus.cc:2468] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:22.169502 21518 consensus_queue.cc:260] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:22.191859 21363 leader_election.cc:304] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:22.193446 21887 raft_consensus.cc:2804] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:22.194142 21521 tablet_bootstrap.cc:492] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:22.193987 21887 raft_consensus.cc:697] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Leader. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:22.196306 21887 consensus_queue.cc:237] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:22.206476 21520 tablet_bootstrap.cc:492] T 033f24b2a37f421cbea89f5c5c61d0a3 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:22.218735 21886 leader_election.cc:290] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Requested vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:22.221331 21519 tablet_bootstrap.cc:654] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:22.222345 21472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033f24b2a37f421cbea89f5c5c61d0a3" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:22.223240 21472 raft_consensus.cc:3060] T 033f24b2a37f421cbea89f5c5c61d0a3 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:22.228344 21518 ts_tablet_manager.cc:1434] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.064s	user 0.004s	sys 0.003s
I20260709 13:22:22.230794 21823 consensus_queue.cc:1048] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:22:22.250150 21518 tablet_bootstrap.cc:492] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:22.252302 21472 raft_consensus.cc:2468] T 033f24b2a37f421cbea89f5c5c61d0a3 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:22.248072 21149 catalog_manager.cc:5697] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 1 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:22.257087 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033f24b2a37f421cbea89f5c5c61d0a3" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:22.258150 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc40f11dce5540d1a056e9bacba075e2" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:22.258297 21322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a6063e86146e8ba5f721d593edf0a" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:22.259007 21322 raft_consensus.cc:3060] T 8f2a6063e86146e8ba5f721d593edf0a P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:22.262055 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033f24b2a37f421cbea89f5c5c61d0a3" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:22.263899 21687 raft_consensus.cc:493] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:22.264569 21687 raft_consensus.cc:515] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:22.269918 21687 leader_election.cc:290] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:22.270527 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc40f11dce5540d1a056e9bacba075e2" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:22.272516 21470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc40f11dce5540d1a056e9bacba075e2" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:22.273190 21470 raft_consensus.cc:2468] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 2.
I20260709 13:22:22.275878 21519 tablet_bootstrap.cc:492] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:22.276738 21519 ts_tablet_manager.cc:1403] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.274s	user 0.009s	sys 0.009s
I20260709 13:22:22.273133 21322 raft_consensus.cc:2468] T 8f2a6063e86146e8ba5f721d593edf0a P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:22.282284 21892 raft_consensus.cc:493] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:22.287858 21892 raft_consensus.cc:515] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:22.289479 21520 tablet_bootstrap.cc:654] T 033f24b2a37f421cbea89f5c5c61d0a3 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:22.299284 21396 raft_consensus.cc:2468] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 2.
I20260709 13:22:22.282691 21214 leader_election.cc:304] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:22.302371 21687 raft_consensus.cc:2804] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:22:22.302928 21687 raft_consensus.cc:493] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:22.303416 21687 raft_consensus.cc:3060] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:22.308923 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37dadd42cd04be5a0e0b339df630a04" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:22.307855 21892 leader_election.cc:290] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
W20260709 13:22:22.311108 21437 leader_election.cc:343] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:22.312646 21519 raft_consensus.cc:359] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:22.313616 21519 raft_consensus.cc:385] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:22.313923 21519 raft_consensus.cc:740] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:22.318857 21519 consensus_queue.cc:260] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:22.322778 21521 tablet_bootstrap.cc:654] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
W20260709 13:22:22.317939 21363 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36863, user_credentials={real_user=slave}} blocked reactor thread for 57970.9us
I20260709 13:22:22.337456 21147 catalog_manager.cc:5697] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "74e2ff1a68ac4c849ee6397ebb011375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:22.354456 21519 ts_tablet_manager.cc:1434] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.077s	user 0.004s	sys 0.004s
I20260709 13:22:22.350502 21687 raft_consensus.cc:515] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:22.354462 21363 leader_election.cc:304] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:22.359948 21519 tablet_bootstrap.cc:492] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:22.366312 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37dadd42cd04be5a0e0b339df630a04" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:22.368414 21823 raft_consensus.cc:2804] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:22.371039 21823 raft_consensus.cc:697] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Leader. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:22.372041 21823 consensus_queue.cc:237] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:22.373795 21439 leader_election.cc:304] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: ec879c0528c44465916efc6c33dde636
I20260709 13:22:22.375290 21892 raft_consensus.cc:2804] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:22.378509 21518 tablet_bootstrap.cc:654] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:22.378314 21892 raft_consensus.cc:697] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:22.380780 21520 tablet_bootstrap.cc:492] T 033f24b2a37f421cbea89f5c5c61d0a3 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:22.379906 21892 consensus_queue.cc:237] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:22.381317 21520 ts_tablet_manager.cc:1403] T 033f24b2a37f421cbea89f5c5c61d0a3 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.177s	user 0.019s	sys 0.012s
I20260709 13:22:22.382385 21519 tablet_bootstrap.cc:654] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
W20260709 13:22:22.383471 21364 leader_election.cc:343] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:22.384600 21364 leader_election.cc:343] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:22.387347 21520 raft_consensus.cc:359] T 033f24b2a37f421cbea89f5c5c61d0a3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:22.388316 21520 raft_consensus.cc:385] T 033f24b2a37f421cbea89f5c5c61d0a3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:22.388659 21520 raft_consensus.cc:740] T 033f24b2a37f421cbea89f5c5c61d0a3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
W20260709 13:22:22.392066 21439 leader_election.cc:343] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:22.392738 21439 leader_election.cc:304] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636
I20260709 13:22:22.389506 21520 consensus_queue.cc:260] T 033f24b2a37f421cbea89f5c5c61d0a3 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:22.393712 21669 raft_consensus.cc:2749] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:22.399427 21520 ts_tablet_manager.cc:1434] T 033f24b2a37f421cbea89f5c5c61d0a3 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.004s
I20260709 13:22:22.401101 21687 leader_election.cc:290] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 3 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:22.403467 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc40f11dce5540d1a056e9bacba075e2" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:22.404129 21400 raft_consensus.cc:3060] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:22.405298 21472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc40f11dce5540d1a056e9bacba075e2" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:22.416216 21836 raft_consensus.cc:493] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:22.416996 21836 raft_consensus.cc:515] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:22.420629 21836 leader_election.cc:290] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:22.422991 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ddeabcec9b64fc68b40c70df90b80ef" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:22.426568 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ddeabcec9b64fc68b40c70df90b80ef" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:22.428342 21521 tablet_bootstrap.cc:492] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:22.428892 21521 ts_tablet_manager.cc:1403] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.247s	user 0.014s	sys 0.014s
I20260709 13:22:22.431289 21518 tablet_bootstrap.cc:492] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:22.431981 21518 ts_tablet_manager.cc:1403] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.200s	user 0.013s	sys 0.002s
I20260709 13:22:22.434614 21289 leader_election.cc:304] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636
I20260709 13:22:22.436204 21518 raft_consensus.cc:359] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:22.436880 21867 raft_consensus.cc:2749] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:22.446971 21518 raft_consensus.cc:385] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:22.447463 21518 raft_consensus.cc:740] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:22.448407 21518 consensus_queue.cc:260] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:22.473788 21520 tablet_bootstrap.cc:492] T 08649134c0c94e7caa55fcc7513eec5d P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:22.475281 21867 raft_consensus.cc:493] T 033f24b2a37f421cbea89f5c5c61d0a3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:22.486881 21518 ts_tablet_manager.cc:1434] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.055s	user 0.008s	sys 0.000s
I20260709 13:22:22.499207 21400 raft_consensus.cc:2468] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 3.
I20260709 13:22:22.500707 21867 raft_consensus.cc:515] T 033f24b2a37f421cbea89f5c5c61d0a3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:22.516252 21867 leader_election.cc:290] T 033f24b2a37f421cbea89f5c5c61d0a3 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:22.518908 21216 leader_election.cc:304] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 43cd181efe6b4b2fb79fd3e997a401ea
I20260709 13:22:22.525740 21808 raft_consensus.cc:2804] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:22.526253 21808 raft_consensus.cc:697] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 3 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:22.527062 21808 consensus_queue.cc:237] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:22.531890 21858 consensus_queue.cc:1048] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:22.539739 21518 tablet_bootstrap.cc:492] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:22.563916 21858 consensus_queue.cc:1048] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:22.571480 21521 raft_consensus.cc:359] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:22.575165 21521 raft_consensus.cc:385] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:22.575506 21521 raft_consensus.cc:740] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:22.577922 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033f24b2a37f421cbea89f5c5c61d0a3" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:22.576944 21521 consensus_queue.cc:260] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:22.580931 21474 raft_consensus.cc:2393] T 033f24b2a37f421cbea89f5c5c61d0a3 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5109d6d6d9404ba39c53508a81fe9f1c in current term 1: Already voted for candidate 74e2ff1a68ac4c849ee6397ebb011375 in this term.
I20260709 13:22:22.582116 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "033f24b2a37f421cbea89f5c5c61d0a3" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:22.595330 21520 tablet_bootstrap.cc:654] T 08649134c0c94e7caa55fcc7513eec5d P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:22.584656 21147 catalog_manager.cc:5697] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "74e2ff1a68ac4c849ee6397ebb011375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } }
W20260709 13:22:22.597275 21196 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:22.599721 21521 ts_tablet_manager.cc:1434] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.170s	user 0.009s	sys 0.002s
I20260709 13:22:22.603183 21519 tablet_bootstrap.cc:492] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:22.603770 21519 ts_tablet_manager.cc:1403] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.244s	user 0.039s	sys 0.028s
I20260709 13:22:22.598680 21898 raft_consensus.cc:493] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:22.609000 21898 raft_consensus.cc:515] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:22.611501 21898 leader_election.cc:290] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:22.612098 21669 consensus_queue.cc:1048] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:22.613148 21289 leader_election.cc:304] T 033f24b2a37f421cbea89f5c5c61d0a3 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:22.614528 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99478ce52e24419ea31804ba62f7a46a" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:22.615391 21248 raft_consensus.cc:2468] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
I20260709 13:22:22.616110 21519 raft_consensus.cc:359] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:22.616902 21519 raft_consensus.cc:385] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:22.617190 21519 raft_consensus.cc:740] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:22.618254 21363 leader_election.cc:304] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:22.619350 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99478ce52e24419ea31804ba62f7a46a" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:22.619935 21474 raft_consensus.cc:2468] T 99478ce52e24419ea31804ba62f7a46a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
I20260709 13:22:22.621006 21886 raft_consensus.cc:2804] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:22.621450 21886 raft_consensus.cc:493] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:22.621865 21886 raft_consensus.cc:3060] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:22.617230 21910 raft_consensus.cc:3060] T 033f24b2a37f421cbea89f5c5c61d0a3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:22.625228 21906 log.cc:927] Time spent T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea: Append to log took a long time: real 0.063s	user 0.003s	sys 0.000s
I20260709 13:22:22.628890 21521 tablet_bootstrap.cc:492] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:22.629012 21887 raft_consensus.cc:493] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:22.629591 21887 raft_consensus.cc:515] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:22.618059 21519 consensus_queue.cc:260] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:22.629439 21518 tablet_bootstrap.cc:654] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:22.631788 21887 leader_election.cc:290] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:22.641091 21519 ts_tablet_manager.cc:1434] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.037s	user 0.010s	sys 0.000s
I20260709 13:22:22.650804 21149 catalog_manager.cc:5697] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 1 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:22.656092 21149 catalog_manager.cc:5697] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 1 to 2, leader changed from 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67) to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 2 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:22.648291 21247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08649134c0c94e7caa55fcc7513eec5d" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:22.661437 21519 tablet_bootstrap.cc:492] T 665861bf917f4d42aa1252c374a656ea P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:22.666237 21326 raft_consensus.cc:3060] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:22.663508 21247 raft_consensus.cc:2468] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
I20260709 13:22:22.667316 21910 raft_consensus.cc:2749] T 033f24b2a37f421cbea89f5c5c61d0a3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:22.666326 21322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08649134c0c94e7caa55fcc7513eec5d" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:22.669417 21831 raft_consensus.cc:493] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:22.669945 21831 raft_consensus.cc:515] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
W20260709 13:22:22.672863 21364 leader_election.cc:343] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:22.672067 21831 leader_election.cc:290] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
W20260709 13:22:22.663980 21438 consensus_peers.cc:597] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:22.678009 21363 leader_election.cc:304] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:22.679807 21887 raft_consensus.cc:2804] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:22.680205 21887 raft_consensus.cc:493] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:22.680533 21887 raft_consensus.cc:3060] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:22.680828 21397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169fb713efc445fb1f0338c7c11203a" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:22.687598 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169fb713efc445fb1f0338c7c11203a" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
W20260709 13:22:22.694681 21215 leader_election.cc:343] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:22.696367 21519 tablet_bootstrap.cc:654] T 665861bf917f4d42aa1252c374a656ea P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
W20260709 13:22:22.700750 21216 leader_election.cc:343] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:22.701210 21216 leader_election.cc:304] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:22.706413 21147 catalog_manager.cc:5697] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 2 to 3, leader changed from 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68) to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 3 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:22.716863 21831 raft_consensus.cc:2749] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:22.723850 21519 tablet_bootstrap.cc:492] T 665861bf917f4d42aa1252c374a656ea P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:22.724673 21519 ts_tablet_manager.cc:1403] T 665861bf917f4d42aa1252c374a656ea P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.067s	user 0.013s	sys 0.005s
I20260709 13:22:22.699942 21892 raft_consensus.cc:493] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:22.734546 21887 raft_consensus.cc:515] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:22.740226 21521 tablet_bootstrap.cc:654] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:22.732200 21858 raft_consensus.cc:493] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:22.717425 21886 raft_consensus.cc:515] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:22.739835 21892 raft_consensus.cc:515] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:22.762204 21519 raft_consensus.cc:359] T 665861bf917f4d42aa1252c374a656ea P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:22.763069 21519 raft_consensus.cc:385] T 665861bf917f4d42aa1252c374a656ea P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:22.763384 21519 raft_consensus.cc:740] T 665861bf917f4d42aa1252c374a656ea P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:22.763846 21892 leader_election.cc:290] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:22.764132 21519 consensus_queue.cc:260] T 665861bf917f4d42aa1252c374a656ea P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:22.749855 21858 raft_consensus.cc:515] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:22.766345 21519 ts_tablet_manager.cc:1434] T 665861bf917f4d42aa1252c374a656ea P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.000s
I20260709 13:22:22.767092 21858 leader_election.cc:290] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:22.780509 21519 tablet_bootstrap.cc:492] T 3f3bf699b02c46c78d330dc7f9d3314b P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:22.784654 21887 leader_election.cc:290] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:22.817070 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08649134c0c94e7caa55fcc7513eec5d" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:22.818019 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d5ddbaefa064931963bc1e27ac99903" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:22.836388 21520 tablet_bootstrap.cc:492] T 08649134c0c94e7caa55fcc7513eec5d P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:22.836957 21520 ts_tablet_manager.cc:1403] T 08649134c0c94e7caa55fcc7513eec5d P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.430s	user 0.018s	sys 0.024s
I20260709 13:22:22.822929 21323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08649134c0c94e7caa55fcc7513eec5d" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:22.839395 21247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d5ddbaefa064931963bc1e27ac99903" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:22.839815 21250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280cfe3edb3f490c8de46d0c80f6da5a" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:22.838703 21400 raft_consensus.cc:1240] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Rejecting Update request from peer 43cd181efe6b4b2fb79fd3e997a401ea for earlier term 2. Current term is 3. Ops: []
I20260709 13:22:22.840425 21249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99478ce52e24419ea31804ba62f7a46a" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:22.841123 21249 raft_consensus.cc:3060] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:22.843295 21518 tablet_bootstrap.cc:492] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:22.843814 21518 ts_tablet_manager.cc:1403] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.322s	user 0.010s	sys 0.010s
I20260709 13:22:22.846014 21248 raft_consensus.cc:3060] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:22.847235 21439 leader_election.cc:343] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:22.847282 21887 consensus_queue.cc:1048] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:22:22.818331 21438 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:38825, user_credentials={real_user=slave}} blocked reactor thread for 81085.9us
W20260709 13:22:22.851258 21437 leader_election.cc:343] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:22.849371 21669 consensus_queue.cc:1059] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:22:22.852105 21437 leader_election.cc:304] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636
I20260709 13:22:22.851184 21518 raft_consensus.cc:359] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:22.853502 21518 raft_consensus.cc:385] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:22.853902 21518 raft_consensus.cc:740] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:22.853060 21519 tablet_bootstrap.cc:654] T 3f3bf699b02c46c78d330dc7f9d3314b P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:22.854650 21518 consensus_queue.cc:260] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:22.857959 21322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280cfe3edb3f490c8de46d0c80f6da5a" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:22.859371 21669 raft_consensus.cc:3055] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:22.859858 21669 raft_consensus.cc:740] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
W20260709 13:22:22.859941 21364 leader_election.cc:343] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:22.860755 21669 consensus_queue.cc:260] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:22.862190 21669 raft_consensus.cc:3060] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:22.867828 21520 raft_consensus.cc:359] T 08649134c0c94e7caa55fcc7513eec5d P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:22.868673 21520 raft_consensus.cc:385] T 08649134c0c94e7caa55fcc7513eec5d P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:22.868963 21520 raft_consensus.cc:740] T 08649134c0c94e7caa55fcc7513eec5d P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:22.869769 21520 consensus_queue.cc:260] T 08649134c0c94e7caa55fcc7513eec5d P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
W20260709 13:22:22.870431 21437 leader_election.cc:343] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:22.876250 21472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99478ce52e24419ea31804ba62f7a46a" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:22.876928 21472 raft_consensus.cc:3060] T 99478ce52e24419ea31804ba62f7a46a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:22.836445 21886 leader_election.cc:290] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:22.884927 21893 raft_consensus.cc:2749] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:22.898216 21520 ts_tablet_manager.cc:1434] T 08649134c0c94e7caa55fcc7513eec5d P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.061s	user 0.009s	sys 0.000s
I20260709 13:22:22.909744 21518 ts_tablet_manager.cc:1434] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.066s	user 0.007s	sys 0.001s
I20260709 13:22:22.921907 21249 raft_consensus.cc:2468] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:22.923807 21248 raft_consensus.cc:2468] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:22.929642 21520 tablet_bootstrap.cc:492] T 1fff6b36d1984d38b6f791e0ac15f56b P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:22.939868 21363 leader_election.cc:304] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:22.941115 21363 leader_election.cc:304] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:22.942435 21823 raft_consensus.cc:2804] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:22.943130 21823 raft_consensus.cc:697] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Leader. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:22.944355 21823 consensus_queue.cc:237] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:22.945780 21834 raft_consensus.cc:2804] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:22.946394 21834 raft_consensus.cc:697] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Leader. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:22.947321 21834 consensus_queue.cc:237] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:22.949236 21472 raft_consensus.cc:2468] T 99478ce52e24419ea31804ba62f7a46a P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:22.960346 21520 tablet_bootstrap.cc:654] T 1fff6b36d1984d38b6f791e0ac15f56b P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:22.964943 21518 tablet_bootstrap.cc:492] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:22.979666 21521 tablet_bootstrap.cc:492] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:22.979382 21925 consensus_queue.cc:1048] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:22.980199 21521 ts_tablet_manager.cc:1403] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.352s	user 0.024s	sys 0.006s
I20260709 13:22:22.980728 21898 consensus_queue.cc:1048] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:22.983011 21521 raft_consensus.cc:359] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:22.983693 21521 raft_consensus.cc:385] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:22.983949 21521 raft_consensus.cc:740] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
W20260709 13:22:22.987560 21438 leader_election.cc:343] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:22.993718 21521 consensus_queue.cc:260] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:22.996918 21887 consensus_queue.cc:1048] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:22:23.006614 21519 tablet_bootstrap.cc:492] T 3f3bf699b02c46c78d330dc7f9d3314b P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:22.996459 21438 leader_election.cc:304] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:23.009642 21518 tablet_bootstrap.cc:654] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:23.019950 21519 ts_tablet_manager.cc:1403] T 3f3bf699b02c46c78d330dc7f9d3314b P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.240s	user 0.040s	sys 0.031s
I20260709 13:22:23.035565 21669 raft_consensus.cc:2749] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:22:23.036641 21438 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:38825, user_credentials={real_user=slave}} blocked reactor thread for 49126.7us
I20260709 13:22:23.051838 21521 ts_tablet_manager.cc:1434] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.071s	user 0.014s	sys 0.001s
I20260709 13:22:23.053068 21887 raft_consensus.cc:493] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:23.053925 21520 tablet_bootstrap.cc:492] T 1fff6b36d1984d38b6f791e0ac15f56b P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:23.059032 21520 ts_tablet_manager.cc:1403] T 1fff6b36d1984d38b6f791e0ac15f56b P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.134s	user 0.018s	sys 0.010s
I20260709 13:22:23.061509 21887 raft_consensus.cc:515] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:23.068334 21521 tablet_bootstrap.cc:492] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:23.069121 21887 leader_election.cc:290] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:23.072145 21326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bf69be9e9240b5ac99adc49258432e" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
W20260709 13:22:23.073714 21364 leader_election.cc:343] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:23.075150 21519 raft_consensus.cc:359] T 3f3bf699b02c46c78d330dc7f9d3314b P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:23.075948 21519 raft_consensus.cc:385] T 3f3bf699b02c46c78d330dc7f9d3314b P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:23.076234 21519 raft_consensus.cc:740] T 3f3bf699b02c46c78d330dc7f9d3314b P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:23.076943 21519 consensus_queue.cc:260] T 3f3bf699b02c46c78d330dc7f9d3314b P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:23.080821 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bf69be9e9240b5ac99adc49258432e" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:23.081529 21474 raft_consensus.cc:2468] T d2bf69be9e9240b5ac99adc49258432e P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
I20260709 13:22:23.086021 21470 raft_consensus.cc:1275] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:22:23.087244 21470 pending_rounds.cc:85] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea: Aborting all ops after (but not including) 1
I20260709 13:22:23.087734 21470 pending_rounds.cc:107] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:22:23.088282 21470 raft_consensus.cc:2889] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:23.097957 21363 leader_election.cc:304] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:23.099153 21823 raft_consensus.cc:2804] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:23.099579 21823 raft_consensus.cc:493] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:23.099956 21823 raft_consensus.cc:3060] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:23.099769 21911 consensus_queue.cc:1048] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:23.105979 21400 raft_consensus.cc:1275] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:22:23.106958 21519 ts_tablet_manager.cc:1434] T 3f3bf699b02c46c78d330dc7f9d3314b P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.087s	user 0.008s	sys 0.000s
I20260709 13:22:23.113466 21519 tablet_bootstrap.cc:492] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:23.109993 21914 consensus_queue.cc:1048] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:23.119215 21520 raft_consensus.cc:359] T 1fff6b36d1984d38b6f791e0ac15f56b P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:23.120044 21520 raft_consensus.cc:385] T 1fff6b36d1984d38b6f791e0ac15f56b P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:23.120342 21520 raft_consensus.cc:740] T 1fff6b36d1984d38b6f791e0ac15f56b P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:23.133183 21831 raft_consensus.cc:493] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:23.133733 21831 raft_consensus.cc:515] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:23.136054 21831 leader_election.cc:290] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:23.140427 21519 tablet_bootstrap.cc:654] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:23.112088 21521 tablet_bootstrap.cc:654] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:23.141753 21669 raft_consensus.cc:493] T 665861bf917f4d42aa1252c374a656ea P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:23.141646 21520 consensus_queue.cc:260] T 1fff6b36d1984d38b6f791e0ac15f56b P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:23.093439 21149 catalog_manager.cc:5697] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "74e2ff1a68ac4c849ee6397ebb011375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:23.142423 21669 raft_consensus.cc:515] T 665861bf917f4d42aa1252c374a656ea P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:23.144726 21669 leader_election.cc:290] T 665861bf917f4d42aa1252c374a656ea P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:23.151078 21326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "665861bf917f4d42aa1252c374a656ea" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:23.151975 21823 raft_consensus.cc:515] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:23.152814 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "665861bf917f4d42aa1252c374a656ea" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:23.153923 21322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fff6b36d1984d38b6f791e0ac15f56b" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
W20260709 13:22:23.152884 21438 leader_election.cc:343] T 665861bf917f4d42aa1252c374a656ea P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:23.154789 21322 raft_consensus.cc:2468] T 1fff6b36d1984d38b6f791e0ac15f56b P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
W20260709 13:22:23.155220 21439 leader_election.cc:343] T 665861bf917f4d42aa1252c374a656ea P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:23.155670 21439 leader_election.cc:304] T 665861bf917f4d42aa1252c374a656ea P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:23.157827 21923 raft_consensus.cc:2749] T 665861bf917f4d42aa1252c374a656ea P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:23.149926 21520 ts_tablet_manager.cc:1434] T 1fff6b36d1984d38b6f791e0ac15f56b P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.091s	user 0.008s	sys 0.001s
I20260709 13:22:23.111472 21518 tablet_bootstrap.cc:492] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:23.161739 21518 ts_tablet_manager.cc:1403] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.197s	user 0.033s	sys 0.015s
I20260709 13:22:23.160863 21149 catalog_manager.cc:5697] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "74e2ff1a68ac4c849ee6397ebb011375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:23.163959 21215 leader_election.cc:304] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:23.165210 21520 tablet_bootstrap.cc:492] T d2bf69be9e9240b5ac99adc49258432e P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:23.165272 21822 raft_consensus.cc:2804] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:23.167568 21910 raft_consensus.cc:493] T 08649134c0c94e7caa55fcc7513eec5d P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:23.167912 21822 raft_consensus.cc:493] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:23.168406 21822 raft_consensus.cc:3060] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:23.168080 21910 raft_consensus.cc:515] T 08649134c0c94e7caa55fcc7513eec5d P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:23.171566 21910 leader_election.cc:290] T 08649134c0c94e7caa55fcc7513eec5d P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:23.184499 21472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fff6b36d1984d38b6f791e0ac15f56b" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:23.185206 21472 raft_consensus.cc:2468] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:23.187212 21823 leader_election.cc:290] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Requested vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:23.187930 21520 tablet_bootstrap.cc:654] T d2bf69be9e9240b5ac99adc49258432e P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:23.184872 21518 raft_consensus.cc:359] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:23.191016 21518 raft_consensus.cc:385] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:23.191370 21518 raft_consensus.cc:740] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:23.194876 21519 tablet_bootstrap.cc:492] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:23.195374 21519 ts_tablet_manager.cc:1403] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.082s	user 0.012s	sys 0.004s
I20260709 13:22:23.197109 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bf69be9e9240b5ac99adc49258432e" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:23.192122 21518 consensus_queue.cc:260] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
W20260709 13:22:23.207450 21364 leader_election.cc:343] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:23.209800 21518 ts_tablet_manager.cc:1434] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.000s
I20260709 13:22:23.210186 21519 raft_consensus.cc:359] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:23.210881 21519 raft_consensus.cc:385] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:23.211148 21519 raft_consensus.cc:740] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:23.211795 21519 consensus_queue.cc:260] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:23.199733 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08649134c0c94e7caa55fcc7513eec5d" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:23.223779 21519 ts_tablet_manager.cc:1434] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.003s
I20260709 13:22:23.231691 21472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bf69be9e9240b5ac99adc49258432e" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:23.224929 21519 tablet_bootstrap.cc:492] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea: Bootstrap starting.
I20260709 13:22:23.232476 21472 raft_consensus.cc:3060] T d2bf69be9e9240b5ac99adc49258432e P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:23.237193 21822 raft_consensus.cc:515] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:23.238232 21521 tablet_bootstrap.cc:492] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:23.238794 21521 ts_tablet_manager.cc:1403] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.178s	user 0.012s	sys 0.010s
I20260709 13:22:23.239724 21822 leader_election.cc:290] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:23.247279 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fff6b36d1984d38b6f791e0ac15f56b" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:23.247996 21324 raft_consensus.cc:3060] T 1fff6b36d1984d38b6f791e0ac15f56b P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:23.247902 21521 raft_consensus.cc:359] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:23.248792 21521 raft_consensus.cc:385] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:23.249125 21521 raft_consensus.cc:740] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:23.252864 21521 consensus_queue.cc:260] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:23.247507 21518 tablet_bootstrap.cc:492] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:23.258538 21521 ts_tablet_manager.cc:1434] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:22:23.259698 21521 tablet_bootstrap.cc:492] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:23.267083 21324 raft_consensus.cc:2468] T 1fff6b36d1984d38b6f791e0ac15f56b P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:23.268337 21215 leader_election.cc:304] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:23.269196 21911 raft_consensus.cc:2804] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:23.269692 21911 raft_consensus.cc:697] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:23.273025 21911 consensus_queue.cc:237] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:23.283752 21518 tablet_bootstrap.cc:654] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:23.278096 21472 raft_consensus.cc:2468] T d2bf69be9e9240b5ac99adc49258432e P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:23.293473 21669 raft_consensus.cc:493] T 3f3bf699b02c46c78d330dc7f9d3314b P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:23.295305 21520 tablet_bootstrap.cc:492] T d2bf69be9e9240b5ac99adc49258432e P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:23.296761 21519 tablet_bootstrap.cc:654] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:23.303694 21520 ts_tablet_manager.cc:1403] T d2bf69be9e9240b5ac99adc49258432e P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.138s	user 0.011s	sys 0.009s
I20260709 13:22:23.305022 21363 leader_election.cc:304] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:23.303318 21669 raft_consensus.cc:515] T 3f3bf699b02c46c78d330dc7f9d3314b P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:23.309101 21925 raft_consensus.cc:2804] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:23.309986 21925 raft_consensus.cc:697] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Leader. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:23.308334 21148 catalog_manager.cc:5697] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:23.310854 21925 consensus_queue.cc:237] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:23.310873 21669 leader_election.cc:290] T 3f3bf699b02c46c78d330dc7f9d3314b P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:23.316828 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3bf699b02c46c78d330dc7f9d3314b" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:23.317890 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08649134c0c94e7caa55fcc7513eec5d" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:23.318382 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3bf699b02c46c78d330dc7f9d3314b" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:23.318739 21248 raft_consensus.cc:2393] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5109d6d6d9404ba39c53508a81fe9f1c in current term 1: Already voted for candidate 74e2ff1a68ac4c849ee6397ebb011375 in this term.
W20260709 13:22:23.320406 21439 leader_election.cc:343] T 3f3bf699b02c46c78d330dc7f9d3314b P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:23.325001 21438 leader_election.cc:343] T 3f3bf699b02c46c78d330dc7f9d3314b P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:23.325361 21438 leader_election.cc:304] T 3f3bf699b02c46c78d330dc7f9d3314b P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:23.326222 21669 raft_consensus.cc:2749] T 3f3bf699b02c46c78d330dc7f9d3314b P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:23.331950 21887 raft_consensus.cc:493] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:23.334542 21521 tablet_bootstrap.cc:654] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:23.335757 21289 leader_election.cc:304] T 08649134c0c94e7caa55fcc7513eec5d P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636
I20260709 13:22:23.337152 21832 raft_consensus.cc:3060] T 08649134c0c94e7caa55fcc7513eec5d P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:23.332546 21887 raft_consensus.cc:515] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:23.340548 21887 leader_election.cc:290] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:23.335816 21470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fff6b36d1984d38b6f791e0ac15f56b" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:23.341969 21470 raft_consensus.cc:3060] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:23.342149 21923 raft_consensus.cc:493] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:23.342746 21923 raft_consensus.cc:515] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:23.345780 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0413c4b049a8412b9b3b8a36bb5d67fd" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
W20260709 13:22:23.348665 21439 leader_election.cc:343] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:23.349575 21326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169fb713efc445fb1f0338c7c11203a" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
W20260709 13:22:23.352005 21364 leader_election.cc:343] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:23.349043 21520 raft_consensus.cc:359] T d2bf69be9e9240b5ac99adc49258432e P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:23.354202 21520 raft_consensus.cc:385] T d2bf69be9e9240b5ac99adc49258432e P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:23.354557 21520 raft_consensus.cc:740] T d2bf69be9e9240b5ac99adc49258432e P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:23.358493 21518 tablet_bootstrap.cc:492] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:23.359203 21518 ts_tablet_manager.cc:1403] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.112s	user 0.009s	sys 0.006s
I20260709 13:22:23.365599 21470 raft_consensus.cc:2468] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:23.365170 21520 consensus_queue.cc:260] T d2bf69be9e9240b5ac99adc49258432e P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:23.368724 21520 ts_tablet_manager.cc:1434] T d2bf69be9e9240b5ac99adc49258432e P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.065s	user 0.008s	sys 0.000s
I20260709 13:22:23.383610 21923 leader_election.cc:290] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:23.395751 21832 raft_consensus.cc:2749] T 08649134c0c94e7caa55fcc7513eec5d P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:23.397758 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0413c4b049a8412b9b3b8a36bb5d67fd" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:23.399840 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169fb713efc445fb1f0338c7c11203a" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:23.400516 21251 raft_consensus.cc:2468] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
I20260709 13:22:23.400627 21518 raft_consensus.cc:359] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:23.401726 21518 raft_consensus.cc:385] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:22:23.401887 21437 leader_election.cc:343] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:23.402156 21518 raft_consensus.cc:740] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:23.402493 21437 leader_election.cc:304] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636
I20260709 13:22:23.404951 21923 raft_consensus.cc:2749] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:23.406877 21363 leader_election.cc:304] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:23.407985 21887 raft_consensus.cc:2804] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:23.407644 21518 consensus_queue.cc:260] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:23.408490 21887 raft_consensus.cc:493] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:23.408999 21887 raft_consensus.cc:3060] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:23.427901 21518 ts_tablet_manager.cc:1434] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.003s
I20260709 13:22:23.429410 21518 tablet_bootstrap.cc:492] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:23.434840 21831 raft_consensus.cc:493] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:23.435499 21831 raft_consensus.cc:515] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:23.450193 21831 leader_election.cc:290] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:23.452158 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37dadd42cd04be5a0e0b339df630a04" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
W20260709 13:22:23.460693 21216 leader_election.cc:343] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:23.459954 21472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37dadd42cd04be5a0e0b339df630a04" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:23.472000 21519 tablet_bootstrap.cc:492] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea: No bootstrap required, opened a new log
I20260709 13:22:23.472237 21520 tablet_bootstrap.cc:492] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:23.472532 21519 ts_tablet_manager.cc:1403] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent bootstrapping tablet: real 0.248s	user 0.029s	sys 0.052s
I20260709 13:22:23.472337 21823 consensus_queue.cc:1048] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:23.470971 21148 catalog_manager.cc:5697] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "74e2ff1a68ac4c849ee6397ebb011375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:23.471997 21472 raft_consensus.cc:2468] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:23.478065 21519 raft_consensus.cc:359] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:23.478845 21519 raft_consensus.cc:385] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:23.479176 21519 raft_consensus.cc:740] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Initialized, Role: FOLLOWER
I20260709 13:22:23.480365 21519 consensus_queue.cc:260] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:23.482741 21519 ts_tablet_manager.cc:1434] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:22:23.483242 21887 raft_consensus.cc:515] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:23.496595 21520 tablet_bootstrap.cc:654] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:23.500572 21248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169fb713efc445fb1f0338c7c11203a" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:23.501175 21248 raft_consensus.cc:3060] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:23.501681 21887 leader_election.cc:290] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:23.502038 21214 leader_election.cc:304] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:23.502777 21521 tablet_bootstrap.cc:492] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:23.503341 21521 ts_tablet_manager.cc:1403] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.244s	user 0.020s	sys 0.017s
I20260709 13:22:23.503494 21911 raft_consensus.cc:2804] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:23.503990 21911 raft_consensus.cc:493] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:23.504807 21911 raft_consensus.cc:3060] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:23.505172 21322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169fb713efc445fb1f0338c7c11203a" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
W20260709 13:22:23.506398 21364 leader_election.cc:343] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:23.508818 21521 raft_consensus.cc:359] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:23.509595 21521 raft_consensus.cc:385] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:23.509936 21521 raft_consensus.cc:740] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:23.510995 21248 raft_consensus.cc:2468] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:23.510742 21521 consensus_queue.cc:260] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:23.512290 21363 leader_election.cc:304] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:23.513052 21521 ts_tablet_manager.cc:1434] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:22:23.514118 21521 tablet_bootstrap.cc:492] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:23.516026 21518 tablet_bootstrap.cc:654] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:23.522809 21831 raft_consensus.cc:493] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:23.523383 21831 raft_consensus.cc:515] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:23.526345 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3449b9c781a847b9ba63813b577eef81" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
W20260709 13:22:23.528110 21216 leader_election.cc:343] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:23.528949 21322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3449b9c781a847b9ba63813b577eef81" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:23.530543 21925 consensus_queue.cc:1048] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:22:23.533679 21215 leader_election.cc:343] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:23.534168 21215 leader_election.cc:304] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:23.535466 21914 raft_consensus.cc:2749] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:23.536454 21831 leader_election.cc:290] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:23.521229 21521 tablet_bootstrap.cc:654] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:23.541698 21518 tablet_bootstrap.cc:492] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:23.542285 21518 ts_tablet_manager.cc:1403] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.113s	user 0.014s	sys 0.000s
I20260709 13:22:23.549885 21834 consensus_queue.cc:1048] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:23.513448 21887 raft_consensus.cc:2804] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:23.553136 21887 raft_consensus.cc:697] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Leader. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:23.558403 21518 raft_consensus.cc:359] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:23.559243 21518 raft_consensus.cc:385] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:23.559541 21518 raft_consensus.cc:740] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:23.560276 21518 consensus_queue.cc:260] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:23.619625 21518 ts_tablet_manager.cc:1434] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.077s	user 0.008s	sys 0.000s
I20260709 13:22:23.627352 21520 tablet_bootstrap.cc:492] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:23.628168 21520 ts_tablet_manager.cc:1403] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.167s	user 0.015s	sys 0.043s
I20260709 13:22:23.513441 21911 raft_consensus.cc:515] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:23.642823 21520 raft_consensus.cc:359] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:23.643896 21520 raft_consensus.cc:385] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:23.644327 21520 raft_consensus.cc:740] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:23.646288 21520 consensus_queue.cc:260] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:23.647508 21911 leader_election.cc:290] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:23.555258 21887 consensus_queue.cc:237] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:23.648720 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37dadd42cd04be5a0e0b339df630a04" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:23.651938 21520 ts_tablet_manager.cc:1434] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.001s
W20260709 13:22:23.652220 21216 leader_election.cc:343] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:23.653546 21520 tablet_bootstrap.cc:492] T a169fb713efc445fb1f0338c7c11203a P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:23.654824 21925 consensus_queue.cc:1048] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:23.658263 21472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37dadd42cd04be5a0e0b339df630a04" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:23.662096 21472 raft_consensus.cc:3060] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:23.677704 21834 raft_consensus.cc:493] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:23.678563 21834 raft_consensus.cc:515] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:23.683364 21518 tablet_bootstrap.cc:492] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:23.686969 21834 leader_election.cc:290] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:23.689164 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3449b9c781a847b9ba63813b577eef81" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:23.689929 21251 raft_consensus.cc:2468] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
I20260709 13:22:23.696511 21363 leader_election.cc:304] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:23.702301 21823 raft_consensus.cc:2804] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:23.702768 21823 raft_consensus.cc:493] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:23.703166 21823 raft_consensus.cc:3060] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:23.705173 21518 tablet_bootstrap.cc:654] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:23.707640 21322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3449b9c781a847b9ba63813b577eef81" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:23.708083 21472 raft_consensus.cc:2468] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
W20260709 13:22:23.709741 21364 leader_election.cc:343] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:23.725018 21520 tablet_bootstrap.cc:654] T a169fb713efc445fb1f0338c7c11203a P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:23.731665 21214 leader_election.cc:304] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:23.742470 21944 raft_consensus.cc:2804] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:23.752300 21944 raft_consensus.cc:697] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:23.767055 21822 consensus_queue.cc:1048] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
W20260709 13:22:23.778221 21196 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:23.781224 21521 tablet_bootstrap.cc:492] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:23.781831 21521 ts_tablet_manager.cc:1403] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.268s	user 0.081s	sys 0.082s
I20260709 13:22:23.787209 21520 tablet_bootstrap.cc:492] T a169fb713efc445fb1f0338c7c11203a P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:23.787242 21518 tablet_bootstrap.cc:492] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:23.787845 21520 ts_tablet_manager.cc:1403] T a169fb713efc445fb1f0338c7c11203a P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.135s	user 0.015s	sys 0.003s
I20260709 13:22:23.787860 21518 ts_tablet_manager.cc:1403] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.126s	user 0.018s	sys 0.010s
I20260709 13:22:23.784579 21944 consensus_queue.cc:237] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:23.791302 21823 raft_consensus.cc:515] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:23.803423 21521 raft_consensus.cc:359] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:23.804577 21521 raft_consensus.cc:385] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:23.804952 21521 raft_consensus.cc:740] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:23.811595 21520 raft_consensus.cc:359] T a169fb713efc445fb1f0338c7c11203a P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:23.812747 21520 raft_consensus.cc:385] T a169fb713efc445fb1f0338c7c11203a P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:23.813117 21520 raft_consensus.cc:740] T a169fb713efc445fb1f0338c7c11203a P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:23.813901 21521 consensus_queue.cc:260] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:23.817189 21521 ts_tablet_manager.cc:1434] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 13:22:23.822157 21521 tablet_bootstrap.cc:492] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:23.848846 21520 consensus_queue.cc:260] T a169fb713efc445fb1f0338c7c11203a P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:23.853920 21518 raft_consensus.cc:359] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:23.855015 21518 raft_consensus.cc:385] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:23.855418 21518 raft_consensus.cc:740] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:23.848088 21836 raft_consensus.cc:493] T d2bf69be9e9240b5ac99adc49258432e P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:23.861353 21836 raft_consensus.cc:515] T d2bf69be9e9240b5ac99adc49258432e P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:23.863619 21836 leader_election.cc:290] T d2bf69be9e9240b5ac99adc49258432e P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:23.865661 21472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bf69be9e9240b5ac99adc49258432e" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:23.866475 21472 raft_consensus.cc:2393] T d2bf69be9e9240b5ac99adc49258432e P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5109d6d6d9404ba39c53508a81fe9f1c in current term 1: Already voted for candidate 74e2ff1a68ac4c849ee6397ebb011375 in this term.
I20260709 13:22:23.867486 21893 raft_consensus.cc:493] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:23.868010 21893 raft_consensus.cc:515] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:23.871439 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b6856cbfc84013bdfa6979b0366468" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:23.872602 21823 leader_election.cc:290] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:23.853066 21145 catalog_manager.cc:5697] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "74e2ff1a68ac4c849ee6397ebb011375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: HEALTHY } } }
W20260709 13:22:23.873066 21437 leader_election.cc:343] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:23.860080 21518 consensus_queue.cc:260] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:23.875573 21822 consensus_queue.cc:1048] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:23.875984 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3449b9c781a847b9ba63813b577eef81" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:23.876129 21323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b6856cbfc84013bdfa6979b0366468" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:23.871030 21893 leader_election.cc:290] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:23.878639 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2bf69be9e9240b5ac99adc49258432e" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:23.880784 21521 tablet_bootstrap.cc:654] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
W20260709 13:22:23.882447 21438 leader_election.cc:343] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:23.882929 21438 leader_election.cc:304] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
W20260709 13:22:23.884022 21364 leader_election.cc:343] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:23.886157 21291 leader_election.cc:304] T d2bf69be9e9240b5ac99adc49258432e P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:23.876566 21518 ts_tablet_manager.cc:1434] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.088s	user 0.006s	sys 0.001s
I20260709 13:22:23.889286 21669 raft_consensus.cc:2749] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:23.894217 21520 ts_tablet_manager.cc:1434] T a169fb713efc445fb1f0338c7c11203a P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.106s	user 0.012s	sys 0.000s
I20260709 13:22:23.896664 21518 tablet_bootstrap.cc:492] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:23.893235 21867 raft_consensus.cc:3060] T d2bf69be9e9240b5ac99adc49258432e P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:23.903748 21249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3449b9c781a847b9ba63813b577eef81" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:23.904450 21249 raft_consensus.cc:3060] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:23.893074 21886 raft_consensus.cc:493] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:23.908942 21886 raft_consensus.cc:515] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:23.910974 21886 leader_election.cc:290] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:23.913506 21472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "665861bf917f4d42aa1252c374a656ea" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:23.914288 21472 raft_consensus.cc:2468] T 665861bf917f4d42aa1252c374a656ea P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
I20260709 13:22:23.916412 21867 raft_consensus.cc:2749] T d2bf69be9e9240b5ac99adc49258432e P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:23.917941 21363 leader_election.cc:304] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:23.919203 21249 raft_consensus.cc:2468] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:23.934899 21363 leader_election.cc:304] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:23.935932 21823 raft_consensus.cc:2804] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:23.936510 21823 raft_consensus.cc:697] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Leader. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:23.938130 21823 consensus_queue.cc:237] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:23.931010 21886 raft_consensus.cc:2804] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:23.939505 21886 raft_consensus.cc:493] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:23.939687 21518 tablet_bootstrap.cc:654] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:23.939874 21886 raft_consensus.cc:3060] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:23.948467 21326 raft_consensus.cc:3060] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:23.952468 21317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "665861bf917f4d42aa1252c374a656ea" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:23.955191 21520 tablet_bootstrap.cc:492] T 665861bf917f4d42aa1252c374a656ea P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:23.965761 21823 consensus_queue.cc:1048] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:23.971922 21518 tablet_bootstrap.cc:492] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:23.972446 21518 ts_tablet_manager.cc:1403] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.076s	user 0.012s	sys 0.008s
I20260709 13:22:23.972069 21886 raft_consensus.cc:515] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:23.976369 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "665861bf917f4d42aa1252c374a656ea" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
W20260709 13:22:23.981876 21364 leader_election.cc:343] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:23.995050 21147 catalog_manager.cc:5697] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:23.988204 21887 consensus_queue.cc:1048] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:22:24.012281 21470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "665861bf917f4d42aa1252c374a656ea" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:24.013124 21470 raft_consensus.cc:3060] T 665861bf917f4d42aa1252c374a656ea P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:24.012281 21521 tablet_bootstrap.cc:492] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:24.016072 21521 ts_tablet_manager.cc:1403] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.194s	user 0.018s	sys 0.004s
I20260709 13:22:24.022889 21518 raft_consensus.cc:359] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:24.023995 21518 raft_consensus.cc:385] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:24.024231 21518 raft_consensus.cc:740] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:24.030710 21521 raft_consensus.cc:359] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:24.031466 21521 raft_consensus.cc:385] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:24.031718 21521 raft_consensus.cc:740] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:24.032325 21521 consensus_queue.cc:260] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:24.034693 21147 catalog_manager.cc:5697] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "74e2ff1a68ac4c849ee6397ebb011375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:24.007216 21886 leader_election.cc:290] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Requested vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:24.046492 21521 ts_tablet_manager.cc:1434] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.030s	user 0.003s	sys 0.003s
I20260709 13:22:24.048475 21521 tablet_bootstrap.cc:492] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:24.052323 21520 tablet_bootstrap.cc:654] T 665861bf917f4d42aa1252c374a656ea P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:24.054780 21521 tablet_bootstrap.cc:654] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:24.061255 21518 consensus_queue.cc:260] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:24.065322 21470 raft_consensus.cc:2468] T 665861bf917f4d42aa1252c374a656ea P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:24.071038 21363 leader_election.cc:304] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:24.075717 21867 raft_consensus.cc:493] T a169fb713efc445fb1f0338c7c11203a P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:24.089159 21867 raft_consensus.cc:515] T a169fb713efc445fb1f0338c7c11203a P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:24.092003 21867 leader_election.cc:290] T a169fb713efc445fb1f0338c7c11203a P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:24.093174 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169fb713efc445fb1f0338c7c11203a" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:24.076265 21914 raft_consensus.cc:493] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:24.098003 21914 raft_consensus.cc:515] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:24.098711 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169fb713efc445fb1f0338c7c11203a" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:24.099509 21251 raft_consensus.cc:2393] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5109d6d6d9404ba39c53508a81fe9f1c in current term 1: Already voted for candidate 74e2ff1a68ac4c849ee6397ebb011375 in this term.
I20260709 13:22:24.100793 21289 leader_election.cc:304] T a169fb713efc445fb1f0338c7c11203a P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636
I20260709 13:22:24.101230 21914 leader_election.cc:290] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:24.102394 21867 raft_consensus.cc:3060] T a169fb713efc445fb1f0338c7c11203a P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:24.102243 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa2f735fa7d14b1e876c094ffbfb2c9d" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
W20260709 13:22:24.103999 21215 leader_election.cc:343] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:24.119848 21363 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36863, user_credentials={real_user=slave}} blocked reactor thread for 48906.4us
I20260709 13:22:24.122146 21887 raft_consensus.cc:2804] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:24.133397 21887 raft_consensus.cc:697] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Leader. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:24.134328 21887 consensus_queue.cc:237] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:24.135001 21520 tablet_bootstrap.cc:492] T 665861bf917f4d42aa1252c374a656ea P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:24.137311 21520 ts_tablet_manager.cc:1403] T 665861bf917f4d42aa1252c374a656ea P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.207s	user 0.006s	sys 0.031s
I20260709 13:22:24.131714 21518 ts_tablet_manager.cc:1434] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.159s	user 0.006s	sys 0.001s
I20260709 13:22:24.156776 21518 tablet_bootstrap.cc:492] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:24.157524 21520 raft_consensus.cc:359] T 665861bf917f4d42aa1252c374a656ea P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:24.158295 21520 raft_consensus.cc:385] T 665861bf917f4d42aa1252c374a656ea P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:24.158600 21520 raft_consensus.cc:740] T 665861bf917f4d42aa1252c374a656ea P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:24.165395 21520 consensus_queue.cc:260] T 665861bf917f4d42aa1252c374a656ea P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:24.167564 21520 ts_tablet_manager.cc:1434] T 665861bf917f4d42aa1252c374a656ea P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 13:22:24.168511 21520 tablet_bootstrap.cc:492] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:24.169234 21867 raft_consensus.cc:2749] T a169fb713efc445fb1f0338c7c11203a P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:24.173916 21520 tablet_bootstrap.cc:654] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:24.184445 21521 tablet_bootstrap.cc:492] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:24.185071 21521 ts_tablet_manager.cc:1403] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.137s	user 0.020s	sys 0.028s
I20260709 13:22:24.203692 21399 raft_consensus.cc:3060] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:24.196622 21397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa2f735fa7d14b1e876c094ffbfb2c9d" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
W20260709 13:22:24.211465 21216 leader_election.cc:343] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:24.212478 21216 leader_election.cc:304] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:24.212119 21944 raft_consensus.cc:493] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:24.214927 21944 raft_consensus.cc:515] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:24.214350 21831 raft_consensus.cc:2749] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:24.222199 21521 raft_consensus.cc:359] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:24.222942 21521 raft_consensus.cc:385] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:24.223223 21521 raft_consensus.cc:740] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:24.223866 21521 consensus_queue.cc:260] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:24.224998 21320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280cfe3edb3f490c8de46d0c80f6da5a" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:24.225975 21521 ts_tablet_manager.cc:1434] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.003s
W20260709 13:22:24.228554 21215 leader_election.cc:343] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:24.233299 21831 consensus_queue.cc:1048] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:24.233873 21469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280cfe3edb3f490c8de46d0c80f6da5a" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:24.234566 21469 raft_consensus.cc:2468] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:24.234812 21867 raft_consensus.cc:493] T 665861bf917f4d42aa1252c374a656ea P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:24.235353 21867 raft_consensus.cc:515] T 665861bf917f4d42aa1252c374a656ea P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:24.238945 21469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "665861bf917f4d42aa1252c374a656ea" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:24.239851 21520 tablet_bootstrap.cc:492] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:24.240038 21867 leader_election.cc:290] T 665861bf917f4d42aa1252c374a656ea P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:24.240459 21214 leader_election.cc:304] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:24.241338 21822 raft_consensus.cc:2804] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:24.241739 21822 raft_consensus.cc:493] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:24.242086 21822 raft_consensus.cc:3060] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:24.240402 21520 ts_tablet_manager.cc:1403] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.072s	user 0.019s	sys 0.017s
I20260709 13:22:24.217267 21944 leader_election.cc:290] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:24.239843 21469 raft_consensus.cc:2393] T 665861bf917f4d42aa1252c374a656ea P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5109d6d6d9404ba39c53508a81fe9f1c in current term 1: Already voted for candidate 74e2ff1a68ac4c849ee6397ebb011375 in this term.
I20260709 13:22:24.244549 21886 consensus_queue.cc:1048] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:24.245659 21518 tablet_bootstrap.cc:654] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:24.252667 21520 raft_consensus.cc:359] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:24.253438 21520 raft_consensus.cc:385] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:24.253744 21520 raft_consensus.cc:740] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:24.254647 21521 tablet_bootstrap.cc:492] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:24.257169 21520 consensus_queue.cc:260] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:24.280938 21518 tablet_bootstrap.cc:492] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:24.289458 21518 ts_tablet_manager.cc:1403] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.133s	user 0.024s	sys 0.015s
I20260709 13:22:24.310451 21520 ts_tablet_manager.cc:1434] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.068s	user 0.006s	sys 0.000s
I20260709 13:22:24.313143 21520 tablet_bootstrap.cc:492] T 3449b9c781a847b9ba63813b577eef81 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:24.323791 21521 tablet_bootstrap.cc:654] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:24.321947 21145 catalog_manager.cc:5697] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "74e2ff1a68ac4c849ee6397ebb011375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:24.315609 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "665861bf917f4d42aa1252c374a656ea" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:24.336892 21959 raft_consensus.cc:493] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:24.337615 21944 consensus_queue.cc:1048] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:24.337479 21959 raft_consensus.cc:515] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:24.341836 21291 leader_election.cc:304] T 665861bf917f4d42aa1252c374a656ea P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:24.343616 21867 raft_consensus.cc:3060] T 665861bf917f4d42aa1252c374a656ea P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:24.343664 21959 leader_election.cc:290] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:24.344215 21326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3bf699b02c46c78d330dc7f9d3314b" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
W20260709 13:22:24.345420 21364 leader_election.cc:343] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:24.350934 21364 consensus_peers.cc:597] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:24.353736 21822 raft_consensus.cc:515] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:24.355254 21470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3bf699b02c46c78d330dc7f9d3314b" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:24.356000 21470 raft_consensus.cc:2468] T 3f3bf699b02c46c78d330dc7f9d3314b P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
I20260709 13:22:24.361603 21470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280cfe3edb3f490c8de46d0c80f6da5a" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:24.362001 21363 leader_election.cc:304] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:24.362650 21470 raft_consensus.cc:3060] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:24.362890 21974 raft_consensus.cc:2804] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:24.363770 21974 raft_consensus.cc:493] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:24.364223 21974 raft_consensus.cc:3060] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:24.371031 21886 consensus_queue.cc:1048] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:24.374616 21886 consensus_queue.cc:1048] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:24.378290 21520 tablet_bootstrap.cc:654] T 3449b9c781a847b9ba63813b577eef81 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:24.389072 21867 raft_consensus.cc:2749] T 665861bf917f4d42aa1252c374a656ea P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:24.402343 21521 tablet_bootstrap.cc:492] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:24.402890 21521 ts_tablet_manager.cc:1403] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.148s	user 0.015s	sys 0.017s
I20260709 13:22:24.423277 21521 raft_consensus.cc:359] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:24.424062 21521 raft_consensus.cc:385] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:24.424340 21521 raft_consensus.cc:740] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:24.399814 21914 raft_consensus.cc:493] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:24.425302 21914 raft_consensus.cc:515] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:24.441550 21521 consensus_queue.cc:260] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:24.498925 21518 raft_consensus.cc:359] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:24.500066 21518 raft_consensus.cc:385] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:24.500502 21518 raft_consensus.cc:740] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:24.510236 21518 consensus_queue.cc:260] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:24.513291 21518 ts_tablet_manager.cc:1434] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.223s	user 0.009s	sys 0.000s
I20260709 13:22:24.446879 21822 leader_election.cc:290] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:24.514940 21470 raft_consensus.cc:2468] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:24.516423 21974 raft_consensus.cc:515] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:24.559840 21974 leader_election.cc:290] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Requested vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
W20260709 13:22:24.524967 21981 log.cc:927] Time spent T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:22:24.562991 21470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3bf699b02c46c78d330dc7f9d3314b" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:24.581689 21518 tablet_bootstrap.cc:492] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
I20260709 13:22:24.584306 21214 leader_election.cc:304] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:24.590608 21836 raft_consensus.cc:493] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:24.591202 21836 raft_consensus.cc:515] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:24.578745 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280cfe3edb3f490c8de46d0c80f6da5a" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:24.608901 21521 ts_tablet_manager.cc:1434] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.206s	user 0.011s	sys 0.018s
I20260709 13:22:24.610710 21521 tablet_bootstrap.cc:492] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:24.584738 21470 raft_consensus.cc:3060] T 3f3bf699b02c46c78d330dc7f9d3314b P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:24.620886 21520 tablet_bootstrap.cc:492] T 3449b9c781a847b9ba63813b577eef81 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:24.621697 21520 ts_tablet_manager.cc:1403] T 3449b9c781a847b9ba63813b577eef81 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.309s	user 0.064s	sys 0.057s
I20260709 13:22:24.626017 21986 raft_consensus.cc:493] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:24.626780 21986 raft_consensus.cc:515] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:24.628476 21520 raft_consensus.cc:359] T 3449b9c781a847b9ba63813b577eef81 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:24.629493 21520 raft_consensus.cc:385] T 3449b9c781a847b9ba63813b577eef81 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:24.629969 21520 raft_consensus.cc:740] T 3449b9c781a847b9ba63813b577eef81 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:24.630709 21831 raft_consensus.cc:2804] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:24.631047 21520 consensus_queue.cc:260] T 3449b9c781a847b9ba63813b577eef81 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:24.633098 21521 tablet_bootstrap.cc:654] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:24.631305 21831 raft_consensus.cc:697] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:24.613057 21324 raft_consensus.cc:3060] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:24.638831 21831 consensus_queue.cc:237] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:24.631711 21836 leader_election.cc:290] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:24.641388 21319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3bf699b02c46c78d330dc7f9d3314b" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:24.657424 21520 ts_tablet_manager.cc:1434] T 3449b9c781a847b9ba63813b577eef81 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 13:22:24.706507 21520 tablet_bootstrap.cc:492] T 3f3bf699b02c46c78d330dc7f9d3314b P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
W20260709 13:22:24.706661 21214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36863, user_credentials={real_user=slave}} blocked reactor thread for 122459us
I20260709 13:22:24.708946 21472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b6856cbfc84013bdfa6979b0366468" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:24.709578 21472 raft_consensus.cc:2468] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:24.685334 21986 leader_election.cc:290] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:24.682667 21324 raft_consensus.cc:2468] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
W20260709 13:22:24.694598 21438 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:38825, user_credentials={real_user=slave}} blocked reactor thread for 80630.9us
I20260709 13:22:24.704386 21914 leader_election.cc:290] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:24.718034 21251 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280cfe3edb3f490c8de46d0c80f6da5a" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:24.706863 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d5ddbaefa064931963bc1e27ac99903" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:24.722285 21472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d5ddbaefa064931963bc1e27ac99903" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:24.723019 21472 raft_consensus.cc:2468] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:24.718600 21214 leader_election.cc:304] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:24.731580 21914 raft_consensus.cc:2804] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:24.725198 21396 raft_consensus.cc:2468] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:24.739593 21214 leader_election.cc:304] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 
W20260709 13:22:24.753048 21364 leader_election.cc:343] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:24.793737 21214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36863, user_credentials={real_user=slave}} blocked reactor thread for 54280.7us
I20260709 13:22:24.794433 21518 tablet_bootstrap.cc:654] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:24.812227 21520 tablet_bootstrap.cc:654] T 3f3bf699b02c46c78d330dc7f9d3314b P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:24.803347 21973 consensus_queue.cc:1048] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:22:24.776813 21470 raft_consensus.cc:2468] T 3f3bf699b02c46c78d330dc7f9d3314b P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:24.756554 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b6856cbfc84013bdfa6979b0366468" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:24.746049 21914 raft_consensus.cc:493] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:24.803313 21944 raft_consensus.cc:2804] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:24.830756 21944 raft_consensus.cc:493] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:24.830942 21326 raft_consensus.cc:3060] T 3449b9c781a847b9ba63813b577eef81 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:24.831187 21944 raft_consensus.cc:3060] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:24.833752 21887 consensus_queue.cc:1048] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:22:24.830410 21914 raft_consensus.cc:3060] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:24.848454 21216 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:43181, user_credentials={real_user=slave}} blocked reactor thread for 88019.7us
W20260709 13:22:24.848853 21215 leader_election.cc:343] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:24.879815 21473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280cfe3edb3f490c8de46d0c80f6da5a" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:24.880746 21473 raft_consensus.cc:2393] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5109d6d6d9404ba39c53508a81fe9f1c in current term 1: Already voted for candidate ec879c0528c44465916efc6c33dde636 in this term.
I20260709 13:22:24.900044 21518 tablet_bootstrap.cc:492] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:24.900838 21518 ts_tablet_manager.cc:1403] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.322s	user 0.027s	sys 0.002s
W20260709 13:22:24.913931 21363 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:35963, user_credentials={real_user=slave}} blocked reactor thread for 87021.1us
W20260709 13:22:24.928098 21289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:35963, user_credentials={real_user=slave}} blocked reactor thread for 74416.2us
I20260709 13:22:24.931668 21521 tablet_bootstrap.cc:492] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636: No bootstrap required, opened a new log
I20260709 13:22:24.932235 21521 ts_tablet_manager.cc:1403] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.322s	user 0.006s	sys 0.029s
I20260709 13:22:24.933525 21520 tablet_bootstrap.cc:492] T 3f3bf699b02c46c78d330dc7f9d3314b P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:24.934127 21520 ts_tablet_manager.cc:1403] T 3f3bf699b02c46c78d330dc7f9d3314b P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.232s	user 0.015s	sys 0.010s
I20260709 13:22:24.936213 21289 leader_election.cc:304] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636
I20260709 13:22:24.940078 21520 raft_consensus.cc:359] T 3f3bf699b02c46c78d330dc7f9d3314b P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:24.944176 21520 raft_consensus.cc:385] T 3f3bf699b02c46c78d330dc7f9d3314b P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:24.944298 21836 raft_consensus.cc:2749] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:24.944535 21520 raft_consensus.cc:740] T 3f3bf699b02c46c78d330dc7f9d3314b P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:24.944851 21518 raft_consensus.cc:359] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:24.941341 21521 raft_consensus.cc:359] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:24.946329 21521 raft_consensus.cc:385] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:24.946650 21521 raft_consensus.cc:740] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:24.949080 21518 raft_consensus.cc:385] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:24.949477 21518 raft_consensus.cc:740] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:24.945429 21520 consensus_queue.cc:260] T 3f3bf699b02c46c78d330dc7f9d3314b P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:24.950138 21518 consensus_queue.cc:260] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:24.957628 21521 consensus_queue.cc:260] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:24.948683 21914 raft_consensus.cc:515] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:24.967607 21320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b6856cbfc84013bdfa6979b0366468" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
W20260709 13:22:24.971073 21215 leader_election.cc:343] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:24.971832 21914 leader_election.cc:290] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
W20260709 13:22:24.980736 21196 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:24.982415 21520 ts_tablet_manager.cc:1434] T 3f3bf699b02c46c78d330dc7f9d3314b P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.000s
I20260709 13:22:24.982803 21521 ts_tablet_manager.cc:1434] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.000s
I20260709 13:22:24.982470 21363 leader_election.cc:304] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:24.994822 21473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b6856cbfc84013bdfa6979b0366468" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:24.995545 21473 raft_consensus.cc:3060] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:25.000612 21518 ts_tablet_manager.cc:1434] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.099s	user 0.007s	sys 0.001s
I20260709 13:22:25.010366 22006 raft_consensus.cc:2804] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:25.011047 22006 raft_consensus.cc:697] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Leader. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:25.012009 22006 consensus_queue.cc:237] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:25.015105 21944 raft_consensus.cc:515] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:25.017875 21944 leader_election.cc:290] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:25.018831 21397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d5ddbaefa064931963bc1e27ac99903" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:25.019721 21397 raft_consensus.cc:3060] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:25.039292 21473 raft_consensus.cc:2468] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:25.063361 21518 tablet_bootstrap.cc:492] T 0413c4b049a8412b9b3b8a36bb5d67fd P 74e2ff1a68ac4c849ee6397ebb011375: Bootstrap starting.
W20260709 13:22:25.066578 21214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36863, user_credentials={real_user=slave}} blocked reactor thread for 58949.9us
W20260709 13:22:25.134454 21215 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:38825, user_credentials={real_user=slave}} blocked reactor thread for 77182.3us
W20260709 13:22:25.134318 21216 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:43181, user_credentials={real_user=slave}} blocked reactor thread for 106558us
I20260709 13:22:25.140595 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d5ddbaefa064931963bc1e27ac99903" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:25.141480 21474 raft_consensus.cc:3060] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:25.143154 21834 raft_consensus.cc:493] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:25.143749 21397 raft_consensus.cc:2468] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:25.143661 21834 raft_consensus.cc:515] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:25.146930 21214 leader_election.cc:304] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:25.150458 21216 leader_election.cc:304] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:25.156987 21822 raft_consensus.cc:2804] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:25.157508 21822 raft_consensus.cc:697] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:25.158409 21822 consensus_queue.cc:237] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:25.159453 21988 raft_consensus.cc:2804] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:25.160035 21988 raft_consensus.cc:697] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:25.160820 21988 consensus_queue.cc:237] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:25.161168 21834 leader_election.cc:290] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:25.162989 21246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa2f735fa7d14b1e876c094ffbfb2c9d" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:25.165313 21246 raft_consensus.cc:2468] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 0.
I20260709 13:22:25.172406 21363 leader_election.cc:304] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:25.188961 21520 tablet_bootstrap.cc:492] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:25.199589 21474 raft_consensus.cc:2468] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:25.177629 21887 raft_consensus.cc:2804] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:25.216182 21518 tablet_bootstrap.cc:654] T 0413c4b049a8412b9b3b8a36bb5d67fd P 74e2ff1a68ac4c849ee6397ebb011375: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:25.214964 21148 catalog_manager.cc:5697] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:25.207302 21887 raft_consensus.cc:493] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:25.226472 21987 raft_consensus.cc:493] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:25.227075 21987 raft_consensus.cc:515] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:25.229117 21987 leader_election.cc:290] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:25.237255 21887 raft_consensus.cc:3060] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:25.209200 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa2f735fa7d14b1e876c094ffbfb2c9d" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
W20260709 13:22:25.262243 21364 leader_election.cc:343] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:25.275739 21520 tablet_bootstrap.cc:654] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:25.300580 21914 consensus_queue.cc:1048] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:25.305373 21518 tablet_bootstrap.cc:492] T 0413c4b049a8412b9b3b8a36bb5d67fd P 74e2ff1a68ac4c849ee6397ebb011375: No bootstrap required, opened a new log
I20260709 13:22:25.306341 21518 ts_tablet_manager.cc:1403] T 0413c4b049a8412b9b3b8a36bb5d67fd P 74e2ff1a68ac4c849ee6397ebb011375: Time spent bootstrapping tablet: real 0.243s	user 0.014s	sys 0.002s
I20260709 13:22:25.309912 21887 raft_consensus.cc:515] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:25.310056 21518 raft_consensus.cc:359] T 0413c4b049a8412b9b3b8a36bb5d67fd P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:25.311093 21518 raft_consensus.cc:385] T 0413c4b049a8412b9b3b8a36bb5d67fd P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:25.311409 21518 raft_consensus.cc:740] T 0413c4b049a8412b9b3b8a36bb5d67fd P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Initialized, Role: FOLLOWER
I20260709 13:22:25.312873 21250 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa2f735fa7d14b1e876c094ffbfb2c9d" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:25.313491 21250 raft_consensus.cc:3060] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:25.330045 21250 raft_consensus.cc:2468] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:25.335079 21147 catalog_manager.cc:5697] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "74e2ff1a68ac4c849ee6397ebb011375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:25.337141 21363 leader_election.cc:304] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:25.340238 21518 consensus_queue.cc:260] T 0413c4b049a8412b9b3b8a36bb5d67fd P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:25.350772 21887 leader_election.cc:290] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:25.350759 21469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0413c4b049a8412b9b3b8a36bb5d67fd" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
W20260709 13:22:25.356240 21216 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:43181, user_credentials={real_user=slave}} blocked reactor thread for 148682us
I20260709 13:22:25.358891 21944 consensus_queue.cc:1048] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:25.343508 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa2f735fa7d14b1e876c094ffbfb2c9d" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:25.374607 21923 raft_consensus.cc:493] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:25.379161 21923 raft_consensus.cc:515] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:25.364212 21959 raft_consensus.cc:2804] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:25.381609 21923 leader_election.cc:290] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:25.382226 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0413c4b049a8412b9b3b8a36bb5d67fd" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:25.382468 21469 raft_consensus.cc:2468] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:25.382898 21246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0413c4b049a8412b9b3b8a36bb5d67fd" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:25.383596 21246 raft_consensus.cc:2468] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 0.
I20260709 13:22:25.382899 21399 raft_consensus.cc:2468] T 0413c4b049a8412b9b3b8a36bb5d67fd P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 0.
I20260709 13:22:25.384848 21437 leader_election.cc:304] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 
W20260709 13:22:25.384284 21364 leader_election.cc:343] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:25.385843 21923 raft_consensus.cc:2804] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:25.386217 21923 raft_consensus.cc:493] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:25.386740 21397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0413c4b049a8412b9b3b8a36bb5d67fd" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:25.388440 21397 raft_consensus.cc:2468] T 0413c4b049a8412b9b3b8a36bb5d67fd P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 0.
I20260709 13:22:25.389156 21923 raft_consensus.cc:3060] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:25.390305 21518 ts_tablet_manager.cc:1434] T 0413c4b049a8412b9b3b8a36bb5d67fd P 74e2ff1a68ac4c849ee6397ebb011375: Time spent starting tablet: real 0.084s	user 0.007s	sys 0.001s
I20260709 13:22:25.381213 21959 raft_consensus.cc:697] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Leader. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:25.397656 21959 consensus_queue.cc:237] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:25.404517 21214 leader_election.cc:304] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:25.405645 21944 raft_consensus.cc:2804] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:25.406152 21944 raft_consensus.cc:493] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:25.406616 21944 raft_consensus.cc:3060] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:25.419653 21520 tablet_bootstrap.cc:492] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:25.420200 21520 ts_tablet_manager.cc:1403] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.243s	user 0.021s	sys 0.010s
I20260709 13:22:25.425830 21520 raft_consensus.cc:359] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:25.426496 21520 raft_consensus.cc:385] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:25.426769 21520 raft_consensus.cc:740] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:25.427416 21520 consensus_queue.cc:260] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:25.427945 21944 raft_consensus.cc:515] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:25.430780 21397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0413c4b049a8412b9b3b8a36bb5d67fd" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:25.431813 21397 raft_consensus.cc:3060] T 0413c4b049a8412b9b3b8a36bb5d67fd P 74e2ff1a68ac4c849ee6397ebb011375 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:25.439548 21397 raft_consensus.cc:2468] T 0413c4b049a8412b9b3b8a36bb5d67fd P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 1.
I20260709 13:22:25.441836 21520 ts_tablet_manager.cc:1434] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:22:25.442020 21216 leader_election.cc:304] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:25.443032 21988 raft_consensus.cc:2804] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:25.443732 21988 raft_consensus.cc:697] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:25.445184 22007 raft_consensus.cc:493] T 3f3bf699b02c46c78d330dc7f9d3314b P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:25.445030 21988 consensus_queue.cc:237] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:25.445693 22007 raft_consensus.cc:515] T 3f3bf699b02c46c78d330dc7f9d3314b P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:25.446808 21470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0413c4b049a8412b9b3b8a36bb5d67fd" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:25.449719 21923 raft_consensus.cc:515] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:25.451120 21470 raft_consensus.cc:2393] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ec879c0528c44465916efc6c33dde636 in current term 1: Already voted for candidate 43cd181efe6b4b2fb79fd3e997a401ea in this term.
I20260709 13:22:25.443212 21520 tablet_bootstrap.cc:492] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c: Bootstrap starting.
I20260709 13:22:25.451476 21886 consensus_queue.cc:1048] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:25.454187 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3bf699b02c46c78d330dc7f9d3314b" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:25.440762 21944 leader_election.cc:290] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 1 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:25.454658 21395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0413c4b049a8412b9b3b8a36bb5d67fd" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:25.455518 21395 raft_consensus.cc:2393] T 0413c4b049a8412b9b3b8a36bb5d67fd P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 43cd181efe6b4b2fb79fd3e997a401ea in current term 1: Already voted for candidate ec879c0528c44465916efc6c33dde636 in this term.
I20260709 13:22:25.457381 21246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0413c4b049a8412b9b3b8a36bb5d67fd" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:25.458943 21437 leader_election.cc:304] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636
I20260709 13:22:25.459789 21999 raft_consensus.cc:2749] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:22:25.460162 21469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3bf699b02c46c78d330dc7f9d3314b" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:25.460827 21923 leader_election.cc:290] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 1 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:25.461937 21289 leader_election.cc:304] T 3f3bf699b02c46c78d330dc7f9d3314b P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:25.462657 21520 tablet_bootstrap.cc:654] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:25.447974 22007 leader_election.cc:290] T 3f3bf699b02c46c78d330dc7f9d3314b P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:25.463045 22002 raft_consensus.cc:2749] T 3f3bf699b02c46c78d330dc7f9d3314b P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:25.490787 21326 raft_consensus.cc:3060] T 3f3bf699b02c46c78d330dc7f9d3314b P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:25.495133 22002 raft_consensus.cc:493] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:25.495704 22002 raft_consensus.cc:515] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:25.497871 22002 leader_election.cc:290] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:25.504261 21246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b6856cbfc84013bdfa6979b0366468" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:25.513497 21823 consensus_queue.cc:1048] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:25.514575 21520 tablet_bootstrap.cc:492] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c: No bootstrap required, opened a new log
I20260709 13:22:25.515072 21520 ts_tablet_manager.cc:1403] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent bootstrapping tablet: real 0.072s	user 0.017s	sys 0.022s
I20260709 13:22:25.517675 21520 raft_consensus.cc:359] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:25.518356 21520 raft_consensus.cc:385] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:25.518641 21520 raft_consensus.cc:740] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Initialized, Role: FOLLOWER
I20260709 13:22:25.519265 21520 consensus_queue.cc:260] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:25.521272 21520 ts_tablet_manager.cc:1434] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:22:25.513671 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b6856cbfc84013bdfa6979b0366468" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:25.525403 21474 raft_consensus.cc:2393] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5109d6d6d9404ba39c53508a81fe9f1c in current term 1: Already voted for candidate ec879c0528c44465916efc6c33dde636 in this term.
I20260709 13:22:25.526818 21289 leader_election.cc:304] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636
I20260709 13:22:25.527937 22007 raft_consensus.cc:3060] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:25.531847 21149 catalog_manager.cc:5697] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "74e2ff1a68ac4c849ee6397ebb011375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:25.535516 22007 raft_consensus.cc:2749] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:25.547372 21145 catalog_manager.cc:5697] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:25.549484 21145 catalog_manager.cc:5697] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:25.550908 21145 catalog_manager.cc:5697] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:25.764479 22018 consensus_queue.cc:1048] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:25.780916 22018 consensus_queue.cc:1048] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:25.849252 22018 consensus_queue.cc:1048] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:25.853411 21988 consensus_queue.cc:1048] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:25.863302 21323 raft_consensus.cc:3060] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:25.876008 22007 raft_consensus.cc:493] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:25.876571 22007 raft_consensus.cc:515] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:25.879961 22006 consensus_queue.cc:1048] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:25.883656 22007 leader_election.cc:290] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:25.884552 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa2f735fa7d14b1e876c094ffbfb2c9d" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:25.885020 21247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa2f735fa7d14b1e876c094ffbfb2c9d" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:25.885730 21247 raft_consensus.cc:2468] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 1.
I20260709 13:22:25.887166 21289 leader_election.cc:304] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:25.897166 22002 raft_consensus.cc:2804] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:25.899036 22002 raft_consensus.cc:493] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 74e2ff1a68ac4c849ee6397ebb011375)
I20260709 13:22:25.899456 22002 raft_consensus.cc:3060] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:25.910223 22002 raft_consensus.cc:515] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:25.911116 21823 consensus_queue.cc:1048] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:25.911401 21323 raft_consensus.cc:1240] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Rejecting Update request from peer 74e2ff1a68ac4c849ee6397ebb011375 for earlier term 1. Current term is 2. Ops: []
I20260709 13:22:25.915874 21823 consensus_queue.cc:1059] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:25.917011 21959 raft_consensus.cc:3055] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:25.917372 21959 raft_consensus.cc:740] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:25.918211 21959 consensus_queue.cc:260] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:25.919167 21959 raft_consensus.cc:3060] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:25.929059 21246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa2f735fa7d14b1e876c094ffbfb2c9d" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:25.933061 22002 leader_election.cc:290] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:25.935835 21395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa2f735fa7d14b1e876c094ffbfb2c9d" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:25.949424 21395 raft_consensus.cc:2468] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 2.
I20260709 13:22:25.954856 21291 leader_election.cc:304] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375; no voters: ec879c0528c44465916efc6c33dde636
I20260709 13:22:25.955542 22002 raft_consensus.cc:2804] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:25.956064 22002 raft_consensus.cc:697] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 LEADER]: Becoming Leader. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Running, Role: LEADER
I20260709 13:22:25.956843 22002 consensus_queue.cc:237] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:25.964799 21148 catalog_manager.cc:5697] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c reported cstate change: term changed from 1 to 2, leader changed from 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67) to 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66). New cstate: current_term: 2 leader_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:25.989647 22018 consensus_queue.cc:1048] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:26.001108 22018 consensus_queue.cc:1048] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:26.209451 11517 tablet_server.cc:179] TabletServer@127.11.63.65:0 shutting down...
W20260709 13:22:26.228139 21437 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:22:26.232141 21437 consensus_peers.cc:597] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:26.257084 21437 consensus_peers.cc:597] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:26.271944 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:26.272825 11517 tablet_replica.cc:333] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.273890 11517 raft_consensus.cc:2243] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:26.275032 11517 raft_consensus.cc:2272] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.277822 11517 tablet_replica.cc:333] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.278461 11517 raft_consensus.cc:2243] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:26.278898 11517 raft_consensus.cc:2272] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.282029 11517 tablet_replica.cc:333] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.282724 11517 raft_consensus.cc:2243] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:26.283440 11517 raft_consensus.cc:2272] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.285658 11517 tablet_replica.cc:333] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.286268 11517 raft_consensus.cc:2243] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:26.287070 11517 raft_consensus.cc:2272] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.288762 11517 tablet_replica.cc:333] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.289331 11517 raft_consensus.cc:2243] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:26.289777 11517 raft_consensus.cc:2272] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.291586 11517 tablet_replica.cc:333] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.292189 11517 raft_consensus.cc:2243] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:26.292650 11517 raft_consensus.cc:2272] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.294551 11517 tablet_replica.cc:333] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.295250 11517 raft_consensus.cc:2243] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:26.295698 11517 raft_consensus.cc:2272] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:26.298637 21289 consensus_peers.cc:597] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:26.298180 11517 tablet_replica.cc:333] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.300132 11517 raft_consensus.cc:2243] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:26.301988 11517 raft_consensus.cc:2272] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.304515 11517 tablet_replica.cc:333] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.305371 11517 raft_consensus.cc:2243] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:22:26.305966 21437 consensus_peers.cc:597] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:26.306370 11517 raft_consensus.cc:2272] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.309038 11517 tablet_replica.cc:333] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.309692 11517 raft_consensus.cc:2243] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:26.310417 11517 raft_consensus.cc:2272] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.312189 11517 tablet_replica.cc:333] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.312726 11517 raft_consensus.cc:2243] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:26.313407 11517 raft_consensus.cc:2272] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.315346 11517 tablet_replica.cc:333] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.315981 11517 raft_consensus.cc:2243] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:26.316711 11517 raft_consensus.cc:2272] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.318869 11517 tablet_replica.cc:333] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.319504 11517 raft_consensus.cc:2243] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:26.320273 11517 raft_consensus.cc:2272] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.322683 11517 tablet_replica.cc:333] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.323392 11517 raft_consensus.cc:2243] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:26.324108 11517 raft_consensus.cc:2272] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.326416 11517 tablet_replica.cc:333] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.326946 11517 raft_consensus.cc:2243] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:26.327593 11517 raft_consensus.cc:2272] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.329769 11517 tablet_replica.cc:333] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.330322 11517 raft_consensus.cc:2243] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:26.330826 11517 raft_consensus.cc:2272] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.332928 11517 tablet_replica.cc:333] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.333526 11517 raft_consensus.cc:2243] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:26.334482 11517 raft_consensus.cc:2272] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.337074 11517 tablet_replica.cc:333] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.337932 11517 raft_consensus.cc:2243] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:26.338542 11517 raft_consensus.cc:2272] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.340672 11517 tablet_replica.cc:333] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.341394 11517 raft_consensus.cc:2243] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:26.342531 11517 raft_consensus.cc:2272] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.344586 11517 tablet_replica.cc:333] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.345168 11517 raft_consensus.cc:2243] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:26.346361 11517 raft_consensus.cc:2272] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.348722 11517 tablet_replica.cc:333] T 6b569ec3d17d412c8c9087446e64bb77 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.349275 11517 raft_consensus.cc:2243] T 6b569ec3d17d412c8c9087446e64bb77 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:26.349748 11517 raft_consensus.cc:2272] T 6b569ec3d17d412c8c9087446e64bb77 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.351579 11517 tablet_replica.cc:333] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.352139 11517 raft_consensus.cc:2243] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:26.352588 11517 raft_consensus.cc:2272] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.354727 11517 tablet_replica.cc:333] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.355271 11517 raft_consensus.cc:2243] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:26.355667 11517 raft_consensus.cc:2272] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.357754 11517 tablet_replica.cc:333] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.358291 11517 raft_consensus.cc:2243] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:26.358958 11517 raft_consensus.cc:2272] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.361303 11517 tablet_replica.cc:333] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
W20260709 13:22:26.362030 21363 consensus_peers.cc:597] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:26.362083 11517 raft_consensus.cc:2243] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:26.362895 11517 raft_consensus.cc:2272] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:26.365028 21437 consensus_peers.cc:597] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:26.365475 11517 tablet_replica.cc:333] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.366190 11517 raft_consensus.cc:2243] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:26.366653 11517 raft_consensus.cc:2272] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:26.367209 21363 consensus_peers.cc:597] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:26.369089 11517 tablet_replica.cc:333] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.370025 11517 raft_consensus.cc:2243] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:26.370723 11517 raft_consensus.cc:2272] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.373677 11517 tablet_replica.cc:333] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
W20260709 13:22:26.373936 21363 consensus_peers.cc:597] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:26.374451 11517 raft_consensus.cc:2243] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:26.375509 11517 raft_consensus.cc:2272] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.377076 21395 raft_consensus.cc:1275] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Refusing update from remote peer 5109d6d6d9404ba39c53508a81fe9f1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:26.378199 11517 tablet_replica.cc:333] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.378583 22007 consensus_queue.cc:1048] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:26.379051 11517 raft_consensus.cc:2243] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:26.379626 11517 raft_consensus.cc:2272] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.381965 11517 tablet_replica.cc:333] T 347c76112a0e430ca7d607278f20c811 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.382774 11517 raft_consensus.cc:2243] T 347c76112a0e430ca7d607278f20c811 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:26.383280 11517 raft_consensus.cc:2272] T 347c76112a0e430ca7d607278f20c811 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.385704 11517 tablet_replica.cc:333] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.386462 11517 raft_consensus.cc:2243] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:26.386974 11517 raft_consensus.cc:2272] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.389511 11517 tablet_replica.cc:333] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.390364 11517 raft_consensus.cc:2243] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:26.391651 11517 raft_consensus.cc:2272] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.394385 11517 tablet_replica.cc:333] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.395109 11517 raft_consensus.cc:2243] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:26.395642 11517 raft_consensus.cc:2272] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:26.396063 21289 consensus_peers.cc:597] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:26.397827 11517 tablet_replica.cc:333] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.398479 11517 raft_consensus.cc:2243] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:26.399662 11517 raft_consensus.cc:2272] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.402977 11517 tablet_replica.cc:333] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.403931 11517 raft_consensus.cc:2243] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:26.404518 11517 raft_consensus.cc:2272] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.407500 11517 tablet_replica.cc:333] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.408238 11517 raft_consensus.cc:2243] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:26.409227 11517 raft_consensus.cc:2272] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.412123 11517 tablet_replica.cc:333] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.412981 11517 raft_consensus.cc:2243] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:26.413597 11517 raft_consensus.cc:2272] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:26.415418 21363 consensus_peers.cc:597] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:26.415927 11517 tablet_replica.cc:333] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
W20260709 13:22:26.416174 21363 consensus_peers.cc:597] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:26.416846 11517 raft_consensus.cc:2243] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:26.417518 11517 raft_consensus.cc:2272] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:26.418946 21289 consensus_peers.cc:597] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:26.420414 11517 tablet_replica.cc:333] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.421069 11517 raft_consensus.cc:2243] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:26.421732 11517 raft_consensus.cc:2272] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.424901 11517 tablet_replica.cc:333] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.425694 11517 raft_consensus.cc:2243] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:26.426275 11517 raft_consensus.cc:2272] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:26.427549 21437 consensus_peers.cc:597] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:26.428741 11517 tablet_replica.cc:333] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.429553 11517 raft_consensus.cc:2243] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:26.430642 11517 raft_consensus.cc:2272] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.432559 11517 tablet_replica.cc:333] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.433076 11517 raft_consensus.cc:2243] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:26.433840 11517 raft_consensus.cc:2272] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.435614 11517 tablet_replica.cc:333] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.436107 11517 raft_consensus.cc:2243] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:26.436525 11517 raft_consensus.cc:2272] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.438266 11517 tablet_replica.cc:333] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.438778 11517 raft_consensus.cc:2243] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:26.439170 11517 raft_consensus.cc:2272] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:26.441021 11517 tablet_replica.cc:333] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636: stopping tablet replica
I20260709 13:22:26.441543 11517 raft_consensus.cc:2243] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:26.442507 11517 raft_consensus.cc:2272] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:26.474748 21363 consensus_peers.cc:597] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:26.508978 11517 tablet_server.cc:196] TabletServer@127.11.63.65:0 shutdown complete.
W20260709 13:22:26.511503 21363 consensus_peers.cc:597] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:26.511941 21363 consensus_peers.cc:597] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:26.539580 21363 consensus_peers.cc:597] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:26.542884 21437 consensus_peers.cc:597] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:26.550344 21289 consensus_peers.cc:597] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:26.579401 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:26.586738 21437 consensus_peers.cc:597] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:26.587175 21289 consensus_peers.cc:597] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:26.592628 22056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:26.593128 22055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:26.594594 22058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:26.597805 11517 server_base.cc:1061] running on GCE node
I20260709 13:22:26.601836 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:26.602123 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:26.602344 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603346602325 us; error 0 us; skew 500 ppm
I20260709 13:22:26.602878 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:26.606530 11517 webserver.cc:533] Webserver started at http://127.11.63.65:43837/ using document root <none> and password file <none>
I20260709 13:22:26.607131 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:26.607357 11517 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:22:26.611450 11517 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260709 13:22:26.620640 22063 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:26.621688 11517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:22:26.622160 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/ts-0-root
uuid: "ec879c0528c44465916efc6c33dde636"
format_stamp: "Formatted at 2026-07-09 13:22:13 on dist-test-slave-qv7r"
I20260709 13:22:26.622643 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783603019296230-11517-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:26.641325 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:26.642931 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:26.644858 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:26.650228 22070 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 13:22:26.729750 21437 consensus_peers.cc:597] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:26.734216 21437 consensus_peers.cc:597] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:26.813151 21363 consensus_peers.cc:597] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:26.924772 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:22:26.925273 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.276s	user 0.008s	sys 0.001s
I20260709 13:22:26.925659 11517 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:22:26.934867 22070 tablet_bootstrap.cc:492] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:26.950265 22070 tablet_bootstrap.cc:492] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:26.951190 22070 tablet_bootstrap.cc:492] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:26.951838 22070 ts_tablet_manager.cc:1403] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 13:22:26.954376 22070 raft_consensus.cc:359] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:26.955070 22070 raft_consensus.cc:740] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:26.955775 22070 consensus_queue.cc:260] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:26.958285 22070 ts_tablet_manager.cc:1434] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:22:26.959148 22070 tablet_bootstrap.cc:492] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:26.975894 22070 tablet_bootstrap.cc:492] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:26.977201 22070 tablet_bootstrap.cc:492] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:26.977840 22070 ts_tablet_manager.cc:1403] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.008s
I20260709 13:22:26.980101 22070 raft_consensus.cc:359] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:26.980832 22070 raft_consensus.cc:740] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:26.981616 22070 consensus_queue.cc:260] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:26.984167 22070 ts_tablet_manager.cc:1434] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:22:26.985101 22070 tablet_bootstrap.cc:492] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:27.001499 22070 tablet_bootstrap.cc:492] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:27.002393 22070 tablet_bootstrap.cc:492] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:27.002955 22070 ts_tablet_manager.cc:1403] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.018s	user 0.016s	sys 0.000s
I20260709 13:22:27.005322 22070 raft_consensus.cc:359] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.006029 22070 raft_consensus.cc:740] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:27.006726 22070 consensus_queue.cc:260] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.009265 22070 ts_tablet_manager.cc:1434] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:22:27.010141 22070 tablet_bootstrap.cc:492] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:27.024524 22070 tablet_bootstrap.cc:492] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:27.025420 22070 tablet_bootstrap.cc:492] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:27.026002 22070 ts_tablet_manager.cc:1403] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.016s	user 0.015s	sys 0.000s
I20260709 13:22:27.028239 22070 raft_consensus.cc:359] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:27.028986 22070 raft_consensus.cc:740] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:27.029836 22070 consensus_queue.cc:260] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:27.031896 22070 ts_tablet_manager.cc:1434] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:22:27.032713 22070 tablet_bootstrap.cc:492] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:27.047629 22070 tablet_bootstrap.cc:492] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:27.048660 22070 tablet_bootstrap.cc:492] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:27.049271 22070 ts_tablet_manager.cc:1403] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 13:22:27.051571 22070 raft_consensus.cc:359] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:27.052287 22070 raft_consensus.cc:740] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:27.053155 22070 consensus_queue.cc:260] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:27.054976 22070 ts_tablet_manager.cc:1434] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:22:27.055790 22070 tablet_bootstrap.cc:492] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:27.090667 22070 tablet_bootstrap.cc:492] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:27.091850 22070 tablet_bootstrap.cc:492] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:27.092531 22070 ts_tablet_manager.cc:1403] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.020s
I20260709 13:22:27.095161 22070 raft_consensus.cc:359] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.095899 22070 raft_consensus.cc:740] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:27.096614 22070 consensus_queue.cc:260] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.098678 22070 ts_tablet_manager.cc:1434] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:22:27.099505 22070 tablet_bootstrap.cc:492] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:27.115803 22070 tablet_bootstrap.cc:492] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:27.116801 22070 tablet_bootstrap.cc:492] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:27.117480 22070 ts_tablet_manager.cc:1403] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.018s	user 0.017s	sys 0.000s
I20260709 13:22:27.120311 22070 raft_consensus.cc:359] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.121052 22070 raft_consensus.cc:740] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:27.121842 22070 consensus_queue.cc:260] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.123948 22070 ts_tablet_manager.cc:1434] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:22:27.124789 22070 tablet_bootstrap.cc:492] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:27.142149 22070 tablet_bootstrap.cc:492] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:27.143247 22070 tablet_bootstrap.cc:492] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:27.143960 22070 ts_tablet_manager.cc:1403] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.003s
I20260709 13:22:27.146634 22070 raft_consensus.cc:359] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.147562 22070 raft_consensus.cc:740] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:27.148471 22070 consensus_queue.cc:260] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.150889 22070 ts_tablet_manager.cc:1434] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:22:27.151816 22070 tablet_bootstrap.cc:492] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:27.170337 11517 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:22:27.170754 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.245s	user 0.226s	sys 0.012s
I20260709 13:22:27.176304 22070 tablet_bootstrap.cc:492] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:27.177358 22070 tablet_bootstrap.cc:492] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:27.178020 22070 ts_tablet_manager.cc:1403] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.026s	user 0.019s	sys 0.004s
I20260709 13:22:27.180594 22070 raft_consensus.cc:359] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.181272 22070 raft_consensus.cc:740] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:27.182039 22070 consensus_queue.cc:260] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.184948 22070 ts_tablet_manager.cc:1434] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:22:27.186087 22070 tablet_bootstrap.cc:492] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:27.206704 22070 tablet_bootstrap.cc:492] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:27.207716 22070 tablet_bootstrap.cc:492] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:27.208385 22070 ts_tablet_manager.cc:1403] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.004s
I20260709 13:22:27.210809 22070 raft_consensus.cc:359] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:27.211365 22070 raft_consensus.cc:740] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:27.212015 22070 consensus_queue.cc:260] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:27.215265 22070 ts_tablet_manager.cc:1434] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:22:27.216081 22070 tablet_bootstrap.cc:492] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:27.233246 22070 tablet_bootstrap.cc:492] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:27.234078 22070 tablet_bootstrap.cc:492] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:27.234606 22070 ts_tablet_manager.cc:1403] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.006s
I20260709 13:22:27.236953 22070 raft_consensus.cc:359] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.237571 22070 raft_consensus.cc:740] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:27.238453 22070 consensus_queue.cc:260] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.240576 22070 ts_tablet_manager.cc:1434] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.001s
I20260709 13:22:27.241487 22070 tablet_bootstrap.cc:492] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:27.274176 22070 tablet_bootstrap.cc:492] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:27.275060 22070 tablet_bootstrap.cc:492] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:27.275750 22070 ts_tablet_manager.cc:1403] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.034s	user 0.019s	sys 0.012s
I20260709 13:22:27.290622 22070 raft_consensus.cc:359] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.291987 22070 raft_consensus.cc:740] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:27.292698 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.65:35963
I20260709 13:22:27.292829 22143 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.65:35963 every 8 connection(s)
I20260709 13:22:27.294013 22070 consensus_queue.cc:260] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.296413 22070 ts_tablet_manager.cc:1434] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.007s
I20260709 13:22:27.297472 22070 tablet_bootstrap.cc:492] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:27.336977 22007 raft_consensus.cc:493] T e4905b7b4db2444484e9403fed337486 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.337771 22007 raft_consensus.cc:515] T e4905b7b4db2444484e9403fed337486 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.341635 22039 raft_consensus.cc:493] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.342480 22051 raft_consensus.cc:493] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.343111 22051 raft_consensus.cc:515] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:27.342339 22039 raft_consensus.cc:515] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.349223 21470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4905b7b4db2444484e9403fed337486" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:27.351435 22051 leader_election.cc:290] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:27.352517 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c6754d9a8e41ceb1fb416dcbbf0234" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:27.365573 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc40f11dce5540d1a056e9bacba075e2" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:27.372448 22038 raft_consensus.cc:493] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.373225 22038 raft_consensus.cc:515] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.376545 22038 leader_election.cc:290] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.380187 22039 leader_election.cc:290] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.381711 22007 leader_election.cc:290] T e4905b7b4db2444484e9403fed337486 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.384709 22074 raft_consensus.cc:493] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.385439 22074 raft_consensus.cc:515] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.387895 22074 leader_election.cc:290] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.390003 22051 raft_consensus.cc:493] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.390609 22051 raft_consensus.cc:515] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.392663 21469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d5ddbaefa064931963bc1e27ac99903" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:27.393343 21469 raft_consensus.cc:2468] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:27.394285 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ddeabcec9b64fc68b40c70df90b80ef" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:27.395212 22051 leader_election.cc:290] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.395793 22051 raft_consensus.cc:493] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.396288 22051 raft_consensus.cc:515] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:27.400210 21363 leader_election.cc:304] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:27.401222 21886 raft_consensus.cc:2804] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:27.401648 21886 raft_consensus.cc:493] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.402447 21886 raft_consensus.cc:3060] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.403869 22051 leader_election.cc:290] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:27.407361 21823 raft_consensus.cc:493] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.405886 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228f7ae94b2848ecb12f93717756ba8b" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:27.407938 21823 raft_consensus.cc:515] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:27.421746 21823 leader_election.cc:290] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:27.424789 22073 raft_consensus.cc:493] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.425550 22073 raft_consensus.cc:515] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.429351 22073 leader_election.cc:290] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:27.431299 22051 raft_consensus.cc:493] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.431969 22051 raft_consensus.cc:515] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.432337 22071 raft_consensus.cc:493] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.433099 22074 raft_consensus.cc:493] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.432927 22071 raft_consensus.cc:515] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.433629 22074 raft_consensus.cc:515] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.435225 22071 leader_election.cc:290] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.435828 22074 leader_election.cc:290] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.437018 21323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4eebc08c8f342a69d7a1248e7df69a1" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:27.438627 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450b7111fe014a51b6e801869913ec52" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:27.437299 22051 leader_election.cc:290] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.441617 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d5ddbaefa064931963bc1e27ac99903" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:27.451654 22073 raft_consensus.cc:493] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.452647 22073 raft_consensus.cc:515] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.462539 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01967f3f14be4ecaa1cc2469a50fc156" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:27.466259 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fe3abd2b2647d384d0e147d1932c0a" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:27.466887 21474 raft_consensus.cc:2468] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:27.470607 22117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d5ddbaefa064931963bc1e27ac99903" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.474169 22039 raft_consensus.cc:493] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.475020 22039 raft_consensus.cc:515] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.476918 21474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37dadd42cd04be5a0e0b339df630a04" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:27.481325 22038 raft_consensus.cc:493] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.491600 22072 raft_consensus.cc:493] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.515908 21474 raft_consensus.cc:2468] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:27.515919 22038 raft_consensus.cc:515] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.518769 22038 leader_election.cc:290] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:27.522334 22115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fe3abd2b2647d384d0e147d1932c0a" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.520294 22117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01967f3f14be4ecaa1cc2469a50fc156" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.523085 22115 raft_consensus.cc:2468] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:27.523811 22038 raft_consensus.cc:493] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.524341 22038 raft_consensus.cc:515] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.525072 22115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37dadd42cd04be5a0e0b339df630a04" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.525715 22115 raft_consensus.cc:2468] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:27.516153 22072 raft_consensus.cc:515] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.527786 22072 leader_election.cc:290] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.529788 22038 leader_election.cc:290] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.532651 21395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37dadd42cd04be5a0e0b339df630a04" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:27.533612 21395 raft_consensus.cc:2468] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:27.535697 22038 raft_consensus.cc:493] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.536594 22038 raft_consensus.cc:515] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.539819 22038 leader_election.cc:290] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:27.540731 22072 raft_consensus.cc:493] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.541437 22072 raft_consensus.cc:515] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.542513 22073 leader_election.cc:290] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.543744 22072 leader_election.cc:290] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:27.543942 21363 leader_election.cc:304] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:27.546586 22071 raft_consensus.cc:2804] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:27.547086 22071 raft_consensus.cc:493] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.547537 22071 raft_consensus.cc:3060] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.554441 21363 leader_election.cc:304] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:27.557066 21395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fe3abd2b2647d384d0e147d1932c0a" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:27.558624 22153 raft_consensus.cc:2804] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:27.559268 22153 raft_consensus.cc:493] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.560123 22153 raft_consensus.cc:3060] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.567584 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0413c4b049a8412b9b3b8a36bb5d67fd" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:27.568358 21396 raft_consensus.cc:2468] T 0413c4b049a8412b9b3b8a36bb5d67fd P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:27.569341 21439 leader_election.cc:304] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:27.568356 21398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e1cba0e20343aeb61c31c917dd1f50" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:27.571656 21398 raft_consensus.cc:2468] T f4e1cba0e20343aeb61c31c917dd1f50 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:27.572578 22072 raft_consensus.cc:2804] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:27.573010 22072 raft_consensus.cc:493] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.573413 22072 raft_consensus.cc:3060] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.576783 22039 leader_election.cc:290] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.582677 22071 raft_consensus.cc:515] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.582896 22072 raft_consensus.cc:515] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.584152 21395 raft_consensus.cc:2393] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 43cd181efe6b4b2fb79fd3e997a401ea in current term 2: Already voted for candidate 74e2ff1a68ac4c849ee6397ebb011375 in this term.
I20260709 13:22:27.587747 22153 raft_consensus.cc:515] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.590723 22153 leader_election.cc:290] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.605875 22115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc40f11dce5540d1a056e9bacba075e2" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.606679 22116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c6754d9a8e41ceb1fb416dcbbf0234" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.607640 22116 raft_consensus.cc:2468] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 1.
I20260709 13:22:27.608731 22111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450b7111fe014a51b6e801869913ec52" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.609011 22113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4905b7b4db2444484e9403fed337486" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.609977 22113 raft_consensus.cc:2468] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 1.
I20260709 13:22:27.610195 22114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228f7ae94b2848ecb12f93717756ba8b" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.610946 22114 raft_consensus.cc:2468] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 1.
I20260709 13:22:27.611099 22073 raft_consensus.cc:493] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.611274 22112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ddeabcec9b64fc68b40c70df90b80ef" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.611704 22073 raft_consensus.cc:515] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.606696 22115 raft_consensus.cc:2468] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 3.
I20260709 13:22:27.614583 22007 raft_consensus.cc:493] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.615620 22007 raft_consensus.cc:515] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.616637 22117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d5ddbaefa064931963bc1e27ac99903" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.617779 22115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4eebc08c8f342a69d7a1248e7df69a1" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.619499 22115 raft_consensus.cc:2468] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:27.622402 22007 leader_election.cc:290] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.618696 22072 leader_election.cc:290] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.625406 22112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280cfe3edb3f490c8de46d0c80f6da5a" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.628279 22117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e1cba0e20343aeb61c31c917dd1f50" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.632138 22111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fe3abd2b2647d384d0e147d1932c0a" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.632918 22111 raft_consensus.cc:2468] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:27.634068 22110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37dadd42cd04be5a0e0b339df630a04" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:27.634922 21437 leader_election.cc:304] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:27.629375 22112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0413c4b049a8412b9b3b8a36bb5d67fd" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.630277 22115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4905b7b4db2444484e9403fed337486" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.630661 22113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fff6b36d1984d38b6f791e0ac15f56b" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.636945 21437 leader_election.cc:304] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:27.637291 22113 raft_consensus.cc:2468] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:27.637751 22038 raft_consensus.cc:2804] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:22:27.631104 22114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37dadd42cd04be5a0e0b339df630a04" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.638231 22038 raft_consensus.cc:493] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.638526 22072 raft_consensus.cc:2804] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:27.634800 22110 raft_consensus.cc:3060] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:22:27.641034 21437 leader_election.cc:343] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:27.636970 22115 raft_consensus.cc:2468] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:27.639298 22038 raft_consensus.cc:3060] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:27.650017 22038 raft_consensus.cc:515] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.651036 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280cfe3edb3f490c8de46d0c80f6da5a" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:27.651916 21323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4905b7b4db2444484e9403fed337486" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:27.651942 21326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fff6b36d1984d38b6f791e0ac15f56b" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:27.639308 22072 raft_consensus.cc:493] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.653602 22007 raft_consensus.cc:493] T 1fff6b36d1984d38b6f791e0ac15f56b P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
W20260709 13:22:27.654390 21363 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36863, user_credentials={real_user=slave}} blocked reactor thread for 100026us
I20260709 13:22:27.658457 21323 raft_consensus.cc:2468] T e4905b7b4db2444484e9403fed337486 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
W20260709 13:22:27.670459 21363 leader_election.cc:343] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:27.660686 22002 raft_consensus.cc:493] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.676451 21289 leader_election.cc:304] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:27.661625 21470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b503b6f79842efbb8a7ecd5c5851e0" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:27.681022 21470 raft_consensus.cc:2468] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 1.
I20260709 13:22:27.685091 21438 leader_election.cc:304] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 
W20260709 13:22:27.672545 21363 leader_election.cc:343] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:27.691810 21289 leader_election.cc:343] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:27.695180 21289 leader_election.cc:304] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636
I20260709 13:22:27.659875 22072 raft_consensus.cc:3060] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.593477 21886 raft_consensus.cc:515] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.698297 21399 raft_consensus.cc:2393] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 43cd181efe6b4b2fb79fd3e997a401ea in current term 2: Already voted for candidate 74e2ff1a68ac4c849ee6397ebb011375 in this term.
I20260709 13:22:27.665542 22110 raft_consensus.cc:2468] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 2.
I20260709 13:22:27.707710 21364 leader_election.cc:304] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:27.694487 22051 raft_consensus.cc:2804] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:27.699523 22151 raft_consensus.cc:2804] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:27.713771 22116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b503b6f79842efbb8a7ecd5c5851e0" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.713932 22153 raft_consensus.cc:493] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.714591 22116 raft_consensus.cc:2468] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 1.
I20260709 13:22:27.714674 22153 raft_consensus.cc:515] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:27.685256 22007 raft_consensus.cc:515] T 1fff6b36d1984d38b6f791e0ac15f56b P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.717916 22038 leader_election.cc:290] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 4 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.718030 22007 leader_election.cc:290] T 1fff6b36d1984d38b6f791e0ac15f56b P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.718870 22007 raft_consensus.cc:2749] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:22:27.719251 22071 leader_election.cc:290] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.689970 22002 raft_consensus.cc:515] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.719846 21326 raft_consensus.cc:2468] T 1fff6b36d1984d38b6f791e0ac15f56b P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:27.706293 22073 leader_election.cc:290] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:27.711548 22051 raft_consensus.cc:493] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.727428 21470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37dadd42cd04be5a0e0b339df630a04" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:27.729274 21470 raft_consensus.cc:2393] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 74e2ff1a68ac4c849ee6397ebb011375 in current term 2: Already voted for candidate 43cd181efe6b4b2fb79fd3e997a401ea in this term.
I20260709 13:22:27.720239 22151 raft_consensus.cc:493] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
W20260709 13:22:27.732699 21437 leader_election.cc:343] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:27.733259 21437 leader_election.cc:304] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:27.735342 21437 leader_election.cc:304] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:27.727077 22051 raft_consensus.cc:3060] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.737191 21823 raft_consensus.cc:493] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.737427 21470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fe3abd2b2647d384d0e147d1932c0a" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:27.738469 21470 raft_consensus.cc:3060] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.738369 21823 raft_consensus.cc:515] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.729692 22156 raft_consensus.cc:493] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
W20260709 13:22:27.742400 21437 leader_election.cc:343] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:27.733873 22151 raft_consensus.cc:3060] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.744802 21437 leader_election.cc:304] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:27.735105 22153 leader_election.cc:290] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:27.749150 21823 leader_election.cc:290] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.749862 22074 raft_consensus.cc:493] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.752020 22157 raft_consensus.cc:2749] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:22:27.751865 22074 raft_consensus.cc:515] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:27.750138 21886 leader_election.cc:290] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.767119 22074 leader_election.cc:290] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:27.770293 22114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73fe3abd2b2647d384d0e147d1932c0a" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:27.771358 22114 raft_consensus.cc:3060] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.772598 22072 raft_consensus.cc:515] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.775061 22073 raft_consensus.cc:2804] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:27.775604 22073 raft_consensus.cc:493] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.775796 22072 leader_election.cc:290] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:27.776047 22073 raft_consensus.cc:3060] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.776783 22072 raft_consensus.cc:493] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.777426 22072 raft_consensus.cc:515] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.747366 22156 raft_consensus.cc:515] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:27.781059 22072 leader_election.cc:290] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:27.782308 22072 raft_consensus.cc:493] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.748971 22110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228f7ae94b2848ecb12f93717756ba8b" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.827713 21470 raft_consensus.cc:2468] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 2.
I20260709 13:22:27.805985 22070 tablet_bootstrap.cc:492] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:27.828711 22038 raft_consensus.cc:2764] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:22:27.833174 21326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b503b6f79842efbb8a7ecd5c5851e0" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:27.833854 21326 raft_consensus.cc:2468] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:27.833907 21320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4eebc08c8f342a69d7a1248e7df69a1" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:27.834527 21320 raft_consensus.cc:3060] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.837146 22070 tablet_bootstrap.cc:492] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:27.783638 22156 leader_election.cc:290] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:27.813763 22002 leader_election.cc:290] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.812180 22115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c6754d9a8e41ceb1fb416dcbbf0234" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.827088 22007 raft_consensus.cc:493] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.792286 22113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b503b6f79842efbb8a7ecd5c5851e0" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.795284 22111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc40f11dce5540d1a056e9bacba075e2" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:27.785790 22051 raft_consensus.cc:515] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:27.848781 21326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b6856cbfc84013bdfa6979b0366468" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:27.783792 22072 raft_consensus.cc:515] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.861518 21326 raft_consensus.cc:2468] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:27.849185 22158 raft_consensus.cc:2749] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:22:27.807125 22112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf088eed547f42aea083b31277ecece1" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.864485 22111 raft_consensus.cc:3060] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:27.853892 22070 ts_tablet_manager.cc:1403] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.557s	user 0.146s	sys 0.198s
I20260709 13:22:27.801777 22108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.803195 22109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d5ddbaefa064931963bc1e27ac99903" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:27.786093 22116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37dadd42cd04be5a0e0b339df630a04" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:27.857651 22115 raft_consensus.cc:2468] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:27.865434 22113 raft_consensus.cc:2468] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:27.871272 22070 raft_consensus.cc:359] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:27.871994 22070 raft_consensus.cc:740] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:27.872507 22114 raft_consensus.cc:2468] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 2.
I20260709 13:22:27.872694 22070 consensus_queue.cc:260] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:27.836747 22039 raft_consensus.cc:493] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.790943 22073 raft_consensus.cc:515] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.836299 22110 raft_consensus.cc:2468] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
W20260709 13:22:27.865614 21364 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:38825, user_credentials={real_user=slave}} blocked reactor thread for 158020us
I20260709 13:22:27.867496 22116 raft_consensus.cc:2393] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 74e2ff1a68ac4c849ee6397ebb011375 in current term 2: Already voted for candidate 43cd181efe6b4b2fb79fd3e997a401ea in this term.
W20260709 13:22:27.867002 21437 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:35963, user_credentials={real_user=slave}} blocked reactor thread for 122287us
I20260709 13:22:27.868002 22051 leader_election.cc:290] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:27.881060 22155 raft_consensus.cc:493] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.868597 21320 raft_consensus.cc:2468] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 2.
I20260709 13:22:27.874826 21438 leader_election.cc:304] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 
I20260709 13:22:27.881628 22155 raft_consensus.cc:515] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:27.882722 21438 leader_election.cc:304] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 
I20260709 13:22:27.883435 22158 raft_consensus.cc:2804] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:27.883711 22155 leader_election.cc:290] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:27.883776 22158 raft_consensus.cc:493] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.876706 21395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c6754d9a8e41ceb1fb416dcbbf0234" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:27.884348 22158 raft_consensus.cc:3060] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.884603 21395 raft_consensus.cc:3060] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.885033 22160 raft_consensus.cc:2804] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:27.888207 22160 raft_consensus.cc:493] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.888602 22160 raft_consensus.cc:3060] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:22:27.886166 21363 leader_election.cc:343] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:27.890002 22110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e532ca31c67547998ba2fb09244573b3" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
W20260709 13:22:27.890430 21363 leader_election.cc:343] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:27.890508 22111 raft_consensus.cc:2468] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 4.
I20260709 13:22:27.890708 22110 raft_consensus.cc:2468] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
W20260709 13:22:27.891186 21363 leader_election.cc:343] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:27.891808 21363 leader_election.cc:304] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:27.886660 22072 leader_election.cc:290] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:27.892323 22158 raft_consensus.cc:515] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.889739 22144 heartbeater.cc:344] Connected to a master server at 127.11.63.126:41481
I20260709 13:22:27.862205 22007 raft_consensus.cc:515] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.893258 22144 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:27.893682 21395 raft_consensus.cc:2468] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 2.
I20260709 13:22:27.894337 22144 heartbeater.cc:507] Master 127.11.63.126:41481 requested a full tablet report, sending...
W20260709 13:22:27.895282 21437 leader_election.cc:343] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:27.895965 21437 leader_election.cc:304] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:27.879944 22039 raft_consensus.cc:515] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.878574 22163 raft_consensus.cc:493] T 5fec8ff4bc5a4d558263e38ae75ece75 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.897801 22163 raft_consensus.cc:515] T 5fec8ff4bc5a4d558263e38ae75ece75 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.898293 22039 leader_election.cc:290] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:27.898515 22072 raft_consensus.cc:2804] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:27.898746 22151 raft_consensus.cc:515] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.899164 22072 raft_consensus.cc:697] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:27.900430 22163 leader_election.cc:290] T 5fec8ff4bc5a4d558263e38ae75ece75 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.900358 22072 consensus_queue.cc:237] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.900969 22151 leader_election.cc:290] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:27.900548 22007 leader_election.cc:290] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.903756 22002 raft_consensus.cc:493] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:27.904201 22002 raft_consensus.cc:515] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.905470 21363 leader_election.cc:304] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:27.906018 22070 ts_tablet_manager.cc:1434] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.040s	user 0.004s	sys 0.002s
I20260709 13:22:27.906360 22154 raft_consensus.cc:2764] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:22:27.906639 22002 leader_election.cc:290] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.906926 22070 tablet_bootstrap.cc:492] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:27.907023 22154 raft_consensus.cc:2804] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:27.907596 22154 raft_consensus.cc:697] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 LEADER]: Becoming Leader. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:27.908618 22154 consensus_queue.cc:237] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.909329 22166 raft_consensus.cc:493] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.910473 21363 leader_election.cc:304] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:27.922432 21363 leader_election.cc:304] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:27.912385 21437 leader_election.cc:304] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 
W20260709 13:22:27.910570 21289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:35963, user_credentials={real_user=slave}} blocked reactor thread for 218784us
I20260709 13:22:27.924589 21470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fff6b36d1984d38b6f791e0ac15f56b" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:27.925308 21470 raft_consensus.cc:2393] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5109d6d6d9404ba39c53508a81fe9f1c in current term 2: Already voted for candidate 43cd181efe6b4b2fb79fd3e997a401ea in this term.
I20260709 13:22:27.925683 22155 raft_consensus.cc:2804] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:27.926107 22155 raft_consensus.cc:493] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.926239 22039 raft_consensus.cc:2804] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:22:27.926504 22155 raft_consensus.cc:3060] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.926849 22039 raft_consensus.cc:697] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 4 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:27.927814 22158 leader_election.cc:290] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:27.927745 22039 consensus_queue.cc:237] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.928791 22110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4eebc08c8f342a69d7a1248e7df69a1" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:27.929663 22110 raft_consensus.cc:3060] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.929905 22113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fff6b36d1984d38b6f791e0ac15f56b" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.930040 22114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b6856cbfc84013bdfa6979b0366468" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:27.930577 22113 raft_consensus.cc:2468] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 1.
I20260709 13:22:27.911713 21291 leader_election.cc:304] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:27.935153 21323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fec8ff4bc5a4d558263e38ae75ece75" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:27.936410 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4905b7b4db2444484e9403fed337486" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:27.937019 21324 raft_consensus.cc:3060] T e4905b7b4db2444484e9403fed337486 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.898296 22073 leader_election.cc:290] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:27.911463 21439 leader_election.cc:304] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: ec879c0528c44465916efc6c33dde636
I20260709 13:22:27.911463 21438 leader_election.cc:304] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 
I20260709 13:22:27.939733 21470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b6856cbfc84013bdfa6979b0366468" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:27.935878 21323 raft_consensus.cc:2468] T 5fec8ff4bc5a4d558263e38ae75ece75 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:27.940768 22168 raft_consensus.cc:2804] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:27.940346 22167 raft_consensus.cc:2804] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:27.943710 22167 raft_consensus.cc:493] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.944077 22167 raft_consensus.cc:3060] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.944454 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228f7ae94b2848ecb12f93717756ba8b" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:27.945075 21325 raft_consensus.cc:2468] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:27.941412 21323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:27.947906 21324 raft_consensus.cc:2468] T e4905b7b4db2444484e9403fed337486 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 2.
I20260709 13:22:27.949208 21439 leader_election.cc:304] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: ec879c0528c44465916efc6c33dde636
I20260709 13:22:27.949954 21472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf088eed547f42aea083b31277ecece1" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:27.950542 21472 raft_consensus.cc:2468] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:27.951103 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c6754d9a8e41ceb1fb416dcbbf0234" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:27.953390 21326 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fff6b36d1984d38b6f791e0ac15f56b" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:27.954053 21439 leader_election.cc:304] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636
I20260709 13:22:27.955227 21363 leader_election.cc:304] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636
I20260709 13:22:27.954946 21326 raft_consensus.cc:3060] T 1fff6b36d1984d38b6f791e0ac15f56b P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.968116 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37dadd42cd04be5a0e0b339df630a04" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:27.969233 21399 raft_consensus.cc:2393] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 43cd181efe6b4b2fb79fd3e997a401ea in current term 2: Already voted for candidate 74e2ff1a68ac4c849ee6397ebb011375 in this term.
I20260709 13:22:27.933606 22165 raft_consensus.cc:493] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.974628 21823 raft_consensus.cc:2749] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:22:27.978824 21437 leader_election.cc:343] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:27.978518 21289 leader_election.cc:304] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 
I20260709 13:22:27.951350 22151 raft_consensus.cc:493] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.974133 22038 raft_consensus.cc:3060] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.952800 22168 raft_consensus.cc:697] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 LEADER]: Becoming Leader. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Running, Role: LEADER
I20260709 13:22:27.986269 21398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc40f11dce5540d1a056e9bacba075e2" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:27.987226 21468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280cfe3edb3f490c8de46d0c80f6da5a" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:27.987507 22002 raft_consensus.cc:2804] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:27.954496 22051 raft_consensus.cc:493] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.987936 22002 raft_consensus.cc:493] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.987972 21468 raft_consensus.cc:2468] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 1.
I20260709 13:22:27.988344 22051 raft_consensus.cc:515] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:27.954507 22164 raft_consensus.cc:2804] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:27.978479 21469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d5ddbaefa064931963bc1e27ac99903" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:27.990506 22164 raft_consensus.cc:697] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:27.991573 22164 consensus_queue.cc:237] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:27.967199 21886 raft_consensus.cc:493] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.964273 22072 raft_consensus.cc:2804] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:27.994644 22072 raft_consensus.cc:493] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.994427 21886 raft_consensus.cc:515] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.995103 22151 raft_consensus.cc:515] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.996824 21886 leader_election.cc:290] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:27.995157 22072 raft_consensus.cc:3060] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:27.998996 21468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4eebc08c8f342a69d7a1248e7df69a1" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:27.979934 22007 raft_consensus.cc:493] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.999863 22007 raft_consensus.cc:515] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:28.001875 22007 leader_election.cc:290] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:27.990438 22168 consensus_queue.cc:237] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:27.988473 22002 raft_consensus.cc:3060] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:22:28.006527 22169 log.cc:927] Time spent T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:22:28.006212 22072 raft_consensus.cc:515] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:27.985683 21289 leader_election.cc:304] T e4905b7b4db2444484e9403fed337486 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 43cd181efe6b4b2fb79fd3e997a401ea
I20260709 13:22:28.010025 22007 raft_consensus.cc:2764] T e4905b7b4db2444484e9403fed337486 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:22:28.011217 21289 leader_election.cc:304] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:28.011888 21319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e532ca31c67547998ba2fb09244573b3" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:28.012643 21319 raft_consensus.cc:2468] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:28.013044 22114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b6856cbfc84013bdfa6979b0366468" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:28.013448 22007 raft_consensus.cc:2804] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:28.013975 22007 raft_consensus.cc:493] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:28.014380 22007 raft_consensus.cc:3060] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.015776 21438 leader_election.cc:304] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 
I20260709 13:22:28.016649 22158 raft_consensus.cc:2804] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:28.016934 22116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1fff6b36d1984d38b6f791e0ac15f56b" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:28.017300 22112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c6754d9a8e41ceb1fb416dcbbf0234" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:28.017946 22117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b503b6f79842efbb8a7ecd5c5851e0" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:28.018625 22117 raft_consensus.cc:3060] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.019043 22111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450b7111fe014a51b6e801869913ec52" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:28.020311 22111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4eebc08c8f342a69d7a1248e7df69a1" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:28.020453 22002 raft_consensus.cc:515] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
W20260709 13:22:28.022406 21289 leader_election.cc:343] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:28.022897 21289 leader_election.cc:304] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636
I20260709 13:22:28.017055 22158 raft_consensus.cc:493] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:27.990664 22051 leader_election.cc:290] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:27.982892 22165 raft_consensus.cc:515] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:28.024658 21148 catalog_manager.cc:5697] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 1 to 2, leader changed from ec879c0528c44465916efc6c33dde636 (127.11.63.65) to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 2 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:28.018019 22112 raft_consensus.cc:3060] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.026113 21289 leader_election.cc:304] T 1fff6b36d1984d38b6f791e0ac15f56b P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 43cd181efe6b4b2fb79fd3e997a401ea
I20260709 13:22:27.995103 21326 raft_consensus.cc:2468] T 1fff6b36d1984d38b6f791e0ac15f56b P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 2.
I20260709 13:22:28.028450 21438 leader_election.cc:304] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 
I20260709 13:22:27.982455 21363 leader_election.cc:304] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: ec879c0528c44465916efc6c33dde636
I20260709 13:22:27.955351 22073 raft_consensus.cc:2804] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:27.966454 21320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450b7111fe014a51b6e801869913ec52" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:28.058395 21396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01967f3f14be4ecaa1cc2469a50fc156" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:28.071574 21396 raft_consensus.cc:2468] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 1.
I20260709 13:22:28.065690 22173 raft_consensus.cc:2764] T 1fff6b36d1984d38b6f791e0ac15f56b P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
W20260709 13:22:28.067137 21363 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.68:36863, user_credentials={real_user=slave}} blocked reactor thread for 85337.3us
I20260709 13:22:28.078076 22002 leader_election.cc:290] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:28.014298 22114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fec8ff4bc5a4d558263e38ae75ece75" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:28.068464 21438 leader_election.cc:304] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 
I20260709 13:22:28.064652 21289 leader_election.cc:304] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 
I20260709 13:22:28.075806 21397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e532ca31c67547998ba2fb09244573b3" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:28.076241 22161 raft_consensus.cc:2804] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:28.031764 22163 raft_consensus.cc:2749] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:22:28.032094 22116 raft_consensus.cc:3060] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.011910 22113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280cfe3edb3f490c8de46d0c80f6da5a" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
W20260709 13:22:28.089653 21289 leader_election.cc:343] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:28.129235 21289 leader_election.cc:304] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636
I20260709 13:22:28.091928 22007 raft_consensus.cc:515] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:28.129747 21400 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf088eed547f42aea083b31277ecece1" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:28.045096 21398 raft_consensus.cc:3060] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:28.045401 22158 raft_consensus.cc:3060] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.081204 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e1cba0e20343aeb61c31c917dd1f50" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:28.071231 22073 raft_consensus.cc:493] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:28.012969 22155 raft_consensus.cc:515] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:27.946153 22166 raft_consensus.cc:515] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
W20260709 13:22:28.135469 21438 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:38825, user_credentials={real_user=slave}} blocked reactor thread for 67075.7us
I20260709 13:22:28.119318 22002 raft_consensus.cc:2804] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:28.135775 22178 raft_consensus.cc:2804] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:27.959826 22160 raft_consensus.cc:515] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:28.095597 21320 raft_consensus.cc:2468] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:28.137228 21399 raft_consensus.cc:3060] T f4e1cba0e20343aeb61c31c917dd1f50 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.117928 22161 raft_consensus.cc:493] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:28.118877 22109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4905b7b4db2444484e9403fed337486" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:28.143985 22073 raft_consensus.cc:3060] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.134112 22072 leader_election.cc:290] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:28.159466 21323 raft_consensus.cc:2468] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 1.
I20260709 13:22:28.140223 22163 raft_consensus.cc:2749] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:22:28.140767 21473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fec8ff4bc5a4d558263e38ae75ece75" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:28.159106 21470 raft_consensus.cc:2393] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5109d6d6d9404ba39c53508a81fe9f1c in current term 2: Already voted for candidate 43cd181efe6b4b2fb79fd3e997a401ea in this term.
I20260709 13:22:28.139843 21317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b503b6f79842efbb8a7ecd5c5851e0" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:28.167912 21397 raft_consensus.cc:2393] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5109d6d6d9404ba39c53508a81fe9f1c in current term 2: Already voted for candidate 74e2ff1a68ac4c849ee6397ebb011375 in this term.
I20260709 13:22:28.173944 22113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e532ca31c67547998ba2fb09244573b3" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:28.174424 22160 leader_election.cc:290] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:28.174696 22113 raft_consensus.cc:2468] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 1.
I20260709 13:22:28.149834 22002 raft_consensus.cc:493] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:28.175810 22002 raft_consensus.cc:3060] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.178025 21317 raft_consensus.cc:2393] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 43cd181efe6b4b2fb79fd3e997a401ea in current term 2: Already voted for candidate 5109d6d6d9404ba39c53508a81fe9f1c in this term.
I20260709 13:22:28.155174 22161 raft_consensus.cc:3060] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.180740 21399 raft_consensus.cc:2468] T f4e1cba0e20343aeb61c31c917dd1f50 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 2.
I20260709 13:22:28.182978 22166 leader_election.cc:290] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:28.188791 21438 leader_election.cc:304] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 
I20260709 13:22:28.199944 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b6856cbfc84013bdfa6979b0366468" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:28.200588 21324 raft_consensus.cc:3060] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.202252 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:28.148609 22039 raft_consensus.cc:2804] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:28.202893 21399 raft_consensus.cc:2468] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 1.
I20260709 13:22:28.200685 21291 leader_election.cc:304] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:28.158862 22109 raft_consensus.cc:3060] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.156885 22178 raft_consensus.cc:697] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:28.207779 22152 raft_consensus.cc:2804] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:28.225800 21289 leader_election.cc:304] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636
I20260709 13:22:28.224637 21145 catalog_manager.cc:5697] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 reported cstate change: term changed from 1 to 2, leader changed from ec879c0528c44465916efc6c33dde636 (127.11.63.65) to 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67). New cstate: current_term: 2 leader_uuid: "74e2ff1a68ac4c849ee6397ebb011375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:28.212579 22051 raft_consensus.cc:2804] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:28.232686 21470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b503b6f79842efbb8a7ecd5c5851e0" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:28.233443 21470 raft_consensus.cc:2393] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5109d6d6d9404ba39c53508a81fe9f1c in current term 2: Already voted for candidate 43cd181efe6b4b2fb79fd3e997a401ea in this term.
I20260709 13:22:28.214017 22039 raft_consensus.cc:697] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:28.241420 21324 raft_consensus.cc:2468] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 2.
I20260709 13:22:28.220103 22165 leader_election.cc:290] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:28.223873 22152 raft_consensus.cc:493] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:28.248091 21470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc40f11dce5540d1a056e9bacba075e2" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:28.232359 22113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01967f3f14be4ecaa1cc2469a50fc156" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:28.251866 22002 raft_consensus.cc:515] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:28.231132 22051 raft_consensus.cc:493] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:28.253209 21398 raft_consensus.cc:2468] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 4.
I20260709 13:22:28.253657 22051 raft_consensus.cc:3060] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.256146 22116 raft_consensus.cc:2468] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 2.
I20260709 13:22:28.257879 22039 consensus_queue.cc:237] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:28.259410 22114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc40f11dce5540d1a056e9bacba075e2" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:28.260222 22114 raft_consensus.cc:2393] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 74e2ff1a68ac4c849ee6397ebb011375 in current term 4: Already voted for candidate 43cd181efe6b4b2fb79fd3e997a401ea in this term.
I20260709 13:22:28.260753 22152 raft_consensus.cc:3060] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.261647 22038 raft_consensus.cc:2749] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:22:28.262429 22115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fec8ff4bc5a4d558263e38ae75ece75" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:28.264415 22111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:28.266806 22002 leader_election.cc:290] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:28.267172 22108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b6856cbfc84013bdfa6979b0366468" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
W20260709 13:22:28.280035 21439 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:43181, user_credentials={real_user=slave}} blocked reactor thread for 187694us
I20260709 13:22:28.244834 22168 raft_consensus.cc:2749] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:22:28.283272 21469 raft_consensus.cc:2468] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 2.
I20260709 13:22:28.283740 22178 consensus_queue.cc:237] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:28.285692 22114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b503b6f79842efbb8a7ecd5c5851e0" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:28.286123 22109 raft_consensus.cc:2468] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 2.
W20260709 13:22:28.287811 21437 leader_election.cc:343] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:28.295609 21437 leader_election.cc:343] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:28.290279 21289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36863, user_credentials={real_user=slave}} blocked reactor thread for 51109.9us
W20260709 13:22:28.296288 21437 leader_election.cc:343] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:28.291888 22113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4e1cba0e20343aeb61c31c917dd1f50" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:28.299314 22110 raft_consensus.cc:2468] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 2.
I20260709 13:22:28.300697 22117 raft_consensus.cc:2468] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 2.
I20260709 13:22:28.301455 22155 leader_election.cc:290] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:28.302188 22114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280cfe3edb3f490c8de46d0c80f6da5a" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
W20260709 13:22:28.292579 21291 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:43181, user_credentials={real_user=slave}} blocked reactor thread for 83421.2us
I20260709 13:22:28.309150 21291 leader_election.cc:304] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
W20260709 13:22:28.297439 21289 leader_election.cc:343] T 5fec8ff4bc5a4d558263e38ae75ece75 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:28.310038 21149 ts_manager.cc:194] Re-registered known tserver with Master: ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:28.310811 21439 leader_election.cc:304] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:28.311556 22174 raft_consensus.cc:2804] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:28.312017 22174 raft_consensus.cc:697] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:28.312863 22174 consensus_queue.cc:237] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:28.316380 22109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228f7ae94b2848ecb12f93717756ba8b" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:28.319079 22109 raft_consensus.cc:3060] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.322691 21363 leader_election.cc:304] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: ec879c0528c44465916efc6c33dde636
I20260709 13:22:28.322961 21147 catalog_manager.cc:5697] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c reported cstate change: term changed from 1 to 2, leader changed from ec879c0528c44465916efc6c33dde636 (127.11.63.65) to 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66). New cstate: current_term: 2 leader_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:28.324263 22154 raft_consensus.cc:2804] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:28.324744 22154 raft_consensus.cc:697] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 LEADER]: Becoming Leader. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:28.325748 22154 consensus_queue.cc:237] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:28.327870 21437 leader_election.cc:304] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:28.306329 22112 raft_consensus.cc:2468] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 2.
I20260709 13:22:28.331295 22168 raft_consensus.cc:2804] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:28.331779 22168 raft_consensus.cc:493] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:28.331928 22151 leader_election.cc:290] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:28.332144 22168 raft_consensus.cc:3060] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.326231 22109 raft_consensus.cc:2468] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 2.
I20260709 13:22:28.341233 21438 leader_election.cc:304] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: ec879c0528c44465916efc6c33dde636
I20260709 13:22:28.345130 22151 raft_consensus.cc:2804] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:28.345711 22110 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e532ca31c67547998ba2fb09244573b3" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:28.346521 22110 raft_consensus.cc:3060] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.347177 22151 raft_consensus.cc:697] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:28.348097 22070 tablet_bootstrap.cc:492] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:28.348158 22151 consensus_queue.cc:237] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:28.349216 22070 tablet_bootstrap.cc:492] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:28.351491 21470 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "280cfe3edb3f490c8de46d0c80f6da5a" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:28.352159 21470 raft_consensus.cc:3060] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.352278 22070 ts_tablet_manager.cc:1403] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.446s	user 0.016s	sys 0.014s
W20260709 13:22:28.354203 21289 leader_election.cc:343] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:28.355729 21289 leader_election.cc:304] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:28.355957 21363 leader_election.cc:304] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636
W20260709 13:22:28.358130 21437 leader_election.cc:343] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:28.362478 21364 leader_election.cc:304] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375; no voters: ec879c0528c44465916efc6c33dde636
I20260709 13:22:28.363896 21323 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ddeabcec9b64fc68b40c70df90b80ef" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:28.364804 21323 raft_consensus.cc:2468] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 1.
I20260709 13:22:28.366020 21886 raft_consensus.cc:2804] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:28.366561 21886 raft_consensus.cc:493] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:28.366984 21886 raft_consensus.cc:3060] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:22:28.362345 21289 leader_election.cc:343] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:28.370062 21289 leader_election.cc:343] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:28.371788 21289 leader_election.cc:343] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:28.361047 22177 raft_consensus.cc:2749] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:22:28.374768 21398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228f7ae94b2848ecb12f93717756ba8b" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:28.343480 22161 raft_consensus.cc:515] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:28.341426 22175 raft_consensus.cc:2804] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:28.378743 21400 raft_consensus.cc:2393] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 43cd181efe6b4b2fb79fd3e997a401ea in current term 2: Already voted for candidate 74e2ff1a68ac4c849ee6397ebb011375 in this term.
I20260709 13:22:28.304840 22073 raft_consensus.cc:515] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:28.385157 21289 leader_election.cc:304] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:28.393265 21289 leader_election.cc:304] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636
I20260709 13:22:28.334820 22167 raft_consensus.cc:515] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:28.376323 22162 raft_consensus.cc:2804] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:28.402926 21470 raft_consensus.cc:2468] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 2.
I20260709 13:22:28.404735 21886 raft_consensus.cc:515] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:28.382057 22152 raft_consensus.cc:515] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:28.410401 22152 leader_election.cc:290] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:28.412356 22073 leader_election.cc:290] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:28.413751 22116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450b7111fe014a51b6e801869913ec52" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:28.399678 22184 raft_consensus.cc:2804] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:28.414748 22184 raft_consensus.cc:697] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 LEADER]: Becoming Leader. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Running, Role: LEADER
I20260709 13:22:28.415259 21317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e532ca31c67547998ba2fb09244573b3" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:28.412087 22070 raft_consensus.cc:359] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:28.416720 22070 raft_consensus.cc:740] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:28.418216 22070 consensus_queue.cc:260] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:28.419812 21289 leader_election.cc:304] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: ec879c0528c44465916efc6c33dde636
I20260709 13:22:28.415575 22184 consensus_queue.cc:237] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:28.422312 22116 raft_consensus.cc:3060] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.425396 22070 ts_tablet_manager.cc:1434] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.073s	user 0.005s	sys 0.002s
I20260709 13:22:28.393996 22175 raft_consensus.cc:697] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:28.431027 21886 leader_election.cc:290] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:28.399735 22176 raft_consensus.cc:2749] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:22:28.404456 22162 raft_consensus.cc:493] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:28.423852 21393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0413c4b049a8412b9b3b8a36bb5d67fd" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:28.440912 21438 leader_election.cc:304] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 
I20260709 13:22:28.443432 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "450b7111fe014a51b6e801869913ec52" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:28.444000 21324 raft_consensus.cc:3060] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.453629 21324 raft_consensus.cc:2468] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 2.
I20260709 13:22:28.449103 21398 raft_consensus.cc:2393] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5109d6d6d9404ba39c53508a81fe9f1c in current term 2: Already voted for candidate 74e2ff1a68ac4c849ee6397ebb011375 in this term.
I20260709 13:22:28.440676 22051 raft_consensus.cc:515] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:28.441907 22002 raft_consensus.cc:2804] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:28.442282 22162 raft_consensus.cc:3060] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.447599 22039 raft_consensus.cc:2804] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:28.447175 21393 raft_consensus.cc:3060] T 0413c4b049a8412b9b3b8a36bb5d67fd P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.475756 22051 leader_election.cc:290] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:28.435463 22175 consensus_queue.cc:237] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:28.478266 21319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:28.481477 22039 raft_consensus.cc:493] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:28.483386 21394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01967f3f14be4ecaa1cc2469a50fc156" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:28.483865 22039 raft_consensus.cc:3060] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.484097 21394 raft_consensus.cc:3060] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.481187 21397 raft_consensus.cc:1275] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Refusing update from remote peer 43cd181efe6b4b2fb79fd3e997a401ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.480737 22158 raft_consensus.cc:515] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:28.495994 21473 raft_consensus.cc:2393] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5109d6d6d9404ba39c53508a81fe9f1c in current term 2: Already voted for candidate 43cd181efe6b4b2fb79fd3e997a401ea in this term.
I20260709 13:22:28.497028 22158 leader_election.cc:290] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:28.497179 22110 raft_consensus.cc:2468] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 2.
I20260709 13:22:28.498865 22167 leader_election.cc:290] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:28.479382 22002 raft_consensus.cc:697] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 LEADER]: Becoming Leader. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Running, Role: LEADER
I20260709 13:22:28.504416 22002 consensus_queue.cc:237] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:28.510231 22168 raft_consensus.cc:515] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:28.512552 22168 leader_election.cc:290] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:28.513034 21322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228f7ae94b2848ecb12f93717756ba8b" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:28.503161 22158 consensus_queue.cc:1048] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:22:28.515637 21319 raft_consensus.cc:2393] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 74e2ff1a68ac4c849ee6397ebb011375 in current term 2: Already voted for candidate 5109d6d6d9404ba39c53508a81fe9f1c in this term.
I20260709 13:22:28.520453 22039 raft_consensus.cc:515] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:28.522126 22112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0413c4b049a8412b9b3b8a36bb5d67fd" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:28.523254 22070 tablet_bootstrap.cc:492] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:28.523439 22115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf088eed547f42aea083b31277ecece1" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:28.526386 21393 raft_consensus.cc:2468] T 0413c4b049a8412b9b3b8a36bb5d67fd P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 2.
I20260709 13:22:28.529789 22113 raft_consensus.cc:1275] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Refusing update from remote peer 43cd181efe6b4b2fb79fd3e997a401ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.521111 21394 raft_consensus.cc:2468] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 2.
I20260709 13:22:28.530503 22161 leader_election.cc:290] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:28.531085 21439 leader_election.cc:304] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:28.531363 21291 leader_election.cc:304] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:28.528630 22112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fec8ff4bc5a4d558263e38ae75ece75" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:28.532150 22176 raft_consensus.cc:2804] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:28.532575 22160 raft_consensus.cc:2804] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:28.532680 22108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ddeabcec9b64fc68b40c70df90b80ef" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:28.533054 21320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ddeabcec9b64fc68b40c70df90b80ef" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:28.529495 22039 leader_election.cc:290] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:28.532609 22176 raft_consensus.cc:697] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 LEADER]: Becoming Leader. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Running, Role: LEADER
I20260709 13:22:28.533830 21320 raft_consensus.cc:3060] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.534752 22176 consensus_queue.cc:237] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:28.533191 22160 raft_consensus.cc:697] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:28.538206 22160 consensus_queue.cc:237] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:28.540937 22116 raft_consensus.cc:2468] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 2.
W20260709 13:22:28.544512 21437 leader_election.cc:343] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:28.545801 21437 leader_election.cc:343] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:28.546289 21437 leader_election.cc:304] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636
I20260709 13:22:28.547684 22151 raft_consensus.cc:3060] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:22:28.550350 21363 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:35963, user_credentials={real_user=slave}} blocked reactor thread for 193324us
I20260709 13:22:28.552095 21363 leader_election.cc:304] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:28.529110 22110 raft_consensus.cc:1275] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 4 FOLLOWER]: Refusing update from remote peer 43cd181efe6b4b2fb79fd3e997a401ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:22:28.552852 22155 raft_consensus.cc:2804] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:28.553575 22155 raft_consensus.cc:697] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 LEADER]: Becoming Leader. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:28.554351 22116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ddeabcec9b64fc68b40c70df90b80ef" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:28.554570 22155 consensus_queue.cc:237] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:28.509011 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5fec8ff4bc5a4d558263e38ae75ece75" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:28.556061 22007 leader_election.cc:290] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:28.556329 21325 raft_consensus.cc:3060] T 5fec8ff4bc5a4d558263e38ae75ece75 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:28.553635 22108 raft_consensus.cc:1275] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Refusing update from remote peer 43cd181efe6b4b2fb79fd3e997a401ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.544910 22162 raft_consensus.cc:515] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
W20260709 13:22:28.566242 21289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.68:36863, user_credentials={real_user=slave}} blocked reactor thread for 139317us
I20260709 13:22:28.568270 22162 leader_election.cc:290] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:28.568539 22108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01967f3f14be4ecaa1cc2469a50fc156" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:28.570029 21317 raft_consensus.cc:2393] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 74e2ff1a68ac4c849ee6397ebb011375 in current term 2: Already voted for candidate 5109d6d6d9404ba39c53508a81fe9f1c in this term.
I20260709 13:22:28.574280 21398 raft_consensus.cc:1275] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Refusing update from remote peer 5109d6d6d9404ba39c53508a81fe9f1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.575693 21438 leader_election.cc:304] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 
I20260709 13:22:28.575812 21397 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:28.576630 21397 raft_consensus.cc:2393] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5109d6d6d9404ba39c53508a81fe9f1c in current term 2: Already voted for candidate 74e2ff1a68ac4c849ee6397ebb011375 in this term.
I20260709 13:22:28.577621 21399 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e532ca31c67547998ba2fb09244573b3" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:28.577802 22152 raft_consensus.cc:2804] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:28.578289 22152 raft_consensus.cc:697] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
W20260709 13:22:28.582456 21289 leader_election.cc:343] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:28.583766 21289 leader_election.cc:304] T 5fec8ff4bc5a4d558263e38ae75ece75 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636
I20260709 13:22:28.579205 22152 consensus_queue.cc:237] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:28.591063 22166 consensus_queue.cc:1048] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:28.597286 22116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:28.599424 22108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e532ca31c67547998ba2fb09244573b3" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:28.600283 22108 raft_consensus.cc:2393] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5109d6d6d9404ba39c53508a81fe9f1c in current term 2: Already voted for candidate 74e2ff1a68ac4c849ee6397ebb011375 in this term.
W20260709 13:22:28.603972 21289 leader_election.cc:343] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:28.604396 21289 leader_election.cc:304] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636
I20260709 13:22:28.604682 22108 raft_consensus.cc:1275] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Refusing update from remote peer 5109d6d6d9404ba39c53508a81fe9f1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.615338 22151 raft_consensus.cc:2749] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:22:28.617532 21437 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:35963, user_credentials={real_user=slave}} blocked reactor thread for 71748.7us
I20260709 13:22:28.619277 22183 raft_consensus.cc:2749] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:22:28.620751 22116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:28.622438 22108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "228f7ae94b2848ecb12f93717756ba8b" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:28.624305 21289 leader_election.cc:304] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636
I20260709 13:22:28.624708 22112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf088eed547f42aea083b31277ecece1" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:28.634734 21393 raft_consensus.cc:1275] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 4 FOLLOWER]: Refusing update from remote peer 43cd181efe6b4b2fb79fd3e997a401ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:22:28.642087 21325 raft_consensus.cc:2468] T 5fec8ff4bc5a4d558263e38ae75ece75 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 2.
I20260709 13:22:28.648145 21320 raft_consensus.cc:2468] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 2.
I20260709 13:22:28.649366 22002 consensus_queue.cc:1048] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:28.649597 21438 leader_election.cc:304] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 
I20260709 13:22:28.652725 22175 raft_consensus.cc:2804] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:28.653107 21438 leader_election.cc:304] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 
I20260709 13:22:28.653294 22175 raft_consensus.cc:697] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:28.653937 21320 raft_consensus.cc:1275] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Refusing update from remote peer 43cd181efe6b4b2fb79fd3e997a401ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.654096 22073 raft_consensus.cc:2804] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:28.654745 22073 raft_consensus.cc:697] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:28.654467 22175 consensus_queue.cc:237] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:28.655750 22073 consensus_queue.cc:237] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:28.656104 22151 consensus_queue.cc:1048] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:28.659163 22110 raft_consensus.cc:1275] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Refusing update from remote peer 74e2ff1a68ac4c849ee6397ebb011375: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.670011 21473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf088eed547f42aea083b31277ecece1" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:28.644217 22183 raft_consensus.cc:2749] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:22:28.678514 22070 tablet_bootstrap.cc:492] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:28.646323 22162 raft_consensus.cc:2749] T 5fec8ff4bc5a4d558263e38ae75ece75 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:22:28.695097 21363 leader_election.cc:343] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:28.696141 21363 leader_election.cc:304] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:28.683672 22108 raft_consensus.cc:2393] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 74e2ff1a68ac4c849ee6397ebb011375 in current term 2: Already voted for candidate 5109d6d6d9404ba39c53508a81fe9f1c in this term.
I20260709 13:22:28.717288 22070 tablet_bootstrap.cc:492] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:28.718196 22070 ts_tablet_manager.cc:1403] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.195s	user 0.012s	sys 0.009s
I20260709 13:22:28.713761 22153 raft_consensus.cc:2749] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:22:28.724583 21437 leader_election.cc:343] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:28.726408 21437 leader_election.cc:343] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:28.730309 21363 leader_election.cc:343] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:22:28.737712 21437 leader_election.cc:343] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:28.720506 22160 consensus_queue.cc:1048] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:22:28.711874 22154 consensus_queue.cc:1048] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:22:28.752135 21471 raft_consensus.cc:1275] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Refusing update from remote peer 74e2ff1a68ac4c849ee6397ebb011375: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.761744 22164 consensus_queue.cc:1048] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:28.767361 21325 raft_consensus.cc:1275] T 1fff6b36d1984d38b6f791e0ac15f56b P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Refusing update from remote peer 43cd181efe6b4b2fb79fd3e997a401ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.734737 22178 consensus_queue.cc:1048] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 13:22:28.796826 22038 consensus_queue.cc:1048] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:28.789115 21363 leader_election.cc:304] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:28.804533 21324 raft_consensus.cc:1275] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Refusing update from remote peer 43cd181efe6b4b2fb79fd3e997a401ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.814232 21393 raft_consensus.cc:1275] T f4e1cba0e20343aeb61c31c917dd1f50 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Refusing update from remote peer 43cd181efe6b4b2fb79fd3e997a401ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.818167 21823 raft_consensus.cc:2749] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:22:28.828892 21473 raft_consensus.cc:2468] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 2.
W20260709 13:22:28.842267 21289 consensus_peers.cc:597] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:28.845294 22070 raft_consensus.cc:359] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:28.846067 22070 raft_consensus.cc:740] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:28.850760 22070 consensus_queue.cc:260] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:28.856928 22038 consensus_queue.cc:1048] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:28.861129 22070 ts_tablet_manager.cc:1434] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.143s	user 0.006s	sys 0.000s
I20260709 13:22:28.862950 22070 tablet_bootstrap.cc:492] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:28.863313 22151 consensus_queue.cc:1048] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
W20260709 13:22:28.885242 21363 consensus_peers.cc:597] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:28.849618 22152 consensus_queue.cc:1048] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:22:28.906809 21363 leader_election.cc:304] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 74e2ff1a68ac4c849ee6397ebb011375; no voters: ec879c0528c44465916efc6c33dde636
I20260709 13:22:28.923298 22155 consensus_queue.cc:1048] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:22:28.923981 22209 log.cc:927] Time spent T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:22:28.925781 22157 raft_consensus.cc:2804] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:28.926406 22157 raft_consensus.cc:697] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 LEADER]: Becoming Leader. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:28.927299 22157 consensus_queue.cc:237] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
W20260709 13:22:28.936681 21363 consensus_peers.cc:597] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:28.958027 22113 raft_consensus.cc:1275] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Refusing update from remote peer 43cd181efe6b4b2fb79fd3e997a401ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.960755 21325 raft_consensus.cc:1275] T e4905b7b4db2444484e9403fed337486 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Refusing update from remote peer 43cd181efe6b4b2fb79fd3e997a401ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.975291 21400 raft_consensus.cc:1275] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Refusing update from remote peer 5109d6d6d9404ba39c53508a81fe9f1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:28.996706 22111 raft_consensus.cc:1275] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Refusing update from remote peer 43cd181efe6b4b2fb79fd3e997a401ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:22:28.998594 21289 consensus_peers.cc:597] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:29.002895 21437 consensus_peers.cc:597] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:29.005774 21472 raft_consensus.cc:1275] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Refusing update from remote peer 74e2ff1a68ac4c849ee6397ebb011375: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:29.006072 21325 raft_consensus.cc:1275] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Refusing update from remote peer 43cd181efe6b4b2fb79fd3e997a401ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:22:29.013885 21437 consensus_peers.cc:597] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:29.008195 22038 consensus_queue.cc:1048] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:22:28.974351 22073 consensus_queue.cc:1048] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 13:22:28.987787 22162 consensus_queue.cc:1048] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:22:29.012120 22071 consensus_queue.cc:1048] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:22:29.039485 22213 consensus_queue.cc:1048] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:22:29.011710 22178 consensus_queue.cc:1048] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:22:29.076807 22114 raft_consensus.cc:1275] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Refusing update from remote peer 74e2ff1a68ac4c849ee6397ebb011375: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:22:29.212126 22222 log.cc:927] Time spent T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:22:29.240101 22070 tablet_bootstrap.cc:492] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:29.253475 22070 tablet_bootstrap.cc:492] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:29.254850 22070 ts_tablet_manager.cc:1403] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.392s	user 0.014s	sys 0.016s
W20260709 13:22:29.258085 21363 consensus_peers.cc:597] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:29.257915 21145 catalog_manager.cc:5697] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 reported cstate change: term changed from 1 to 2, leader changed from ec879c0528c44465916efc6c33dde636 (127.11.63.65) to 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67). New cstate: current_term: 2 leader_uuid: "74e2ff1a68ac4c849ee6397ebb011375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:29.259764 22156 consensus_queue.cc:1048] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:22:29.263139 21363 consensus_peers.cc:597] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:29.285581 22070 raft_consensus.cc:359] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:29.286600 22070 raft_consensus.cc:740] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:29.287472 22070 consensus_queue.cc:260] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:29.314874 22070 ts_tablet_manager.cc:1434] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.059s	user 0.007s	sys 0.000s
W20260709 13:22:29.326256 21437 consensus_peers.cc:597] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:29.336720 21289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:35963, user_credentials={real_user=slave}} blocked reactor thread for 313069us
I20260709 13:22:29.348176 21467 raft_consensus.cc:1275] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Refusing update from remote peer 5109d6d6d9404ba39c53508a81fe9f1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:22:29.350615 21289 consensus_peers.cc:597] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:29.350766 22002 consensus_queue.cc:1048] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:22:29.357116 21289 consensus_peers.cc:597] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:29.358390 21289 consensus_peers.cc:597] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:29.359735 22070 tablet_bootstrap.cc:492] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:29.362601 21399 raft_consensus.cc:1275] T 0413c4b049a8412b9b3b8a36bb5d67fd P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Refusing update from remote peer 43cd181efe6b4b2fb79fd3e997a401ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:29.380389 21325 raft_consensus.cc:1275] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Refusing update from remote peer 43cd181efe6b4b2fb79fd3e997a401ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:29.386567 22116 raft_consensus.cc:1275] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Refusing update from remote peer 43cd181efe6b4b2fb79fd3e997a401ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:29.400872 22180 consensus_queue.cc:1048] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:22:29.386610 22159 consensus_queue.cc:1048] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:22:29.405283 21398 raft_consensus.cc:1275] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Refusing update from remote peer 5109d6d6d9404ba39c53508a81fe9f1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:22:29.435032 21437 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:35963, user_credentials={real_user=slave}} blocked reactor thread for 107846us
W20260709 13:22:29.441926 21363 consensus_peers.cc:597] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:29.443094 21363 consensus_peers.cc:597] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:29.446260 22070 tablet_bootstrap.cc:492] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:22:29.446607 21437 consensus_peers.cc:597] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:29.448582 21439 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:43181, user_credentials={real_user=slave}} blocked reactor thread for 48463.9us
W20260709 13:22:29.453419 21437 consensus_peers.cc:597] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:29.457866 22183 consensus_queue.cc:1048] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:22:29.473515 21363 consensus_peers.cc:597] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:29.474287 21363 consensus_peers.cc:597] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:29.479825 22070 tablet_bootstrap.cc:492] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:29.480669 22070 ts_tablet_manager.cc:1403] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.121s	user 0.008s	sys 0.009s
W20260709 13:22:29.566390 22240 log.cc:927] Time spent T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:22:29.564858 22070 raft_consensus.cc:359] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:29.569628 22070 raft_consensus.cc:740] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:29.574113 22070 consensus_queue.cc:260] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:29.579236 22070 ts_tablet_manager.cc:1434] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.098s	user 0.009s	sys 0.000s
I20260709 13:22:29.452994 22073 consensus_queue.cc:1048] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
W20260709 13:22:29.580420 21438 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.66:38825, user_credentials={real_user=slave}} blocked reactor thread for 183604us
W20260709 13:22:29.594751 21439 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:43181, user_credentials={real_user=slave}} blocked reactor thread for 112771us
I20260709 13:22:29.634382 21472 raft_consensus.cc:1275] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Refusing update from remote peer 74e2ff1a68ac4c849ee6397ebb011375: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:29.647864 21317 raft_consensus.cc:1275] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Refusing update from remote peer 43cd181efe6b4b2fb79fd3e997a401ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:29.674394 22039 consensus_queue.cc:1048] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:29.688156 22110 raft_consensus.cc:3060] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:29.688589 22109 raft_consensus.cc:1275] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Refusing update from remote peer 43cd181efe6b4b2fb79fd3e997a401ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:22:29.702950 21437 consensus_peers.cc:597] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:29.703737 21437 consensus_peers.cc:597] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:29.703864 22250 log.cc:927] Time spent T 0413c4b049a8412b9b3b8a36bb5d67fd P 74e2ff1a68ac4c849ee6397ebb011375: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
W20260709 13:22:29.706054 21437 consensus_peers.cc:597] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:29.706735 21437 consensus_peers.cc:597] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:29.708647 21321 raft_consensus.cc:1275] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Refusing update from remote peer 74e2ff1a68ac4c849ee6397ebb011375: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:29.712126 22178 consensus_queue.cc:1048] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:29.720577 22074 consensus_queue.cc:1048] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
W20260709 13:22:29.744028 21437 consensus_peers.cc:597] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:29.759037 21289 consensus_peers.cc:597] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:29.764982 21320 raft_consensus.cc:1275] T 5fec8ff4bc5a4d558263e38ae75ece75 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Refusing update from remote peer 43cd181efe6b4b2fb79fd3e997a401ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:29.775547 21145 catalog_manager.cc:5697] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 reported cstate change: term changed from 1 to 2, leader changed from ec879c0528c44465916efc6c33dde636 (127.11.63.65) to 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67). New cstate: current_term: 2 leader_uuid: "74e2ff1a68ac4c849ee6397ebb011375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:29.777835 21147 catalog_manager.cc:5697] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 3 to 4, leader changed from ec879c0528c44465916efc6c33dde636 (127.11.63.65) to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 4 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:29.791769 22153 consensus_queue.cc:1048] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
W20260709 13:22:29.803364 21437 consensus_peers.cc:597] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:29.812881 22217 consensus_queue.cc:1048] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:29.821617 22108 raft_consensus.cc:1275] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Refusing update from remote peer 5109d6d6d9404ba39c53508a81fe9f1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:29.822322 22070 tablet_bootstrap.cc:492] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
W20260709 13:22:29.840133 21363 consensus_peers.cc:597] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:29.891057 21291 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:43181, user_credentials={real_user=slave}} blocked reactor thread for 72020us
W20260709 13:22:29.902058 21363 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:35963, user_credentials={real_user=slave}} blocked reactor thread for 56039.1us
W20260709 13:22:29.903218 21363 consensus_peers.cc:597] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:29.907640 21289 consensus_peers.cc:597] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:29.909663 21147 catalog_manager.cc:5697] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 1 to 2, leader changed from ec879c0528c44465916efc6c33dde636 (127.11.63.65) to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 2 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:29.911384 21147 catalog_manager.cc:5697] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 1 to 2, leader changed from ec879c0528c44465916efc6c33dde636 (127.11.63.65) to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 2 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:29.912855 21147 catalog_manager.cc:5697] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 1 to 2, leader changed from ec879c0528c44465916efc6c33dde636 (127.11.63.65) to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 2 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:29.914675 21147 catalog_manager.cc:5697] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 1 to 2, leader changed from ec879c0528c44465916efc6c33dde636 (127.11.63.65) to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 2 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:29.916924 21147 catalog_manager.cc:5697] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 1 to 2, leader changed from ec879c0528c44465916efc6c33dde636 (127.11.63.65) to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 2 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:29.929373 21147 catalog_manager.cc:5697] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 1 to 2, leader changed from ec879c0528c44465916efc6c33dde636 (127.11.63.65) to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 2 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:29.933928 22070 tablet_bootstrap.cc:492] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:29.950402 21147 catalog_manager.cc:5697] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 1 to 2, leader changed from ec879c0528c44465916efc6c33dde636 (127.11.63.65) to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 2 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } }
W20260709 13:22:29.970762 21289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:35963, user_credentials={real_user=slave}} blocked reactor thread for 62595.5us
I20260709 13:22:29.976325 22176 consensus_queue.cc:1048] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:29.995108 22070 tablet_bootstrap.cc:492] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:29.995455 21147 catalog_manager.cc:5697] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 1 to 2, leader changed from ec879c0528c44465916efc6c33dde636 (127.11.63.65) to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 2 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:29.997824 21147 catalog_manager.cc:5697] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 1 to 2, leader changed from ec879c0528c44465916efc6c33dde636 (127.11.63.65) to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 2 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:29.997398 22070 ts_tablet_manager.cc:1403] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.175s	user 0.020s	sys 0.000s
I20260709 13:22:29.999730 21147 catalog_manager.cc:5697] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 1 to 2, leader changed from ec879c0528c44465916efc6c33dde636 (127.11.63.65) to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 2 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:30.013545 22070 raft_consensus.cc:359] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:30.014344 22070 raft_consensus.cc:740] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:30.014812 21149 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33328
I20260709 13:22:30.015201 22070 consensus_queue.cc:260] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:30.016546 21149 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33328 (request call id 1) took 1847 ms. Trace:
I20260709 13:22:30.017853 22070 ts_tablet_manager.cc:1434] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:22:30.017344 21149 rpcz_store.cc:276] 0709 13:22:28.169043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:28.169263 (+   220us) service_pool.cc:224] Handling call
0709 13:22:28.305909 (+136646us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b54003ab954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31c090 00007f4c6d3baf3a 00007f4c731711cc 00007f4c7326f37e 00007f4c717302a4 00007f4c71730232 00007f4c71730161 00007f4c717300dc 00007f4c7172f392 00007f4c6b6ed48c 00007f4c6b6ec8c6
0709 13:22:30.016464 (+1710555us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3660,"reported_tablets":45,"spinlock_wait_cycles":37716352}
I20260709 13:22:30.018816 22070 tablet_bootstrap.cc:492] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
W20260709 13:22:30.019526 21437 consensus_peers.cc:597] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:30.024269 22144 heartbeater.cc:499] Master 127.11.63.126:41481 was elected leader, sending a full tablet report...
I20260709 13:22:30.029481 22116 raft_consensus.cc:3060] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:30.138924 22070 tablet_bootstrap.cc:492] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:30.140098 22070 tablet_bootstrap.cc:492] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:30.140793 22070 ts_tablet_manager.cc:1403] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.122s	user 0.029s	sys 0.039s
I20260709 13:22:30.143465 22070 raft_consensus.cc:359] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:30.145130 22070 raft_consensus.cc:740] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:30.145874 22070 consensus_queue.cc:260] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:30.148339 22070 ts_tablet_manager.cc:1434] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:22:30.149255 22070 tablet_bootstrap.cc:492] T 347c76112a0e430ca7d607278f20c811 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:30.149808 21145 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47836 (request call id 110) took 1359 ms. Trace:
I20260709 13:22:30.150211 21145 rpcz_store.cc:276] 0709 13:22:28.790623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:28.790822 (+   199us) service_pool.cc:224] Handling call
0709 13:22:30.149750 (+1358928us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:22:30.007216 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:22:30.007806 (+   590us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:22:30.007843 (+    37us) write_op.cc:435] Acquired schema lock
0709 13:22:30.007860 (+    17us) tablet.cc:662] Decoding operations
0709 13:22:30.008235 (+   375us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:22:30.008322 (+    87us) write_op.cc:641] Partition lock acquired for write op
0709 13:22:30.008346 (+    24us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:22:30.008666 (+   320us) tablet.cc:701] Row locks acquired
0709 13:22:30.008682 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:22:30.008872 (+   190us) write_op.cc:270] Start()
0709 13:22:30.008980 (+   108us) write_op.cc:276] Timestamp: P: 1783603350008827 usec, L: 0
0709 13:22:30.009002 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:22:30.010613 (+  1611us) log.cc:844] Serialized 922 byte log entry
0709 13:22:30.119277 (+108664us) op_driver.cc:464] REPLICATION: finished
0709 13:22:30.142300 (+ 23023us) write_op.cc:301] APPLY: starting
0709 13:22:30.142384 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 13:22:30.142492 (+   108us) tablet.cc:1370] finished BulkCheckPresence
0709 13:22:30.142614 (+   122us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:22:30.143034 (+   420us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:22:30.143188 (+   154us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:22:30.143229 (+    41us) write_op.cc:312] APPLY: finished
0709 13:22:30.143405 (+   176us) log.cc:844] Serialized 28 byte log entry
0709 13:22:30.143560 (+   155us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:30.143637 (+    77us) write_op.cc:454] Released schema lock
0709 13:22:30.143674 (+    37us) write_op.cc:341] FINISH: Updating metrics
0709 13:22:30.144010 (+   336us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:30.144037 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":794143,"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":22776,"apply.run_cpu_time_us":1629,"apply.run_wall_time_us":1701,"num_ops":2,"prepare.queue_time_us":218315,"prepare.run_cpu_time_us":7839,"prepare.run_wall_time_us":46947,"raft.queue_time_us":67921,"raft.run_cpu_time_us":15836,"raft.run_wall_time_us":19654,"replication_time_us":110105,"thread_start_us":202140,"threads_started":4,"wal-append.queue_time_us":38633}]]}
I20260709 13:22:30.177731 21148 catalog_manager.cc:5697] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c reported cstate change: term changed from 1 to 2, leader changed from ec879c0528c44465916efc6c33dde636 (127.11.63.65) to 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66). New cstate: current_term: 2 leader_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:30.178802 21148 catalog_manager.cc:5697] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c reported cstate change: term changed from 1 to 2, leader changed from ec879c0528c44465916efc6c33dde636 (127.11.63.65) to 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66). New cstate: current_term: 2 leader_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:30.180312 21148 catalog_manager.cc:5697] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c reported cstate change: term changed from 1 to 2, leader changed from ec879c0528c44465916efc6c33dde636 (127.11.63.65) to 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66). New cstate: current_term: 2 leader_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:30.182127 21147 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47842 (request call id 97) took 1307 ms. Trace:
I20260709 13:22:30.182493 21147 rpcz_store.cc:276] 0709 13:22:28.875001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:28.875232 (+   231us) service_pool.cc:224] Handling call
0709 13:22:30.182052 (+1306820us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:22:30.067289 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:22:30.067925 (+   636us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:22:30.067960 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:22:30.067975 (+    15us) tablet.cc:662] Decoding operations
0709 13:22:30.069016 (+  1041us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:22:30.069068 (+    52us) write_op.cc:641] Partition lock acquired for write op
0709 13:22:30.069100 (+    32us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:22:30.133718 (+ 64618us) tablet.cc:701] Row locks acquired
0709 13:22:30.133739 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:22:30.133927 (+   188us) write_op.cc:270] Start()
0709 13:22:30.134056 (+   129us) write_op.cc:276] Timestamp: P: 1783603350133868 usec, L: 0
0709 13:22:30.134078 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:22:30.141159 (+  7081us) log.cc:844] Serialized 4384 byte log entry
0709 13:22:30.148088 (+  6929us) op_driver.cc:464] REPLICATION: finished
0709 13:22:30.152266 (+  4178us) spinlock_profiling.cc:243] Waited 4.03 ms on lock 0x7b54003ab954. stack: 00007f4c6a293343 00007f4c6a293240 00007f4c69c221d4 00007f4c73960304 00007f4c73960299 00007f4c6d31b571 00007f4c6d3ade28 00007f4c70de8581 00007f4c70de7d58 00007f4c70dec9f1 00007f4c70dec9a9 00007f4c70dec939 00007f4c70dec901 00007f4c70debb2d 000000000038f354 000000000038f199
0709 13:22:30.158160 (+  5894us) write_op.cc:301] APPLY: starting
0709 13:22:30.158243 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 13:22:30.158506 (+   263us) tablet.cc:1370] finished BulkCheckPresence
0709 13:22:30.158544 (+    38us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:22:30.159757 (+  1213us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:22:30.159974 (+   217us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 13:22:30.160021 (+    47us) write_op.cc:312] APPLY: finished
0709 13:22:30.160322 (+   301us) log.cc:844] Serialized 82 byte log entry
0709 13:22:30.166045 (+  5723us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:30.166202 (+   157us) write_op.cc:454] Released schema lock
0709 13:22:30.166260 (+    58us) write_op.cc:341] FINISH: Updating metrics
0709 13:22:30.166721 (+   461us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:30.166761 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":498878,"reported_tablets":12,"child_traces":[["op",{"apply.queue_time_us":9843,"apply.run_cpu_time_us":2965,"apply.run_wall_time_us":8509,"num_ops":11,"prepare.queue_time_us":2780,"prepare.run_cpu_time_us":6875,"prepare.run_wall_time_us":91295,"replication_time_us":13890,"spinlock_wait_cycles":9266432}]]}
I20260709 13:22:30.199674 22070 tablet_bootstrap.cc:492] T 347c76112a0e430ca7d607278f20c811 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:30.211869 22070 tablet_bootstrap.cc:492] T 347c76112a0e430ca7d607278f20c811 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:30.211457 21149 catalog_manager.cc:5697] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 reported cstate change: term changed from 1 to 2, leader changed from ec879c0528c44465916efc6c33dde636 (127.11.63.65) to 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67). New cstate: current_term: 2 leader_uuid: "74e2ff1a68ac4c849ee6397ebb011375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:30.212639 22070 ts_tablet_manager.cc:1403] T 347c76112a0e430ca7d607278f20c811 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.064s	user 0.014s	sys 0.005s
I20260709 13:22:30.215150 22070 raft_consensus.cc:359] T 347c76112a0e430ca7d607278f20c811 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:30.215783 22070 raft_consensus.cc:740] T 347c76112a0e430ca7d607278f20c811 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:30.216590 22070 consensus_queue.cc:260] T 347c76112a0e430ca7d607278f20c811 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:30.217449 21148 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47834 (request call id 98) took 1201 ms. Trace:
I20260709 13:22:30.217923 21148 rpcz_store.cc:276] 0709 13:22:29.015605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:22:29.016186 (+   581us) service_pool.cc:224] Handling call
0709 13:22:30.217392 (+1201206us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:22:30.188743 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:22:30.196940 (+  8197us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:22:30.196978 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:22:30.196996 (+    18us) tablet.cc:662] Decoding operations
0709 13:22:30.197625 (+   629us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:22:30.197690 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 13:22:30.197719 (+    29us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:22:30.198050 (+   331us) tablet.cc:701] Row locks acquired
0709 13:22:30.198070 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:22:30.198211 (+   141us) write_op.cc:270] Start()
0709 13:22:30.198322 (+   111us) write_op.cc:276] Timestamp: P: 1783603350198166 usec, L: 0
0709 13:22:30.198347 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:22:30.198976 (+   629us) log.cc:844] Serialized 1307 byte log entry
0709 13:22:30.202457 (+  3481us) op_driver.cc:464] REPLICATION: finished
0709 13:22:30.202891 (+   434us) write_op.cc:301] APPLY: starting
0709 13:22:30.202979 (+    88us) tablet.cc:1367] starting BulkCheckPresence
0709 13:22:30.203136 (+   157us) tablet.cc:1370] finished BulkCheckPresence
0709 13:22:30.203239 (+   103us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:22:30.203674 (+   435us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:22:30.203846 (+   172us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:22:30.203906 (+    60us) write_op.cc:312] APPLY: finished
0709 13:22:30.204119 (+   213us) log.cc:844] Serialized 34 byte log entry
0709 13:22:30.204289 (+   170us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:30.204386 (+    97us) write_op.cc:454] Released schema lock
0709 13:22:30.204427 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 13:22:30.204748 (+   321us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:22:30.204777 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":242492,"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":245,"apply.run_cpu_time_us":1822,"apply.run_wall_time_us":1821,"num_ops":3,"prepare.queue_time_us":110,"prepare.run_cpu_time_us":3125,"prepare.run_wall_time_us":10531,"replication_time_us":3996}]]}
I20260709 13:22:30.221822 22070 ts_tablet_manager.cc:1434] T 347c76112a0e430ca7d607278f20c811 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:22:30.222682 22070 tablet_bootstrap.cc:492] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:30.271229 22070 tablet_bootstrap.cc:492] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:30.272475 22070 tablet_bootstrap.cc:492] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:30.273212 22070 ts_tablet_manager.cc:1403] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.051s	user 0.017s	sys 0.005s
I20260709 13:22:30.276136 22070 raft_consensus.cc:359] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:30.276883 22070 raft_consensus.cc:740] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:30.277671 22070 consensus_queue.cc:260] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:30.290169 22070 ts_tablet_manager.cc:1434] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:22:30.291193 22070 tablet_bootstrap.cc:492] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:30.337368 22070 tablet_bootstrap.cc:492] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:30.338598 22070 tablet_bootstrap.cc:492] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:30.339286 22070 ts_tablet_manager.cc:1403] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.048s	user 0.012s	sys 0.019s
I20260709 13:22:30.342577 22070 raft_consensus.cc:359] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:30.343221 22070 raft_consensus.cc:740] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:30.343819 22070 consensus_queue.cc:260] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:30.346035 22070 ts_tablet_manager.cc:1434] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:22:30.347107 22070 tablet_bootstrap.cc:492] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:30.370106 22070 tablet_bootstrap.cc:492] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:30.371165 22070 tablet_bootstrap.cc:492] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:30.371927 22070 ts_tablet_manager.cc:1403] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.004s
I20260709 13:22:30.374306 22070 raft_consensus.cc:359] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:30.375090 22070 raft_consensus.cc:740] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:30.375617 22070 consensus_queue.cc:260] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:30.377610 22070 ts_tablet_manager.cc:1434] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:22:30.378753 22070 tablet_bootstrap.cc:492] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:30.393165 22070 tablet_bootstrap.cc:492] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:30.394228 22070 tablet_bootstrap.cc:492] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:30.394948 22070 ts_tablet_manager.cc:1403] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.001s
I20260709 13:22:30.397927 22070 raft_consensus.cc:359] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:30.398623 22070 raft_consensus.cc:740] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:30.399166 22070 consensus_queue.cc:260] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:30.401017 22070 ts_tablet_manager.cc:1434] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:22:30.402204 22070 tablet_bootstrap.cc:492] T 6b569ec3d17d412c8c9087446e64bb77 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:30.421006 22070 tablet_bootstrap.cc:492] T 6b569ec3d17d412c8c9087446e64bb77 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:30.421916 22070 tablet_bootstrap.cc:492] T 6b569ec3d17d412c8c9087446e64bb77 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:30.422605 22070 ts_tablet_manager.cc:1403] T 6b569ec3d17d412c8c9087446e64bb77 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.007s
I20260709 13:22:30.424880 22070 raft_consensus.cc:359] T 6b569ec3d17d412c8c9087446e64bb77 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:30.425333 22070 raft_consensus.cc:740] T 6b569ec3d17d412c8c9087446e64bb77 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:30.425887 22070 consensus_queue.cc:260] T 6b569ec3d17d412c8c9087446e64bb77 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:30.427727 22070 ts_tablet_manager.cc:1434] T 6b569ec3d17d412c8c9087446e64bb77 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:22:30.428792 22070 tablet_bootstrap.cc:492] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:30.443536 22070 tablet_bootstrap.cc:492] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:30.444507 22070 tablet_bootstrap.cc:492] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:30.445163 22070 ts_tablet_manager.cc:1403] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.004s
I20260709 13:22:30.447538 22070 raft_consensus.cc:359] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:30.448218 22070 raft_consensus.cc:740] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:30.448833 22070 consensus_queue.cc:260] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:30.450656 22070 ts_tablet_manager.cc:1434] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.003s
I20260709 13:22:30.451401 22070 tablet_bootstrap.cc:492] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:30.468925 22070 tablet_bootstrap.cc:492] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:30.469942 22070 tablet_bootstrap.cc:492] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:30.470619 22070 ts_tablet_manager.cc:1403] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.007s
I20260709 13:22:30.473996 22070 raft_consensus.cc:359] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:30.474745 22070 raft_consensus.cc:740] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:30.475461 22070 consensus_queue.cc:260] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:30.477401 22070 ts_tablet_manager.cc:1434] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:22:30.478257 22070 tablet_bootstrap.cc:492] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:30.495630 22070 tablet_bootstrap.cc:492] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:30.496560 22070 tablet_bootstrap.cc:492] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:30.497232 22070 ts_tablet_manager.cc:1403] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.000s
I20260709 13:22:30.499703 22070 raft_consensus.cc:359] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:30.500370 22070 raft_consensus.cc:740] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:30.501543 22070 consensus_queue.cc:260] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:30.503539 22070 ts_tablet_manager.cc:1434] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:22:30.504405 22070 tablet_bootstrap.cc:492] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:30.681264 22235 raft_consensus.cc:493] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:30.682698 22235 raft_consensus.cc:515] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:30.770134 22070 tablet_bootstrap.cc:492] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:30.771886 22070 tablet_bootstrap.cc:492] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:30.773339 22070 ts_tablet_manager.cc:1403] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.269s	user 0.026s	sys 0.006s
I20260709 13:22:30.776821 21823 raft_consensus.cc:493] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:30.777628 21823 raft_consensus.cc:515] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:30.778771 22070 raft_consensus.cc:359] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:30.779788 22070 raft_consensus.cc:740] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:30.780691 21823 leader_election.cc:290] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:30.787637 22115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
W20260709 13:22:30.790418 21363 leader_election.cc:343] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:30.795317 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:30.796270 21324 raft_consensus.cc:2468] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 2.
I20260709 13:22:30.753304 22235 leader_election.cc:290] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:30.799376 21398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:30.800272 21398 raft_consensus.cc:2468] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 2.
I20260709 13:22:30.818661 22116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:30.821513 22110 raft_consensus.cc:3060] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:30.828598 22070 consensus_queue.cc:260] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:30.850430 21291 leader_election.cc:304] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:30.882068 22070 ts_tablet_manager.cc:1434] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.108s	user 0.006s	sys 0.005s
W20260709 13:22:30.888581 21289 leader_election.cc:343] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:30.888219 21364 leader_election.cc:304] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375; no voters: ec879c0528c44465916efc6c33dde636
I20260709 13:22:30.897792 22156 raft_consensus.cc:2804] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:22:30.898420 22156 raft_consensus.cc:493] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:30.898947 22156 raft_consensus.cc:3060] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:30.895985 22294 raft_consensus.cc:2804] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:22:30.924001 22294 raft_consensus.cc:493] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:30.932698 22294 raft_consensus.cc:3060] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:22:30.934895 21289 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:35963, user_credentials={real_user=slave}} blocked reactor thread for 87034us
I20260709 13:22:30.933920 22070 tablet_bootstrap.cc:492] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:30.963610 22113 raft_consensus.cc:3060] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:30.976140 22156 raft_consensus.cc:515] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:30.978431 22156 leader_election.cc:290] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 3 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:30.980199 22115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
W20260709 13:22:30.982244 21363 leader_election.cc:343] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 3 election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:30.990038 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:30.994513 22294 raft_consensus.cc:515] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:30.996997 21324 raft_consensus.cc:2393] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 74e2ff1a68ac4c849ee6397ebb011375 in current term 3: Already voted for candidate 5109d6d6d9404ba39c53508a81fe9f1c in this term.
I20260709 13:22:31.000362 21364 leader_election.cc:304] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:31.001226 22156 raft_consensus.cc:2749] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:22:31.003002 22294 leader_election.cc:290] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 3 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:31.003780 22116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:31.003983 21398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:31.004841 21398 raft_consensus.cc:2393] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5109d6d6d9404ba39c53508a81fe9f1c in current term 3: Already voted for candidate 74e2ff1a68ac4c849ee6397ebb011375 in this term.
W20260709 13:22:31.005450 21289 leader_election.cc:343] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 3 election: Tablet error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:31.006336 21291 leader_election.cc:304] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636
I20260709 13:22:31.009119 22294 raft_consensus.cc:2749] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:22:31.021014 22070 tablet_bootstrap.cc:492] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:31.022059 22070 tablet_bootstrap.cc:492] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:31.022758 22070 ts_tablet_manager.cc:1403] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.089s	user 0.009s	sys 0.009s
I20260709 13:22:31.028671 22070 raft_consensus.cc:359] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:31.029287 22070 raft_consensus.cc:740] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.030040 22070 consensus_queue.cc:260] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:31.032537 22070 ts_tablet_manager.cc:1434] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.003s
I20260709 13:22:31.033710 22070 tablet_bootstrap.cc:492] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:31.055434 22070 tablet_bootstrap.cc:492] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:31.056252 22070 tablet_bootstrap.cc:492] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:31.056849 22070 ts_tablet_manager.cc:1403] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.007s
I20260709 13:22:31.059124 22070 raft_consensus.cc:359] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:31.059741 22070 raft_consensus.cc:740] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.060380 22070 consensus_queue.cc:260] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:31.061933 22070 ts_tablet_manager.cc:1434] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:22:31.063081 22070 tablet_bootstrap.cc:492] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:31.080212 22070 tablet_bootstrap.cc:492] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:31.081012 22070 tablet_bootstrap.cc:492] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:31.081688 22070 ts_tablet_manager.cc:1403] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 13:22:31.084262 22070 raft_consensus.cc:359] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:31.084841 22070 raft_consensus.cc:740] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.085620 22070 consensus_queue.cc:260] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:31.087947 22070 ts_tablet_manager.cc:1434] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:22:31.089238 22070 tablet_bootstrap.cc:492] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:31.111830 22070 tablet_bootstrap.cc:492] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:31.112702 22070 tablet_bootstrap.cc:492] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:31.113180 22070 ts_tablet_manager.cc:1403] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.003s
I20260709 13:22:31.115501 22070 raft_consensus.cc:359] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:31.116173 22070 raft_consensus.cc:740] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.116884 22070 consensus_queue.cc:260] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:31.119131 22070 ts_tablet_manager.cc:1434] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 13:22:31.121023 22070 tablet_bootstrap.cc:492] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:31.139276 22070 tablet_bootstrap.cc:492] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:31.140441 22070 tablet_bootstrap.cc:492] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:31.141157 22070 ts_tablet_manager.cc:1403] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.004s
I20260709 13:22:31.143842 22070 raft_consensus.cc:359] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:31.144567 22070 raft_consensus.cc:740] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.145318 22070 consensus_queue.cc:260] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:31.147393 22070 ts_tablet_manager.cc:1434] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:22:31.148562 22070 tablet_bootstrap.cc:492] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:31.168529 22070 tablet_bootstrap.cc:492] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:31.169469 22070 tablet_bootstrap.cc:492] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:31.170167 22070 ts_tablet_manager.cc:1403] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.000s
I20260709 13:22:31.172741 22070 raft_consensus.cc:359] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:31.173945 22070 raft_consensus.cc:740] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.174615 22070 consensus_queue.cc:260] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:31.176730 22070 ts_tablet_manager.cc:1434] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:22:31.177836 22070 tablet_bootstrap.cc:492] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:31.307547 22070 tablet_bootstrap.cc:492] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:31.308416 22070 tablet_bootstrap.cc:492] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:31.309072 22070 ts_tablet_manager.cc:1403] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.132s	user 0.033s	sys 0.054s
I20260709 13:22:31.311811 22070 raft_consensus.cc:359] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:31.312378 22070 raft_consensus.cc:740] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.313123 22070 consensus_queue.cc:260] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:31.315138 22070 ts_tablet_manager.cc:1434] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:22:31.316236 22070 tablet_bootstrap.cc:492] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:31.341987 22070 tablet_bootstrap.cc:492] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:31.342914 22070 tablet_bootstrap.cc:492] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:31.343461 22070 ts_tablet_manager.cc:1403] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.027s	user 0.019s	sys 0.005s
I20260709 13:22:31.345204 22070 raft_consensus.cc:359] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:31.345744 22070 raft_consensus.cc:740] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.346396 22070 consensus_queue.cc:260] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:31.348265 22070 ts_tablet_manager.cc:1434] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:22:31.350302 22070 tablet_bootstrap.cc:492] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:31.366587 22070 tablet_bootstrap.cc:492] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:31.367314 22070 tablet_bootstrap.cc:492] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:31.367846 22070 ts_tablet_manager.cc:1403] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.006s
I20260709 13:22:31.369860 22070 raft_consensus.cc:359] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:31.370441 22070 raft_consensus.cc:740] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.371121 22070 consensus_queue.cc:260] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:31.372913 22070 ts_tablet_manager.cc:1434] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.002s
I20260709 13:22:31.376541 22070 tablet_bootstrap.cc:492] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:31.389047 22070 tablet_bootstrap.cc:492] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:31.389938 22070 tablet_bootstrap.cc:492] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:31.390504 22070 ts_tablet_manager.cc:1403] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.003s
I20260709 13:22:31.392452 22070 raft_consensus.cc:359] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:31.393106 22070 raft_consensus.cc:740] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.393675 22070 consensus_queue.cc:260] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:31.395152 22070 ts_tablet_manager.cc:1434] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:22:31.396168 22070 tablet_bootstrap.cc:492] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:31.410257 22070 tablet_bootstrap.cc:492] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:31.411191 22070 tablet_bootstrap.cc:492] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:31.411876 22070 ts_tablet_manager.cc:1403] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.000s
I20260709 13:22:31.414187 22070 raft_consensus.cc:359] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:31.414820 22070 raft_consensus.cc:740] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.415468 22070 consensus_queue.cc:260] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:31.417475 22070 ts_tablet_manager.cc:1434] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:22:31.418294 22070 tablet_bootstrap.cc:492] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:31.433640 22070 tablet_bootstrap.cc:492] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:31.434512 22070 tablet_bootstrap.cc:492] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:31.435141 22070 ts_tablet_manager.cc:1403] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
W20260709 13:22:31.435425 21437 consensus_peers.cc:597] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:31.437304 22070 raft_consensus.cc:359] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:31.438030 22070 raft_consensus.cc:740] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.438726 22070 consensus_queue.cc:260] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:31.440289 22070 ts_tablet_manager.cc:1434] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:22:31.441709 22070 tablet_bootstrap.cc:492] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:31.458387 22070 tablet_bootstrap.cc:492] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:31.459576 22070 tablet_bootstrap.cc:492] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:31.460470 22070 ts_tablet_manager.cc:1403] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.004s
I20260709 13:22:31.463167 22070 raft_consensus.cc:359] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:31.463742 22070 raft_consensus.cc:740] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.464391 22070 consensus_queue.cc:260] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:31.467437 22070 ts_tablet_manager.cc:1434] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:22:31.468305 22070 tablet_bootstrap.cc:492] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:31.484503 22070 tablet_bootstrap.cc:492] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:31.485406 22070 tablet_bootstrap.cc:492] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:31.486035 22070 ts_tablet_manager.cc:1403] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.018s	user 0.017s	sys 0.000s
I20260709 13:22:31.488062 22070 raft_consensus.cc:359] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:31.488757 22070 raft_consensus.cc:740] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.489398 22070 consensus_queue.cc:260] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:31.491694 22070 ts_tablet_manager.cc:1434] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:22:31.493353 22070 tablet_bootstrap.cc:492] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:31.508914 22070 tablet_bootstrap.cc:492] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:31.509752 22070 tablet_bootstrap.cc:492] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:31.510367 22070 ts_tablet_manager.cc:1403] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 13:22:31.512487 22070 raft_consensus.cc:359] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:31.513510 22070 raft_consensus.cc:740] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.514175 22070 consensus_queue.cc:260] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:31.516208 22070 ts_tablet_manager.cc:1434] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:22:31.517807 22070 tablet_bootstrap.cc:492] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636: Bootstrap starting.
I20260709 13:22:31.540378 22070 tablet_bootstrap.cc:492] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:31.541399 22070 tablet_bootstrap.cc:492] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636: Bootstrap complete.
I20260709 13:22:31.542100 22070 ts_tablet_manager.cc:1403] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.006s
I20260709 13:22:31.544762 22070 raft_consensus.cc:359] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:31.545465 22070 raft_consensus.cc:740] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec879c0528c44465916efc6c33dde636, State: Initialized, Role: FOLLOWER
I20260709 13:22:31.546342 22070 consensus_queue.cc:260] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:31.547981 22070 ts_tablet_manager.cc:1434] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:22:31.616079 22113 raft_consensus.cc:3060] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:31.628969 22113 raft_consensus.cc:3060] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:31.709863 22113 raft_consensus.cc:3060] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:31.753096 22113 raft_consensus.cc:3060] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:31.968588 22113 raft_consensus.cc:3060] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:32.009971 22113 raft_consensus.cc:3060] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:32.531478 22333 raft_consensus.cc:493] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.531975 22333 raft_consensus.cc:515] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:32.534754 22333 leader_election.cc:290] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:32.552923 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:32.553144 21398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375" is_pre_election: true
I20260709 13:22:32.553809 21398 raft_consensus.cc:2368] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec879c0528c44465916efc6c33dde636 for earlier term 2. Current term is 3.
I20260709 13:22:32.553809 21324 raft_consensus.cc:2368] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec879c0528c44465916efc6c33dde636 for earlier term 2. Current term is 3.
I20260709 13:22:32.555135 22066 leader_election.cc:400] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181) with higher term. Message: Invalid argument: T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec879c0528c44465916efc6c33dde636 for earlier term 2. Current term is 3.
I20260709 13:22:32.555814 22066 leader_election.cc:403] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:22:32.556367 22065 leader_election.cc:400] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825) with higher term. Message: Invalid argument: T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec879c0528c44465916efc6c33dde636 for earlier term 2. Current term is 3.
I20260709 13:22:32.556690 22333 raft_consensus.cc:3060] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:22:32.563223 22333 raft_consensus.cc:2749] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181) with higher term. Message: Invalid argument: T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec879c0528c44465916efc6c33dde636 for earlier term 2. Current term is 3.
I20260709 13:22:32.922721 22329 raft_consensus.cc:493] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:32.923208 22329 raft_consensus.cc:515] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:32.925077 22329 leader_election.cc:290] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:32.926044 22115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636" is_pre_election: true
I20260709 13:22:32.926831 22115 raft_consensus.cc:2468] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 3.
I20260709 13:22:32.928830 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" is_pre_election: true
I20260709 13:22:32.929015 21363 leader_election.cc:304] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:32.929510 21325 raft_consensus.cc:2468] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 3.
I20260709 13:22:32.930241 22329 raft_consensus.cc:2804] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:22:32.931509 22329 raft_consensus.cc:493] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:32.931805 22329 raft_consensus.cc:3060] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:32.940932 22329 raft_consensus.cc:515] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:32.943064 22329 leader_election.cc:290] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 4 election: Requested vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:32.943617 22115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ec879c0528c44465916efc6c33dde636"
I20260709 13:22:32.943646 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "311e1d9a839a418296e3f320abad6c88" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:32.944473 22115 raft_consensus.cc:3060] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:32.944520 21325 raft_consensus.cc:3060] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:32.950963 21325 raft_consensus.cc:2468] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 4.
I20260709 13:22:32.951808 22115 raft_consensus.cc:2468] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74e2ff1a68ac4c849ee6397ebb011375 in term 4.
I20260709 13:22:32.952271 21364 leader_election.cc:304] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:32.953035 21823 raft_consensus.cc:2804] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:22:32.953526 21823 raft_consensus.cc:697] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 4 LEADER]: Becoming Leader. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:32.954453 21823 consensus_queue.cc:237] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:32.964629 21148 catalog_manager.cc:5697] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 reported cstate change: term changed from 1 to 4, leader changed from ec879c0528c44465916efc6c33dde636 (127.11.63.65) to 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67). New cstate: current_term: 4 leader_uuid: "74e2ff1a68ac4c849ee6397ebb011375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:33.362847 21325 raft_consensus.cc:1275] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 4 FOLLOWER]: Refusing update from remote peer 74e2ff1a68ac4c849ee6397ebb011375: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:22:33.364125 22329 consensus_queue.cc:1048] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:33.392963 22115 raft_consensus.cc:1275] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 4 FOLLOWER]: Refusing update from remote peer 74e2ff1a68ac4c849ee6397ebb011375: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:22:33.394526 22329 consensus_queue.cc:1048] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:22:33.641463 22349 log.cc:927] Time spent T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636: Append to log took a long time: real 0.053s	user 0.004s	sys 0.000s
I20260709 13:22:37.629954 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:37.692836 21398 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a169fb713efc445fb1f0338c7c11203a"
dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
mode: GRACEFUL
new_leader_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:35242
I20260709 13:22:37.693702 21398 raft_consensus.cc:606] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Received request to transfer leadership to TS ec879c0528c44465916efc6c33dde636
I20260709 13:22:37.708887 21467 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a1bf171a09b745ec963d790f1eefe432"
dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
mode: GRACEFUL
new_leader_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:56486
I20260709 13:22:37.709496 21467 raft_consensus.cc:606] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Received request to transfer leadership to TS ec879c0528c44465916efc6c33dde636
I20260709 13:22:37.712280 21467 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad13ab5b32d74d80990d10ba376cc2c7"
dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
mode: GRACEFUL
new_leader_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:56486
I20260709 13:22:37.712838 21467 raft_consensus.cc:606] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Received request to transfer leadership to TS ec879c0528c44465916efc6c33dde636
I20260709 13:22:37.715584 21469 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cdc82236ff024407ae854c96dd8c89ec"
dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
mode: GRACEFUL
new_leader_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
 from {username='slave'} at 127.0.0.1:56486
I20260709 13:22:37.716188 21469 raft_consensus.cc:606] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Received request to transfer leadership to TS 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:37.720666 21467 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2b503b6f79842efbb8a7ecd5c5851e0"
dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
mode: GRACEFUL
new_leader_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:56486
I20260709 13:22:37.721405 21467 raft_consensus.cc:606] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Received request to transfer leadership to TS ec879c0528c44465916efc6c33dde636
I20260709 13:22:37.724200 21467 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dcf55c037e6c48bc9d73f87a54b26f92"
dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
mode: GRACEFUL
new_leader_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:56486
I20260709 13:22:37.724841 21467 raft_consensus.cc:606] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Received request to transfer leadership to TS ec879c0528c44465916efc6c33dde636
I20260709 13:22:37.726056 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:22:37.727916 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:37.846129 22365 raft_consensus.cc:993] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375: : Instructing follower ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:37.846737 22365 raft_consensus.cc:1081] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Signalling peer ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:37.850660 22113 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a169fb713efc445fb1f0338c7c11203a"
dest_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:37042
I20260709 13:22:37.851293 22113 raft_consensus.cc:493] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:37.851653 22113 raft_consensus.cc:3060] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:37.858171 22360 raft_consensus.cc:993] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea: : Instructing follower ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:37.858908 22360 raft_consensus.cc:1081] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Signalling peer ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:37.861109 22113 raft_consensus.cc:515] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:37.861529 22116 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ad13ab5b32d74d80990d10ba376cc2c7"
dest_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:37026
I20260709 13:22:37.862660 22116 raft_consensus.cc:493] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:37.862988 22116 raft_consensus.cc:3060] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:37.864866 22113 leader_election.cc:290] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 2 election: Requested vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:37.866585 21394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169fb713efc445fb1f0338c7c11203a" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:37.867182 21394 raft_consensus.cc:3055] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:37.867465 21394 raft_consensus.cc:740] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:37.865865 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a169fb713efc445fb1f0338c7c11203a" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:37.868331 21325 raft_consensus.cc:3060] T a169fb713efc445fb1f0338c7c11203a P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:37.868183 21394 consensus_queue.cc:260] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:37.870502 21394 raft_consensus.cc:3060] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:37.873215 22116 raft_consensus.cc:515] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:37.876535 21325 raft_consensus.cc:2468] T a169fb713efc445fb1f0338c7c11203a P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 2.
I20260709 13:22:37.877915 22065 leader_election.cc:304] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:37.878646 21398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad13ab5b32d74d80990d10ba376cc2c7" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:37.879388 21398 raft_consensus.cc:3060] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:37.885730 21394 raft_consensus.cc:2468] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 2.
I20260709 13:22:37.886468 22368 raft_consensus.cc:2804] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:37.887965 22368 raft_consensus.cc:697] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [term 2 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:37.890012 22116 leader_election.cc:290] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 2 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:37.888890 22368 consensus_queue.cc:237] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:37.898962 21398 raft_consensus.cc:2468] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 2.
I20260709 13:22:37.900398 22066 leader_election.cc:304] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:37.901338 22369 raft_consensus.cc:2804] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:37.910230 21145 catalog_manager.cc:5697] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 1 to 2, leader changed from 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67) to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 2 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:37.921272 22369 raft_consensus.cc:697] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 [term 2 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:37.921765 21467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad13ab5b32d74d80990d10ba376cc2c7" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:37.922472 21467 raft_consensus.cc:3055] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:37.922827 21467 raft_consensus.cc:740] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:37.922451 22369 consensus_queue.cc:237] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:37.923532 21467 consensus_queue.cc:260] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:37.924860 21467 raft_consensus.cc:3060] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:37.925278 22360 raft_consensus.cc:993] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea: : Instructing follower ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:37.925971 22360 raft_consensus.cc:1081] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Signalling peer ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:37.928110 22115 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dcf55c037e6c48bc9d73f87a54b26f92"
dest_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:37026
I20260709 13:22:37.928817 22115 raft_consensus.cc:493] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:37.929339 22115 raft_consensus.cc:3060] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:37.937520 22115 raft_consensus.cc:515] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:37.938055 21467 raft_consensus.cc:2468] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 2.
I20260709 13:22:37.939721 22115 leader_election.cc:290] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 2 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:37.940966 21398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcf55c037e6c48bc9d73f87a54b26f92" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:37.941869 21469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcf55c037e6c48bc9d73f87a54b26f92" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:37.941897 21398 raft_consensus.cc:3060] T dcf55c037e6c48bc9d73f87a54b26f92 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:37.942636 21469 raft_consensus.cc:3055] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:37.943013 21469 raft_consensus.cc:740] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:37.943709 21469 consensus_queue.cc:260] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:37.945111 21469 raft_consensus.cc:3060] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:37.951442 21469 raft_consensus.cc:2468] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 2.
I20260709 13:22:37.952549 22064 leader_election.cc:304] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:37.953485 22369 raft_consensus.cc:2804] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:37.953979 22369 raft_consensus.cc:697] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [term 2 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:37.954128 21398 raft_consensus.cc:2468] T dcf55c037e6c48bc9d73f87a54b26f92 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 2.
I20260709 13:22:37.954906 22369 consensus_queue.cc:237] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:37.969175 21145 catalog_manager.cc:5697] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 1 to 2, leader changed from 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68) to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 2 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:37.971333 21145 catalog_manager.cc:5697] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 1 to 2, leader changed from 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68) to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 2 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:38.040779 22360 raft_consensus.cc:993] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea: : Instructing follower 5109d6d6d9404ba39c53508a81fe9f1c to start an election
I20260709 13:22:38.041687 22353 raft_consensus.cc:1081] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Signalling peer 5109d6d6d9404ba39c53508a81fe9f1c to start an election
I20260709 13:22:38.044644 21325 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cdc82236ff024407ae854c96dd8c89ec"
dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
 from {username='slave'} at 127.0.0.1:50454
I20260709 13:22:38.045550 21325 raft_consensus.cc:493] T cdc82236ff024407ae854c96dd8c89ec P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:38.045936 21325 raft_consensus.cc:3060] T cdc82236ff024407ae854c96dd8c89ec P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:38.050839 21325 raft_consensus.cc:515] T cdc82236ff024407ae854c96dd8c89ec P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:38.052700 21325 leader_election.cc:290] T cdc82236ff024407ae854c96dd8c89ec P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:38.053483 21398 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc82236ff024407ae854c96dd8c89ec" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:38.053881 21469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdc82236ff024407ae854c96dd8c89ec" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:38.054157 21398 raft_consensus.cc:3060] T cdc82236ff024407ae854c96dd8c89ec P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:38.054517 21469 raft_consensus.cc:3055] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:38.054795 21469 raft_consensus.cc:740] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:38.055495 21469 consensus_queue.cc:260] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:38.056514 21469 raft_consensus.cc:3060] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:38.059971 21398 raft_consensus.cc:2468] T cdc82236ff024407ae854c96dd8c89ec P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 2.
I20260709 13:22:38.061254 21291 leader_election.cc:304] T cdc82236ff024407ae854c96dd8c89ec P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375; no voters: 
I20260709 13:22:38.062282 22351 raft_consensus.cc:2804] T cdc82236ff024407ae854c96dd8c89ec P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:38.062733 22351 raft_consensus.cc:697] T cdc82236ff024407ae854c96dd8c89ec P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 LEADER]: Becoming Leader. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Running, Role: LEADER
I20260709 13:22:38.063592 21469 raft_consensus.cc:2468] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 2.
I20260709 13:22:38.063719 22360 raft_consensus.cc:993] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea: : Instructing follower ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:38.063617 22351 consensus_queue.cc:237] T cdc82236ff024407ae854c96dd8c89ec P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:38.064499 22360 raft_consensus.cc:1081] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Signalling peer ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:38.066723 22115 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a1bf171a09b745ec963d790f1eefe432"
dest_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:37026
I20260709 13:22:38.067299 22115 raft_consensus.cc:493] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:38.067739 22115 raft_consensus.cc:3060] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:38.078876 22115 raft_consensus.cc:515] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:38.081207 22115 leader_election.cc:290] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 2 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:38.081537 21469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1bf171a09b745ec963d790f1eefe432" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:38.080539 21145 catalog_manager.cc:5697] T cdc82236ff024407ae854c96dd8c89ec P 5109d6d6d9404ba39c53508a81fe9f1c reported cstate change: term changed from 1 to 2, leader changed from 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68) to 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66). New cstate: current_term: 2 leader_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:38.081931 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1bf171a09b745ec963d790f1eefe432" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:38.083425 21469 raft_consensus.cc:3055] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:38.084112 21469 raft_consensus.cc:740] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:38.083842 21325 raft_consensus.cc:3060] T a1bf171a09b745ec963d790f1eefe432 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:38.085105 21469 consensus_queue.cc:260] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:38.094772 21469 raft_consensus.cc:3060] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:38.111897 22360 raft_consensus.cc:993] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea: : Instructing follower ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:38.112471 22360 raft_consensus.cc:1081] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Signalling peer ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:38.117460 22116 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d2b503b6f79842efbb8a7ecd5c5851e0"
dest_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:37026
I20260709 13:22:38.117899 21325 raft_consensus.cc:2468] T a1bf171a09b745ec963d790f1eefe432 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 2.
I20260709 13:22:38.118062 22116 raft_consensus.cc:493] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:38.118535 22116 raft_consensus.cc:3060] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:38.118573 21469 raft_consensus.cc:2468] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 2.
I20260709 13:22:38.119249 22065 leader_election.cc:304] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:38.120069 22369 raft_consensus.cc:2804] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:38.120518 22369 raft_consensus.cc:697] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 [term 2 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:38.121345 22369 consensus_queue.cc:237] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:38.125247 22116 raft_consensus.cc:515] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:38.127388 22116 leader_election.cc:290] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 3 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:38.128484 21469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b503b6f79842efbb8a7ecd5c5851e0" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:38.129091 21469 raft_consensus.cc:3055] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:38.129271 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b503b6f79842efbb8a7ecd5c5851e0" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:38.129396 21469 raft_consensus.cc:740] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:38.129994 21325 raft_consensus.cc:3060] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:38.130527 21469 consensus_queue.cc:260] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:38.131788 21469 raft_consensus.cc:3060] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:38.136375 21325 raft_consensus.cc:2468] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 3.
I20260709 13:22:38.137648 22065 leader_election.cc:304] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:38.138456 22369 raft_consensus.cc:2804] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:38.138417 21469 raft_consensus.cc:2468] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 3.
I20260709 13:22:38.139113 22369 raft_consensus.cc:697] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 3 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:38.139622 21145 catalog_manager.cc:5697] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 1 to 2, leader changed from 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68) to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 2 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:38.140017 22369 consensus_queue.cc:237] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:38.153782 21145 catalog_manager.cc:5697] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 2 to 3, leader changed from 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68) to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 3 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:38.329779 21325 raft_consensus.cc:1275] T a169fb713efc445fb1f0338c7c11203a P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:38.331254 22369 consensus_queue.cc:1048] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:38.344542 21398 raft_consensus.cc:1275] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:38.346084 22372 consensus_queue.cc:1048] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:38.450693 21469 raft_consensus.cc:1275] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:38.451999 22372 consensus_queue.cc:1048] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:38.461637 21398 raft_consensus.cc:1275] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:38.463173 22368 consensus_queue.cc:1048] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:38.531361 21469 raft_consensus.cc:1275] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:38.532416 21467 raft_consensus.cc:1275] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Refusing update from remote peer 5109d6d6d9404ba39c53508a81fe9f1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:38.532831 22368 consensus_queue.cc:1048] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:38.533993 22351 consensus_queue.cc:1048] T cdc82236ff024407ae854c96dd8c89ec P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:38.539945 21324 raft_consensus.cc:1275] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:38.547350 22372 consensus_queue.cc:1048] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:38.554459 21394 raft_consensus.cc:1275] T dcf55c037e6c48bc9d73f87a54b26f92 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:38.556515 22369 consensus_queue.cc:1048] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:38.556728 21394 raft_consensus.cc:1275] T cdc82236ff024407ae854c96dd8c89ec P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Refusing update from remote peer 5109d6d6d9404ba39c53508a81fe9f1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:38.560170 22299 consensus_queue.cc:1048] T cdc82236ff024407ae854c96dd8c89ec P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:38.575325 21473 raft_consensus.cc:1275] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:38.579597 22372 consensus_queue.cc:1048] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:38.616264 21324 raft_consensus.cc:1275] T a1bf171a09b745ec963d790f1eefe432 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:38.617854 22369 consensus_queue.cc:1048] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:38.638340 21467 raft_consensus.cc:1275] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:38.642871 22369 consensus_queue.cc:1048] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:38.729151 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:38.775652 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:38.821424 21467 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05b6856cbfc84013bdfa6979b0366468"
dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
mode: GRACEFUL
new_leader_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:56486
I20260709 13:22:38.822045 21467 raft_consensus.cc:606] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Received request to transfer leadership to TS ec879c0528c44465916efc6c33dde636
I20260709 13:22:38.825038 21467 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ddeabcec9b64fc68b40c70df90b80ef"
dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
mode: GRACEFUL
new_leader_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:56486
I20260709 13:22:38.825466 21467 raft_consensus.cc:606] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Received request to transfer leadership to TS ec879c0528c44465916efc6c33dde636
I20260709 13:22:38.828066 21394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6898b14b98d944278cfd7cb32d964074"
dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
mode: GRACEFUL
new_leader_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:35242
I20260709 13:22:38.828537 21394 raft_consensus.cc:606] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Received request to transfer leadership to TS ec879c0528c44465916efc6c33dde636
I20260709 13:22:38.831298 21473 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8188c780f29f41218f57313c157dae63"
dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
mode: GRACEFUL
new_leader_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:56486
I20260709 13:22:38.832008 21473 raft_consensus.cc:606] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Received request to transfer leadership to TS ec879c0528c44465916efc6c33dde636
I20260709 13:22:38.838478 21467 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4eebc08c8f342a69d7a1248e7df69a1"
dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
mode: GRACEFUL
new_leader_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:56486
I20260709 13:22:38.839399 21467 raft_consensus.cc:606] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Received request to transfer leadership to TS ec879c0528c44465916efc6c33dde636
I20260709 13:22:38.840876 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:22:38.842497 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:38.952680 22353 raft_consensus.cc:993] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea: : Instructing follower ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:38.953163 22402 raft_consensus.cc:1081] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Signalling peer ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:38.954622 22115 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4eebc08c8f342a69d7a1248e7df69a1"
dest_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:37026
I20260709 13:22:38.955091 22115 raft_consensus.cc:493] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:38.955446 22115 raft_consensus.cc:3060] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:38.961162 22115 raft_consensus.cc:515] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:38.962813 22115 leader_election.cc:290] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 3 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:38.964309 21467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4eebc08c8f342a69d7a1248e7df69a1" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:38.964677 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4eebc08c8f342a69d7a1248e7df69a1" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:38.965037 21467 raft_consensus.cc:3055] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:38.965299 21324 raft_consensus.cc:3060] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:38.965426 21467 raft_consensus.cc:740] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:38.970515 21467 consensus_queue.cc:260] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:38.971737 21467 raft_consensus.cc:3060] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:38.980829 21324 raft_consensus.cc:2468] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 3.
I20260709 13:22:38.982056 22065 leader_election.cc:304] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:38.982769 22369 raft_consensus.cc:2804] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:38.982892 21467 raft_consensus.cc:2468] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 3.
I20260709 13:22:38.983297 22369 raft_consensus.cc:697] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 3 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:38.984208 22369 consensus_queue.cc:237] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:38.994626 21147 catalog_manager.cc:5697] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 2 to 3, leader changed from 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68) to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 3 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:39.039562 22353 raft_consensus.cc:993] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea: : Instructing follower ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:39.040069 22402 raft_consensus.cc:1081] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Signalling peer ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:39.041724 22115 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1ddeabcec9b64fc68b40c70df90b80ef"
dest_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:37026
I20260709 13:22:39.042261 22115 raft_consensus.cc:493] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:39.042878 22115 raft_consensus.cc:3060] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:39.047907 22115 raft_consensus.cc:515] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:39.049405 22115 leader_election.cc:290] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 3 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:39.050416 21473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ddeabcec9b64fc68b40c70df90b80ef" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:39.050611 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ddeabcec9b64fc68b40c70df90b80ef" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:39.051098 21473 raft_consensus.cc:3055] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:39.051182 21324 raft_consensus.cc:3060] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:39.051565 21473 raft_consensus.cc:740] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:39.052392 21473 consensus_queue.cc:260] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:39.054046 21473 raft_consensus.cc:3060] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:39.057679 21324 raft_consensus.cc:2468] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 3.
I20260709 13:22:39.058835 22065 leader_election.cc:304] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:39.059680 22369 raft_consensus.cc:2804] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:39.060194 22369 raft_consensus.cc:697] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [term 3 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:39.059878 21473 raft_consensus.cc:2468] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 3.
I20260709 13:22:39.061039 22369 consensus_queue.cc:237] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:39.072678 21147 catalog_manager.cc:5697] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 2 to 3, leader changed from 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68) to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 3 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:39.182282 22365 raft_consensus.cc:993] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375: : Instructing follower ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:39.182761 22367 raft_consensus.cc:1081] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Signalling peer ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:39.184301 22115 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6898b14b98d944278cfd7cb32d964074"
dest_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:37042
I20260709 13:22:39.184759 22115 raft_consensus.cc:493] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:39.185101 22115 raft_consensus.cc:3060] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:39.191344 22115 raft_consensus.cc:515] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:39.193979 21393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6898b14b98d944278cfd7cb32d964074" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:39.194337 22115 leader_election.cc:290] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 2 election: Requested vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:39.194629 21393 raft_consensus.cc:3055] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:39.194962 21393 raft_consensus.cc:740] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:39.195966 21393 consensus_queue.cc:260] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:39.197345 21393 raft_consensus.cc:3060] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:39.198067 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6898b14b98d944278cfd7cb32d964074" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:39.198613 21324 raft_consensus.cc:3060] T 6898b14b98d944278cfd7cb32d964074 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:39.203661 21393 raft_consensus.cc:2468] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 2.
I20260709 13:22:39.203846 21324 raft_consensus.cc:2468] T 6898b14b98d944278cfd7cb32d964074 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 2.
I20260709 13:22:39.204957 22066 leader_election.cc:304] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:39.205892 22369 raft_consensus.cc:2804] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:39.206375 22369 raft_consensus.cc:697] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [term 2 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:39.207100 22369 consensus_queue.cc:237] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:39.216536 21147 catalog_manager.cc:5697] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 1 to 2, leader changed from 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67) to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 2 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:39.242291 22382 raft_consensus.cc:993] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea: : Instructing follower ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:39.242940 22353 raft_consensus.cc:1081] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Signalling peer ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:39.244601 22115 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "05b6856cbfc84013bdfa6979b0366468"
dest_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:37026
I20260709 13:22:39.245117 22115 raft_consensus.cc:493] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:39.245417 22115 raft_consensus.cc:3060] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:39.250025 22115 raft_consensus.cc:515] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:39.251478 22115 leader_election.cc:290] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 3 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:39.252470 21473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b6856cbfc84013bdfa6979b0366468" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:39.252465 21324 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b6856cbfc84013bdfa6979b0366468" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:39.253139 21473 raft_consensus.cc:3055] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:39.253228 21324 raft_consensus.cc:3060] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:39.253603 21473 raft_consensus.cc:740] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:39.254392 21473 consensus_queue.cc:260] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:39.255512 21473 raft_consensus.cc:3060] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:39.258798 21324 raft_consensus.cc:2468] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 3.
I20260709 13:22:39.259907 22065 leader_election.cc:304] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:39.260195 21473 raft_consensus.cc:2468] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 3.
I20260709 13:22:39.260695 22369 raft_consensus.cc:2804] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:39.261163 22369 raft_consensus.cc:697] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [term 3 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:39.261955 22369 consensus_queue.cc:237] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:39.272141 21147 catalog_manager.cc:5697] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 2 to 3, leader changed from 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68) to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 3 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:39.313053 22382 raft_consensus.cc:993] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea: : Instructing follower ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:39.313632 22353 raft_consensus.cc:1081] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Signalling peer ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:39.315168 22116 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8188c780f29f41218f57313c157dae63"
dest_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:37026
I20260709 13:22:39.315692 22116 raft_consensus.cc:493] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:39.316092 22116 raft_consensus.cc:3060] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:39.321317 22116 raft_consensus.cc:515] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:39.322902 22116 leader_election.cc:290] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 2 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:39.323802 21393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8188c780f29f41218f57313c157dae63" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:39.324025 21473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8188c780f29f41218f57313c157dae63" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:39.324476 21393 raft_consensus.cc:3060] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:39.324632 21473 raft_consensus.cc:3055] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:39.325098 21473 raft_consensus.cc:740] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:39.325882 21473 consensus_queue.cc:260] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:39.329967 21473 raft_consensus.cc:3060] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:39.330457 21393 raft_consensus.cc:2468] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 2.
I20260709 13:22:39.331970 22066 leader_election.cc:304] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:39.332731 22369 raft_consensus.cc:2804] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:39.333227 22369 raft_consensus.cc:697] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 [term 2 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:39.334535 22369 consensus_queue.cc:237] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:39.336465 21473 raft_consensus.cc:2468] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 2.
I20260709 13:22:39.349068 21147 catalog_manager.cc:5697] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 1 to 2, leader changed from 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68) to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 2 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:39.400851 21467 raft_consensus.cc:1275] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:39.402189 22372 consensus_queue.cc:1048] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:39.416182 21325 raft_consensus.cc:1275] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:39.417697 22369 consensus_queue.cc:1048] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:39.597424 21325 raft_consensus.cc:1275] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:39.597424 21398 raft_consensus.cc:1275] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:39.599377 22368 consensus_queue.cc:1048] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:39.599434 22369 consensus_queue.cc:1048] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:39.611910 21469 raft_consensus.cc:1275] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:39.619176 22372 consensus_queue.cc:1048] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:39.633525 21325 raft_consensus.cc:1275] T 6898b14b98d944278cfd7cb32d964074 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:39.638568 22368 consensus_queue.cc:1048] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:22:39.698791 21325 raft_consensus.cc:1275] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:39.700124 22368 consensus_queue.cc:1048] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:39.709549 21469 raft_consensus.cc:1275] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:39.711360 22372 consensus_queue.cc:1048] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:39.843346 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:39.887251 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:39.910514 21469 raft_consensus.cc:1275] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:39.911733 22369 consensus_queue.cc:1048] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:39.934424 21398 raft_consensus.cc:1275] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:39.935863 22369 consensus_queue.cc:1048] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:39.961217 21398 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "18a79b2b6d2c4326ab9e4332e0d7406f"
dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
mode: GRACEFUL
new_leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
 from {username='slave'} at 127.0.0.1:35242
I20260709 13:22:39.961848 21398 raft_consensus.cc:606] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Received request to transfer leadership to TS 43cd181efe6b4b2fb79fd3e997a401ea
I20260709 13:22:39.964761 21398 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "799c8e732d4d46e3b26dcd77f458dcbe"
dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
mode: GRACEFUL
new_leader_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
 from {username='slave'} at 127.0.0.1:35242
I20260709 13:22:39.965274 21398 raft_consensus.cc:606] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Received request to transfer leadership to TS 5109d6d6d9404ba39c53508a81fe9f1c
I20260709 13:22:39.968158 21398 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf088eed547f42aea083b31277ecece1"
dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
mode: GRACEFUL
new_leader_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:35242
I20260709 13:22:39.968655 21398 raft_consensus.cc:606] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 LEADER]: Received request to transfer leadership to TS ec879c0528c44465916efc6c33dde636
I20260709 13:22:39.971220 21398 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d31c45d90b684fb18b03156866654665"
dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
mode: GRACEFUL
new_leader_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:35242
I20260709 13:22:39.971676 21398 raft_consensus.cc:606] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Received request to transfer leadership to TS ec879c0528c44465916efc6c33dde636
I20260709 13:22:39.973961 21393 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e532ca31c67547998ba2fb09244573b3"
dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
mode: GRACEFUL
new_leader_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:35242
I20260709 13:22:39.974551 21393 raft_consensus.cc:606] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 LEADER]: Received request to transfer leadership to TS ec879c0528c44465916efc6c33dde636
I20260709 13:22:39.976187 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:22:39.978621 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:40.010720 22367 raft_consensus.cc:993] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375: : Instructing follower ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:40.011338 22412 raft_consensus.cc:1081] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 LEADER]: Signalling peer ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:40.012857 22116 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e532ca31c67547998ba2fb09244573b3"
dest_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:37042
I20260709 13:22:40.013356 22116 raft_consensus.cc:493] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:40.013749 22116 raft_consensus.cc:3060] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:40.019563 22116 raft_consensus.cc:515] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:40.021075 22116 leader_election.cc:290] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 3 election: Requested vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:40.022179 21393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e532ca31c67547998ba2fb09244573b3" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:40.022202 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e532ca31c67547998ba2fb09244573b3" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:40.022792 21393 raft_consensus.cc:3055] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:40.022933 21325 raft_consensus.cc:3060] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:40.023119 21393 raft_consensus.cc:740] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:40.023941 21393 consensus_queue.cc:260] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:40.025008 21393 raft_consensus.cc:3060] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:40.030638 21325 raft_consensus.cc:2468] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 3.
I20260709 13:22:40.030658 21393 raft_consensus.cc:2468] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 3.
I20260709 13:22:40.032546 22066 leader_election.cc:304] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:40.033401 22368 raft_consensus.cc:2804] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:40.033916 22368 raft_consensus.cc:697] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 3 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:40.034796 22368 consensus_queue.cc:237] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:40.048748 21148 catalog_manager.cc:5697] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 2 to 3, leader changed from 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67) to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 3 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:40.090950 22367 raft_consensus.cc:993] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375: : Instructing follower 5109d6d6d9404ba39c53508a81fe9f1c to start an election
I20260709 13:22:40.091485 22412 raft_consensus.cc:1081] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Signalling peer 5109d6d6d9404ba39c53508a81fe9f1c to start an election
I20260709 13:22:40.093032 21325 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "799c8e732d4d46e3b26dcd77f458dcbe"
dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
 from {username='slave'} at 127.0.0.1:50446
I20260709 13:22:40.093489 21325 raft_consensus.cc:493] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:40.093854 21325 raft_consensus.cc:3060] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.098544 21325 raft_consensus.cc:515] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:40.100014 21325 leader_election.cc:290] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:40.100742 21393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "799c8e732d4d46e3b26dcd77f458dcbe" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:40.100984 21469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "799c8e732d4d46e3b26dcd77f458dcbe" candidate_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:40.101431 21393 raft_consensus.cc:3055] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:40.101588 21469 raft_consensus.cc:3060] T 799c8e732d4d46e3b26dcd77f458dcbe P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.101840 21393 raft_consensus.cc:740] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:40.102720 21393 consensus_queue.cc:260] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:40.103740 21393 raft_consensus.cc:3060] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.107007 21469 raft_consensus.cc:2468] T 799c8e732d4d46e3b26dcd77f458dcbe P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 2.
I20260709 13:22:40.108093 21289 leader_election.cc:304] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 
I20260709 13:22:40.108843 22426 raft_consensus.cc:2804] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:40.109318 22426 raft_consensus.cc:697] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 LEADER]: Becoming Leader. State: Replica: 5109d6d6d9404ba39c53508a81fe9f1c, State: Running, Role: LEADER
I20260709 13:22:40.110230 21393 raft_consensus.cc:2468] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5109d6d6d9404ba39c53508a81fe9f1c in term 2.
I20260709 13:22:40.110143 22426 consensus_queue.cc:237] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:40.121364 21148 catalog_manager.cc:5697] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c reported cstate change: term changed from 1 to 2, leader changed from 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67) to 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66). New cstate: current_term: 2 leader_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:40.258437 22367 raft_consensus.cc:993] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375: : Instructing follower 43cd181efe6b4b2fb79fd3e997a401ea to start an election
I20260709 13:22:40.258952 22412 raft_consensus.cc:1081] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Signalling peer 43cd181efe6b4b2fb79fd3e997a401ea to start an election
I20260709 13:22:40.260339 21469 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "18a79b2b6d2c4326ab9e4332e0d7406f"
dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
 from {username='slave'} at 127.0.0.1:48500
I20260709 13:22:40.260829 21469 raft_consensus.cc:493] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:40.261168 21469 raft_consensus.cc:3060] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.266058 21469 raft_consensus.cc:515] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:40.267809 21469 leader_election.cc:290] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Requested vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:40.268890 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a79b2b6d2c4326ab9e4332e0d7406f" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:40.268890 21393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18a79b2b6d2c4326ab9e4332e0d7406f" candidate_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:40.269713 21325 raft_consensus.cc:3060] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.269814 21393 raft_consensus.cc:3055] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:40.270262 21393 raft_consensus.cc:740] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:40.270915 21393 consensus_queue.cc:260] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:40.272121 21393 raft_consensus.cc:3060] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.275430 21325 raft_consensus.cc:2468] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 2.
I20260709 13:22:40.276597 21438 leader_election.cc:304] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, 5109d6d6d9404ba39c53508a81fe9f1c; no voters: 
I20260709 13:22:40.277582 22425 raft_consensus.cc:2804] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:40.278003 22425 raft_consensus.cc:697] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Becoming Leader. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:40.278774 21393 raft_consensus.cc:2468] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cd181efe6b4b2fb79fd3e997a401ea in term 2.
I20260709 13:22:40.278803 22425 consensus_queue.cc:237] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:40.288733 21148 catalog_manager.cc:5697] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea reported cstate change: term changed from 1 to 2, leader changed from 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67) to 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68). New cstate: current_term: 2 leader_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:40.307845 22367 raft_consensus.cc:993] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375: : Instructing follower ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:40.308399 22412 raft_consensus.cc:1081] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Signalling peer ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:40.309996 22115 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d31c45d90b684fb18b03156866654665"
dest_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:37042
I20260709 13:22:40.310539 22115 raft_consensus.cc:493] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:40.310833 22115 raft_consensus.cc:3060] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.316942 22115 raft_consensus.cc:515] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:40.318874 22115 leader_election.cc:290] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 2 election: Requested vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:40.320062 21393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d31c45d90b684fb18b03156866654665" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:40.320600 21325 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d31c45d90b684fb18b03156866654665" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:40.320766 21393 raft_consensus.cc:3055] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:40.321336 21325 raft_consensus.cc:3060] T d31c45d90b684fb18b03156866654665 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.321309 21393 raft_consensus.cc:740] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:40.322650 21393 consensus_queue.cc:260] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:40.323920 21393 raft_consensus.cc:3060] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:40.329316 21325 raft_consensus.cc:2468] T d31c45d90b684fb18b03156866654665 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 2.
I20260709 13:22:40.330735 22065 leader_election.cc:304] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:40.331791 22368 raft_consensus.cc:2804] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:40.332245 22368 raft_consensus.cc:697] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [term 2 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:40.332939 22368 consensus_queue.cc:237] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } }
I20260709 13:22:40.334525 21393 raft_consensus.cc:2468] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 2.
I20260709 13:22:40.342685 21148 catalog_manager.cc:5697] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 1 to 2, leader changed from 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67) to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 2 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:40.478960 22367 raft_consensus.cc:993] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375: : Instructing follower ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:40.481515 22367 raft_consensus.cc:1081] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 LEADER]: Signalling peer ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:40.483276 22116 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cf088eed547f42aea083b31277ecece1"
dest_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:37042
I20260709 13:22:40.483810 22116 raft_consensus.cc:493] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:40.484160 22116 raft_consensus.cc:3060] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:40.491640 22116 raft_consensus.cc:515] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:40.493932 22116 leader_election.cc:290] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 3 election: Requested vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863)
I20260709 13:22:40.494630 21393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf088eed547f42aea083b31277ecece1" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "74e2ff1a68ac4c849ee6397ebb011375"
I20260709 13:22:40.494700 21467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf088eed547f42aea083b31277ecece1" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:40.495302 21393 raft_consensus.cc:3055] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:40.495400 21467 raft_consensus.cc:3060] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:40.495702 21393 raft_consensus.cc:740] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 74e2ff1a68ac4c849ee6397ebb011375, State: Running, Role: LEADER
I20260709 13:22:40.496529 21393 consensus_queue.cc:260] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:40.497526 21393 raft_consensus.cc:3060] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:40.500528 21467 raft_consensus.cc:2468] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 3.
I20260709 13:22:40.501843 22064 leader_election.cc:304] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:40.502591 22368 raft_consensus.cc:2804] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:40.502990 21393 raft_consensus.cc:2468] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 3.
I20260709 13:22:40.503722 22368 raft_consensus.cc:697] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [term 3 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:40.504642 22368 consensus_queue.cc:237] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:40.514518 21324 raft_consensus.cc:1275] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:40.515933 22368 consensus_queue.cc:1048] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:40.516206 21148 catalog_manager.cc:5697] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 2 to 3, leader changed from 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67) to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 3 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:40.535562 21393 raft_consensus.cc:1275] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:40.537639 22372 consensus_queue.cc:1048] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:40.571887 21394 raft_consensus.cc:1275] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Refusing update from remote peer 5109d6d6d9404ba39c53508a81fe9f1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.573125 22426 consensus_queue.cc:1048] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:40.592116 21467 raft_consensus.cc:1275] T 799c8e732d4d46e3b26dcd77f458dcbe P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Refusing update from remote peer 5109d6d6d9404ba39c53508a81fe9f1c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.594580 22396 consensus_queue.cc:1048] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:40.693812 21393 raft_consensus.cc:1275] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Refusing update from remote peer 43cd181efe6b4b2fb79fd3e997a401ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.696916 22436 consensus_queue.cc:1048] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:40.746965 21394 raft_consensus.cc:1275] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.772907 22369 consensus_queue.cc:1048] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:40.790082 21325 raft_consensus.cc:1275] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Refusing update from remote peer 43cd181efe6b4b2fb79fd3e997a401ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.796346 22436 consensus_queue.cc:1048] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:40.817526 21325 raft_consensus.cc:1275] T d31c45d90b684fb18b03156866654665 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:40.916510 22372 consensus_queue.cc:1048] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:40.919041 21469 raft_consensus.cc:1275] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:40.921538 22456 consensus_queue.cc:1048] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:40.983783 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:22:41.123492 21437 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:35963, user_credentials={real_user=slave}} blocked reactor thread for 188400us
I20260709 13:22:41.221647 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:41.275331 21398 raft_consensus.cc:1275] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:41.304188 22455 consensus_queue.cc:1048] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:22:41.404145 21467 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff163c4709bd4437bd70e20440389d62"
dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
mode: GRACEFUL
new_leader_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:56486
I20260709 13:22:41.405493 21467 raft_consensus.cc:606] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Received request to transfer leadership to TS ec879c0528c44465916efc6c33dde636
I20260709 13:22:41.416188 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:22:41.417984 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:41.421499 22465 raft_consensus.cc:993] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea: : Instructing follower ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:41.422361 22467 raft_consensus.cc:1081] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Signalling peer ec879c0528c44465916efc6c33dde636 to start an election
I20260709 13:22:41.425940 22112 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ff163c4709bd4437bd70e20440389d62"
dest_uuid: "ec879c0528c44465916efc6c33dde636"
 from {username='slave'} at 127.0.0.1:37026
I20260709 13:22:41.426581 22112 raft_consensus.cc:493] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:41.426949 22112 raft_consensus.cc:3060] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:41.442040 22112 raft_consensus.cc:515] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:41.447747 21473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff163c4709bd4437bd70e20440389d62" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea"
I20260709 13:22:41.448412 21473 raft_consensus.cc:3055] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:41.448758 21473 raft_consensus.cc:740] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 43cd181efe6b4b2fb79fd3e997a401ea, State: Running, Role: LEADER
I20260709 13:22:41.449586 21473 consensus_queue.cc:260] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:41.450436 21473 raft_consensus.cc:3060] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:41.463480 22112 leader_election.cc:290] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 2 election: Requested vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:41.479001 21473 raft_consensus.cc:2468] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 2.
I20260709 13:22:41.480613 22064 leader_election.cc:304] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636; no voters: 
I20260709 13:22:41.481499 22456 raft_consensus.cc:2804] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:41.482059 22456 raft_consensus.cc:697] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [term 2 LEADER]: Becoming Leader. State: Replica: ec879c0528c44465916efc6c33dde636, State: Running, Role: LEADER
I20260709 13:22:41.483170 22456 consensus_queue.cc:237] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:41.490119 21319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff163c4709bd4437bd70e20440389d62" candidate_uuid: "ec879c0528c44465916efc6c33dde636" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5109d6d6d9404ba39c53508a81fe9f1c"
I20260709 13:22:41.490796 21319 raft_consensus.cc:3060] T ff163c4709bd4437bd70e20440389d62 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:41.509909 21319 raft_consensus.cc:2468] T ff163c4709bd4437bd70e20440389d62 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec879c0528c44465916efc6c33dde636 in term 2.
I20260709 13:22:41.510831 21324 raft_consensus.cc:1240] T ff163c4709bd4437bd70e20440389d62 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Rejecting Update request from peer 43cd181efe6b4b2fb79fd3e997a401ea for earlier term 1. Current term is 2. Ops: []
I20260709 13:22:41.513087 21148 catalog_manager.cc:5697] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 reported cstate change: term changed from 1 to 2, leader changed from 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68) to ec879c0528c44465916efc6c33dde636 (127.11.63.65). New cstate: current_term: 2 leader_uuid: "ec879c0528c44465916efc6c33dde636" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:41.971776 21467 raft_consensus.cc:1275] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:41.973917 22455 consensus_queue.cc:1048] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
W20260709 13:22:41.989113 21180 debug-util.cc:398] Leaking SignalData structure 0x7b080031fe40 after lost signal to thread 11520
W20260709 13:22:41.990535 21180 debug-util.cc:398] Leaking SignalData structure 0x7b08003c0720 after lost signal to thread 21183
W20260709 13:22:41.992003 21180 debug-util.cc:398] Leaking SignalData structure 0x7b0800394dc0 after lost signal to thread 21350
W20260709 13:22:41.993309 21180 debug-util.cc:398] Leaking SignalData structure 0x7b0800162560 after lost signal to thread 21424
W20260709 13:22:41.995108 21180 debug-util.cc:398] Leaking SignalData structure 0x7b08000bb7c0 after lost signal to thread 21498
W20260709 13:22:41.996562 21180 debug-util.cc:398] Leaking SignalData structure 0x7b0800115760 after lost signal to thread 22143
I20260709 13:22:41.996562 21325 raft_consensus.cc:1275] T ff163c4709bd4437bd70e20440389d62 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Refusing update from remote peer ec879c0528c44465916efc6c33dde636: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:41.999500 22368 consensus_queue.cc:1048] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:42.418933 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:42.456828 11517 tablet_server.cc:179] TabletServer@127.11.63.65:35963 shutting down...
W20260709 13:22:42.466360 21363 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 51 similar messages]
W20260709 13:22:42.473157 21363 consensus_peers.cc:597] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:42.519126 21363 consensus_peers.cc:597] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:42.543635 21437 consensus_peers.cc:597] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:42.548333 21437 consensus_peers.cc:597] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:42.561887 21363 consensus_peers.cc:597] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:42.565510 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:42.568979 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.570268 11517 raft_consensus.cc:2243] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.570916 11517 raft_consensus.cc:2272] T d275cc2417214c6f80ff88e7fc3916b7 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.574831 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.575758 11517 raft_consensus.cc:2243] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:42.576857 11517 raft_consensus.cc:2272] T cf088eed547f42aea083b31277ecece1 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.580101 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.581097 11517 raft_consensus.cc:2243] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.581724 11517 raft_consensus.cc:2272] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.585309 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.586609 11517 raft_consensus.cc:2243] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.587256 11517 raft_consensus.cc:2272] T 0413c4b049a8412b9b3b8a36bb5d67fd P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.591079 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.592167 11517 raft_consensus.cc:2243] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.592876 11517 raft_consensus.cc:2272] T 311e1d9a839a418296e3f320abad6c88 P ec879c0528c44465916efc6c33dde636 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.597208 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.598346 11517 raft_consensus.cc:2243] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:42.599699 11517 raft_consensus.cc:2272] T 05b6856cbfc84013bdfa6979b0366468 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.603197 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.604485 11517 raft_consensus.cc:2243] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:22:42.606097 21363 consensus_peers.cc:597] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:42.606105 11517 raft_consensus.cc:2272] T 5c0c8b12f7474d579a8805d344efb092 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.610352 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.611373 11517 raft_consensus.cc:2243] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:42.612655 11517 raft_consensus.cc:2272] T a1bf171a09b745ec963d790f1eefe432 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.616277 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.617367 11517 raft_consensus.cc:2243] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.618008 11517 raft_consensus.cc:2272] T 51766d5a86a6498b8e30249efa0ac990 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:42.619264 21437 consensus_peers.cc:597] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:42.620061 21437 consensus_peers.cc:597] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:42.621929 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.623009 11517 raft_consensus.cc:2243] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.623620 11517 raft_consensus.cc:2272] T 08649134c0c94e7caa55fcc7513eec5d P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.627177 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.628216 11517 raft_consensus.cc:2243] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.628806 11517 raft_consensus.cc:2272] T 49acbb28158c4780add00f91662769da P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.632627 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.633701 11517 raft_consensus.cc:2243] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.634330 11517 raft_consensus.cc:2272] T 200a5b44be554a6bb18a21a29836902e P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.637948 11517 tablet_replica.cc:333] stopping tablet replica
W20260709 13:22:42.638249 21289 consensus_peers.cc:597] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:42.639107 11517 raft_consensus.cc:2243] T 6b569ec3d17d412c8c9087446e64bb77 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.639791 11517 raft_consensus.cc:2272] T 6b569ec3d17d412c8c9087446e64bb77 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:42.640812 21289 consensus_peers.cc:597] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:42.641574 21289 consensus_peers.cc:597] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:42.643841 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.644960 11517 raft_consensus.cc:2243] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.645689 11517 raft_consensus.cc:2272] T aa2f735fa7d14b1e876c094ffbfb2c9d P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.649364 11517 tablet_replica.cc:333] stopping tablet replica
W20260709 13:22:42.649959 21289 consensus_peers.cc:597] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:42.650518 11517 raft_consensus.cc:2243] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:22:42.650719 21289 consensus_peers.cc:597] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:42.651958 11517 raft_consensus.cc:2272] T d31c45d90b684fb18b03156866654665 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:42.652661 21289 consensus_peers.cc:597] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:42.655562 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.656646 11517 raft_consensus.cc:2243] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.657261 11517 raft_consensus.cc:2272] T 99478ce52e24419ea31804ba62f7a46a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:42.659368 21289 consensus_peers.cc:597] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:42.660771 11517 tablet_replica.cc:333] stopping tablet replica
W20260709 13:22:42.661542 21289 consensus_peers.cc:597] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:42.661841 11517 raft_consensus.cc:2243] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.662860 11517 raft_consensus.cc:2272] T 7772ca3f93fe431e8108b4e7ab98786e P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.666271 11517 tablet_replica.cc:333] stopping tablet replica
W20260709 13:22:42.666824 21363 consensus_peers.cc:597] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:42.667371 11517 raft_consensus.cc:2243] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:42.668875 11517 raft_consensus.cc:2272] T 1ddeabcec9b64fc68b40c70df90b80ef P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.672266 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.673221 11517 raft_consensus.cc:2243] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:42.674440 11517 raft_consensus.cc:2272] T d4eebc08c8f342a69d7a1248e7df69a1 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.677662 11517 tablet_replica.cc:333] stopping tablet replica
W20260709 13:22:42.676470 21363 consensus_peers.cc:597] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:42.678579 11517 raft_consensus.cc:2243] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.679216 11517 raft_consensus.cc:2272] T 3449b9c781a847b9ba63813b577eef81 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.687168 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.688416 11517 raft_consensus.cc:2243] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.689486 11517 raft_consensus.cc:2272] T f301fe3cc72b4e439644a7d1617921aa P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.693728 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.694837 11517 raft_consensus.cc:2243] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.695606 11517 raft_consensus.cc:2272] T 5fec8ff4bc5a4d558263e38ae75ece75 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:42.698587 21363 consensus_peers.cc:597] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:42.700292 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.701333 11517 raft_consensus.cc:2243] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:42.702322 11517 raft_consensus.cc:2272] T 6898b14b98d944278cfd7cb32d964074 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:42.704918 21363 consensus_peers.cc:597] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:42.705603 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.706287 11517 raft_consensus.cc:2243] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.706773 11517 raft_consensus.cc:2272] T f4e1cba0e20343aeb61c31c917dd1f50 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.709386 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.710098 11517 raft_consensus.cc:2243] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.710556 11517 raft_consensus.cc:2272] T 6558bd3928cd424fa13c61523335998a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.713544 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.714432 11517 raft_consensus.cc:2243] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:42.715628 11517 raft_consensus.cc:2272] T 8188c780f29f41218f57313c157dae63 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.718809 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.719791 11517 raft_consensus.cc:2243] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:22:42.720227 21437 consensus_peers.cc:597] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:42.720484 11517 raft_consensus.cc:2272] T 1fff6b36d1984d38b6f791e0ac15f56b P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.723721 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.724695 11517 raft_consensus.cc:2243] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.725324 11517 raft_consensus.cc:2272] T 450b7111fe014a51b6e801869913ec52 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.727690 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.728348 11517 raft_consensus.cc:2243] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:42.729297 11517 raft_consensus.cc:2272] T d2b503b6f79842efbb8a7ecd5c5851e0 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.732327 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.733130 11517 raft_consensus.cc:2243] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:42.734356 11517 raft_consensus.cc:2272] T ad13ab5b32d74d80990d10ba376cc2c7 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.737815 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.738549 11517 raft_consensus.cc:2243] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.739070 11517 raft_consensus.cc:2272] T 34c6754d9a8e41ceb1fb416dcbbf0234 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.741849 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.742714 11517 raft_consensus.cc:2243] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.743503 11517 raft_consensus.cc:2272] T 73fe3abd2b2647d384d0e147d1932c0a P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.747430 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.748238 11517 raft_consensus.cc:2243] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.748824 11517 raft_consensus.cc:2272] T bc40f11dce5540d1a056e9bacba075e2 P ec879c0528c44465916efc6c33dde636 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.752190 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.753036 11517 raft_consensus.cc:2243] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.753665 11517 raft_consensus.cc:2272] T e4905b7b4db2444484e9403fed337486 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.756788 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.757604 11517 raft_consensus.cc:2243] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.758173 11517 raft_consensus.cc:2272] T 280cfe3edb3f490c8de46d0c80f6da5a P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.760615 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.761302 11517 raft_consensus.cc:2243] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.761910 11517 raft_consensus.cc:2272] T 01967f3f14be4ecaa1cc2469a50fc156 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.765086 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.765758 11517 raft_consensus.cc:2243] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:42.766541 11517 raft_consensus.cc:2272] T ff163c4709bd4437bd70e20440389d62 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.769688 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.770537 11517 raft_consensus.cc:2243] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:42.771728 11517 raft_consensus.cc:2272] T a169fb713efc445fb1f0338c7c11203a P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.774767 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.775523 11517 raft_consensus.cc:2243] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.776152 11517 raft_consensus.cc:2272] T 228f7ae94b2848ecb12f93717756ba8b P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.779429 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.780184 11517 raft_consensus.cc:2243] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.780706 11517 raft_consensus.cc:2272] T 8f2a6063e86146e8ba5f721d593edf0a P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.784085 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.784917 11517 raft_consensus.cc:2243] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:42.786069 11517 raft_consensus.cc:2272] T e532ca31c67547998ba2fb09244573b3 P ec879c0528c44465916efc6c33dde636 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:42.787694 21437 consensus_peers.cc:597] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:42.789178 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.790141 11517 raft_consensus.cc:2243] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.790814 11517 raft_consensus.cc:2272] T 2d5ddbaefa064931963bc1e27ac99903 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:42.794618 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.795511 11517 raft_consensus.cc:2243] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.796079 11517 raft_consensus.cc:2272] T d37dadd42cd04be5a0e0b339df630a04 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:42.790925 21437 consensus_peers.cc:597] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:42.799461 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.800235 11517 raft_consensus.cc:2243] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:42.801359 11517 raft_consensus.cc:2272] T dcf55c037e6c48bc9d73f87a54b26f92 P ec879c0528c44465916efc6c33dde636 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:42.802687 21289 consensus_peers.cc:597] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:42.804651 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:42.805490 11517 raft_consensus.cc:2243] T 347c76112a0e430ca7d607278f20c811 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:42.806059 11517 raft_consensus.cc:2272] T 347c76112a0e430ca7d607278f20c811 P ec879c0528c44465916efc6c33dde636 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:42.825812 21363 consensus_peers.cc:597] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:42.827591 21437 consensus_peers.cc:597] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:42.830404 21437 consensus_peers.cc:597] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:42.891762 21437 consensus_peers.cc:597] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:42.894018 21437 consensus_peers.cc:597] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:42.897183 11517 tablet_server.cc:196] TabletServer@127.11.63.65:35963 shutdown complete.
W20260709 13:22:42.906883 21437 consensus_peers.cc:597] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:42.914593 21363 consensus_peers.cc:597] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Couldn't send request to peer ec879c0528c44465916efc6c33dde636. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:42.999984 11517 tablet_server.cc:179] TabletServer@127.11.63.66:0 shutting down...
W20260709 13:22:43.049767 21438 consensus_peers.cc:597] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:43.104673 21364 consensus_peers.cc:597] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:43.122844 21438 consensus_peers.cc:597] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:43.132949 21364 consensus_peers.cc:597] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:43.139679 21364 consensus_peers.cc:597] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:43.174695 21438 consensus_peers.cc:597] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:43.176003 21438 consensus_peers.cc:597] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:43.190130 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:22:43.199445 21364 consensus_peers.cc:597] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:43.201375 21364 consensus_peers.cc:597] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:43.202163 11517 tablet_replica.cc:333] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.203657 11517 raft_consensus.cc:2243] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:22:43.205868 21438 consensus_peers.cc:597] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:43.207653 11517 raft_consensus.cc:2272] T aa2f735fa7d14b1e876c094ffbfb2c9d P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.210824 11517 tablet_replica.cc:333] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.212427 11517 raft_consensus.cc:2243] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.213037 11517 raft_consensus.cc:2272] T 05b6856cbfc84013bdfa6979b0366468 P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.221592 11517 tablet_replica.cc:333] T 3f3bf699b02c46c78d330dc7f9d3314b P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.222479 11517 raft_consensus.cc:2243] T 3f3bf699b02c46c78d330dc7f9d3314b P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.223065 11517 raft_consensus.cc:2272] T 3f3bf699b02c46c78d330dc7f9d3314b P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.227536 11517 tablet_replica.cc:333] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.228487 11517 raft_consensus.cc:2243] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.229100 11517 raft_consensus.cc:2272] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:43.237519 21364 consensus_peers.cc:597] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:43.237969 11517 tablet_replica.cc:333] T 08649134c0c94e7caa55fcc7513eec5d P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.242098 11517 raft_consensus.cc:2243] T 08649134c0c94e7caa55fcc7513eec5d P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.242837 11517 raft_consensus.cc:2272] T 08649134c0c94e7caa55fcc7513eec5d P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.247213 11517 tablet_replica.cc:333] T 1fff6b36d1984d38b6f791e0ac15f56b P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.248217 11517 raft_consensus.cc:2243] T 1fff6b36d1984d38b6f791e0ac15f56b P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.248911 11517 raft_consensus.cc:2272] T 1fff6b36d1984d38b6f791e0ac15f56b P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.252008 11517 tablet_replica.cc:333] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.252887 11517 raft_consensus.cc:2243] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.253515 11517 raft_consensus.cc:2272] T 450b7111fe014a51b6e801869913ec52 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:43.254379 21364 consensus_peers.cc:597] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:43.256743 11517 tablet_replica.cc:333] T 033f24b2a37f421cbea89f5c5c61d0a3 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.257601 11517 raft_consensus.cc:2243] T 033f24b2a37f421cbea89f5c5c61d0a3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.258201 11517 raft_consensus.cc:2272] T 033f24b2a37f421cbea89f5c5c61d0a3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.266980 11517 tablet_replica.cc:333] T 6898b14b98d944278cfd7cb32d964074 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.270228 11517 raft_consensus.cc:2243] T 6898b14b98d944278cfd7cb32d964074 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.270814 11517 raft_consensus.cc:2272] T 6898b14b98d944278cfd7cb32d964074 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.273948 11517 tablet_replica.cc:333] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.274734 11517 raft_consensus.cc:2243] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.275720 11517 raft_consensus.cc:2272] T 9805db96a0ab4ac3a686eb8ec904fb24 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.279654 11517 tablet_replica.cc:333] T 37a3c02685464cf1af805559b46c4356 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.280303 11517 raft_consensus.cc:2243] T 37a3c02685464cf1af805559b46c4356 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.280844 11517 raft_consensus.cc:2272] T 37a3c02685464cf1af805559b46c4356 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.283655 11517 tablet_replica.cc:333] T d2bf69be9e9240b5ac99adc49258432e P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.284596 11517 raft_consensus.cc:2243] T d2bf69be9e9240b5ac99adc49258432e P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.285146 11517 raft_consensus.cc:2272] T d2bf69be9e9240b5ac99adc49258432e P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.289184 11517 tablet_replica.cc:333] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.290006 11517 raft_consensus.cc:2243] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:22:43.290114 21364 consensus_peers.cc:597] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:43.290977 11517 raft_consensus.cc:2272] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.294552 11517 tablet_replica.cc:333] T a1bf171a09b745ec963d790f1eefe432 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.295369 11517 raft_consensus.cc:2243] T a1bf171a09b745ec963d790f1eefe432 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.295941 11517 raft_consensus.cc:2272] T a1bf171a09b745ec963d790f1eefe432 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.298170 11517 tablet_replica.cc:333] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.298749 11517 raft_consensus.cc:2243] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.299214 11517 raft_consensus.cc:2272] T d2b503b6f79842efbb8a7ecd5c5851e0 P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.301414 11517 tablet_replica.cc:333] T d31c45d90b684fb18b03156866654665 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.302160 11517 raft_consensus.cc:2243] T d31c45d90b684fb18b03156866654665 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.302654 11517 raft_consensus.cc:2272] T d31c45d90b684fb18b03156866654665 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.305115 11517 tablet_replica.cc:333] T 3449b9c781a847b9ba63813b577eef81 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
W20260709 13:22:43.305615 21438 consensus_peers.cc:597] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:43.305955 11517 raft_consensus.cc:2243] T 3449b9c781a847b9ba63813b577eef81 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.306519 11517 raft_consensus.cc:2272] T 3449b9c781a847b9ba63813b577eef81 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.308691 11517 tablet_replica.cc:333] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.309291 11517 raft_consensus.cc:2243] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.309834 11517 raft_consensus.cc:2272] T 1ddeabcec9b64fc68b40c70df90b80ef P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.311980 11517 tablet_replica.cc:333] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.312580 11517 raft_consensus.cc:2243] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.313380 11517 raft_consensus.cc:2272] T f301fe3cc72b4e439644a7d1617921aa P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.315418 11517 tablet_replica.cc:333] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.315971 11517 raft_consensus.cc:2243] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.316407 11517 raft_consensus.cc:2272] T d4eebc08c8f342a69d7a1248e7df69a1 P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.318500 11517 tablet_replica.cc:333] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.319073 11517 raft_consensus.cc:2243] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.319872 11517 raft_consensus.cc:2272] T 799c8e732d4d46e3b26dcd77f458dcbe P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.322376 11517 tablet_replica.cc:333] T 6b569ec3d17d412c8c9087446e64bb77 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.323098 11517 raft_consensus.cc:2243] T 6b569ec3d17d412c8c9087446e64bb77 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.323652 11517 raft_consensus.cc:2272] T 6b569ec3d17d412c8c9087446e64bb77 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.326331 11517 tablet_replica.cc:333] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
W20260709 13:22:43.326261 21364 consensus_peers.cc:597] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:43.327162 11517 raft_consensus.cc:2243] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.328090 11517 raft_consensus.cc:2272] T 200a5b44be554a6bb18a21a29836902e P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.330266 11517 tablet_replica.cc:333] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.330818 11517 raft_consensus.cc:2243] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.331279 11517 raft_consensus.cc:2272] T 311e1d9a839a418296e3f320abad6c88 P 5109d6d6d9404ba39c53508a81fe9f1c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.333437 11517 tablet_replica.cc:333] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.334005 11517 raft_consensus.cc:2243] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.334810 11517 raft_consensus.cc:2272] T a85e6742ffca4d9c962b8613986c3896 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.336871 11517 tablet_replica.cc:333] T ff163c4709bd4437bd70e20440389d62 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.337415 11517 raft_consensus.cc:2243] T ff163c4709bd4437bd70e20440389d62 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.337939 11517 raft_consensus.cc:2272] T ff163c4709bd4437bd70e20440389d62 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.340519 11517 tablet_replica.cc:333] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.341359 11517 raft_consensus.cc:2243] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.342308 11517 raft_consensus.cc:2272] T 01967f3f14be4ecaa1cc2469a50fc156 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.345759 11517 tablet_replica.cc:333] T d7c665892b494d9db72239758708b947 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.346504 11517 raft_consensus.cc:2243] T d7c665892b494d9db72239758708b947 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.346979 11517 raft_consensus.cc:2272] T d7c665892b494d9db72239758708b947 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.349691 11517 tablet_replica.cc:333] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.350387 11517 raft_consensus.cc:2243] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.351295 11517 raft_consensus.cc:2272] T 280cfe3edb3f490c8de46d0c80f6da5a P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.355368 11517 tablet_replica.cc:333] T e4905b7b4db2444484e9403fed337486 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.356093 11517 raft_consensus.cc:2243] T e4905b7b4db2444484e9403fed337486 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.356629 11517 raft_consensus.cc:2272] T e4905b7b4db2444484e9403fed337486 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.359841 11517 tablet_replica.cc:333] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.360558 11517 raft_consensus.cc:2243] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.361632 11517 raft_consensus.cc:2272] T 5957ec3bcdcf40838d52d01a2a65281f P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.364650 11517 tablet_replica.cc:333] T 665861bf917f4d42aa1252c374a656ea P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.365370 11517 raft_consensus.cc:2243] T 665861bf917f4d42aa1252c374a656ea P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.365927 11517 raft_consensus.cc:2272] T 665861bf917f4d42aa1252c374a656ea P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:43.367549 21438 consensus_peers.cc:597] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:43.368793 11517 tablet_replica.cc:333] T a169fb713efc445fb1f0338c7c11203a P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.369541 11517 raft_consensus.cc:2243] T a169fb713efc445fb1f0338c7c11203a P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.370075 11517 raft_consensus.cc:2272] T a169fb713efc445fb1f0338c7c11203a P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.373106 11517 tablet_replica.cc:333] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
W20260709 13:22:43.373890 21364 consensus_peers.cc:597] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:43.373930 11517 raft_consensus.cc:2243] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.375250 11517 raft_consensus.cc:2272] T 228f7ae94b2848ecb12f93717756ba8b P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.377830 11517 tablet_replica.cc:333] T 8f2a6063e86146e8ba5f721d593edf0a P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.378417 11517 raft_consensus.cc:2243] T 8f2a6063e86146e8ba5f721d593edf0a P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.378866 11517 raft_consensus.cc:2272] T 8f2a6063e86146e8ba5f721d593edf0a P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.381255 11517 tablet_replica.cc:333] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.381877 11517 raft_consensus.cc:2243] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.382339 11517 raft_consensus.cc:2272] T e532ca31c67547998ba2fb09244573b3 P 5109d6d6d9404ba39c53508a81fe9f1c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.384277 11517 tablet_replica.cc:333] T 7a5e0169610b4059877434f37effb058 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.384946 11517 raft_consensus.cc:2243] T 7a5e0169610b4059877434f37effb058 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.385389 11517 raft_consensus.cc:2272] T 7a5e0169610b4059877434f37effb058 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.387977 11517 tablet_replica.cc:333] T 51766d5a86a6498b8e30249efa0ac990 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.388535 11517 raft_consensus.cc:2243] T 51766d5a86a6498b8e30249efa0ac990 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.388963 11517 raft_consensus.cc:2272] T 51766d5a86a6498b8e30249efa0ac990 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.391034 11517 tablet_replica.cc:333] T 5fec8ff4bc5a4d558263e38ae75ece75 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.391542 11517 raft_consensus.cc:2243] T 5fec8ff4bc5a4d558263e38ae75ece75 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.392004 11517 raft_consensus.cc:2272] T 5fec8ff4bc5a4d558263e38ae75ece75 P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.393929 11517 tablet_replica.cc:333] T cdc82236ff024407ae854c96dd8c89ec P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.394398 11517 raft_consensus.cc:2243] T cdc82236ff024407ae854c96dd8c89ec P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.395254 11517 raft_consensus.cc:2272] T cdc82236ff024407ae854c96dd8c89ec P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.397226 11517 tablet_replica.cc:333] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.397821 11517 raft_consensus.cc:2243] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.398612 11517 raft_consensus.cc:2272] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.400568 11517 tablet_replica.cc:333] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.401046 11517 raft_consensus.cc:2243] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.401800 11517 raft_consensus.cc:2272] T 49acbb28158c4780add00f91662769da P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.403762 11517 tablet_replica.cc:333] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.404384 11517 raft_consensus.cc:2243] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.405326 11517 raft_consensus.cc:2272] T d275cc2417214c6f80ff88e7fc3916b7 P 5109d6d6d9404ba39c53508a81fe9f1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.408113 11517 tablet_replica.cc:333] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c: stopping tablet replica
I20260709 13:22:43.408726 11517 raft_consensus.cc:2243] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.409235 11517 raft_consensus.cc:2272] T 18a79b2b6d2c4326ab9e4332e0d7406f P 5109d6d6d9404ba39c53508a81fe9f1c [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:43.435580 21438 consensus_peers.cc:597] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:43.484421 21438 consensus_peers.cc:597] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Couldn't send request to peer 5109d6d6d9404ba39c53508a81fe9f1c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:43.510439 11517 tablet_server.cc:196] TabletServer@127.11.63.66:0 shutdown complete.
I20260709 13:22:43.642514 11517 tablet_server.cc:179] TabletServer@127.11.63.67:0 shutting down...
I20260709 13:22:43.733073 22382 raft_consensus.cc:493] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:43.733631 22382 raft_consensus.cc:515] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:43.735917 22382 leader_election.cc:290] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
W20260709 13:22:43.744030 21437 leader_election.cc:336] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111)
I20260709 13:22:43.746639 22382 raft_consensus.cc:493] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:43.747236 22382 raft_consensus.cc:515] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:43.752512 22492 raft_consensus.cc:493] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:43.753139 22492 raft_consensus.cc:515] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
W20260709 13:22:43.754642 21439 consensus_peers.cc:597] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Couldn't send request to peer 74e2ff1a68ac4c849ee6397ebb011375. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:43.756599 22492 leader_election.cc:290] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:43.752570 22382 leader_election.cc:290] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
W20260709 13:22:43.761694 21437 leader_election.cc:336] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111)
W20260709 13:22:43.763554 21438 leader_election.cc:336] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Network error: Client connection negotiation failed: client connection to 127.11.63.66:38825: connect: Connection refused (error 111)
I20260709 13:22:43.764088 21438 leader_election.cc:304] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:43.767836 22436 raft_consensus.cc:2749] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:22:43.768488 22436 raft_consensus.cc:493] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:43.768924 22436 raft_consensus.cc:515] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:43.771600 22436 leader_election.cc:290] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
W20260709 13:22:43.769331 21437 leader_election.cc:336] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111)
W20260709 13:22:43.777172 21438 leader_election.cc:336] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Network error: Client connection negotiation failed: client connection to 127.11.63.66:38825: connect: Connection refused (error 111)
W20260709 13:22:43.778864 21437 leader_election.cc:336] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111)
W20260709 13:22:43.780251 21439 leader_election.cc:336] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:43.780684 21437 leader_election.cc:304] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:43.780762 21439 leader_election.cc:304] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636
I20260709 13:22:43.781359 22492 raft_consensus.cc:2749] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:22:43.781719 22436 raft_consensus.cc:2749] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:22:43.783253 21439 consensus_peers.cc:597] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Couldn't send request to peer 74e2ff1a68ac4c849ee6397ebb011375. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:43.783828 21439 leader_election.cc:336] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:22:43.784252 21439 leader_election.cc:304] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636
I20260709 13:22:43.784965 22436 raft_consensus.cc:2749] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:22:43.792022 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:43.793056 11517 tablet_replica.cc:333] T 0413c4b049a8412b9b3b8a36bb5d67fd P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.794236 11517 raft_consensus.cc:2243] T 0413c4b049a8412b9b3b8a36bb5d67fd P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.795079 11517 raft_consensus.cc:2272] T 0413c4b049a8412b9b3b8a36bb5d67fd P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.797763 11517 tablet_replica.cc:333] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.798630 11517 raft_consensus.cc:2243] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.799161 11517 raft_consensus.cc:2272] T aa2f735fa7d14b1e876c094ffbfb2c9d P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.801926 11517 tablet_replica.cc:333] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.802666 11517 raft_consensus.cc:2243] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.803449 11517 raft_consensus.cc:2272] T 3449b9c781a847b9ba63813b577eef81 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.805716 11517 tablet_replica.cc:333] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.806682 11517 raft_consensus.cc:2243] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.812525 11517 raft_consensus.cc:2272] T 08649134c0c94e7caa55fcc7513eec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.815513 11517 tablet_replica.cc:333] T 9805db96a0ab4ac3a686eb8ec904fb24 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.816380 11517 raft_consensus.cc:2243] T 9805db96a0ab4ac3a686eb8ec904fb24 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.816941 11517 raft_consensus.cc:2272] T 9805db96a0ab4ac3a686eb8ec904fb24 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.820580 11517 tablet_replica.cc:333] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.821499 11517 raft_consensus.cc:2243] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.822559 11517 raft_consensus.cc:2272] T 73fe3abd2b2647d384d0e147d1932c0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:43.823550 21439 consensus_peers.cc:597] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Couldn't send request to peer 74e2ff1a68ac4c849ee6397ebb011375. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:43.825517 11517 tablet_replica.cc:333] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.826507 11517 raft_consensus.cc:2243] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.827543 11517 raft_consensus.cc:2272] T 033f24b2a37f421cbea89f5c5c61d0a3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:43.827723 21439 consensus_peers.cc:597] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Couldn't send request to peer 74e2ff1a68ac4c849ee6397ebb011375. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:43.830593 11517 tablet_replica.cc:333] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.831581 11517 raft_consensus.cc:2243] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.832240 11517 raft_consensus.cc:2272] T 6898b14b98d944278cfd7cb32d964074 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.835402 11517 tablet_replica.cc:333] T f4e1cba0e20343aeb61c31c917dd1f50 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.836336 11517 raft_consensus.cc:2243] T f4e1cba0e20343aeb61c31c917dd1f50 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.836980 11517 raft_consensus.cc:2272] T f4e1cba0e20343aeb61c31c917dd1f50 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.839938 11517 tablet_replica.cc:333] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.841038 11517 raft_consensus.cc:2243] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.841804 11517 raft_consensus.cc:2272] T bc40f11dce5540d1a056e9bacba075e2 P 74e2ff1a68ac4c849ee6397ebb011375 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.844928 11517 tablet_replica.cc:333] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
W20260709 13:22:43.845784 21439 consensus_peers.cc:597] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Couldn't send request to peer 74e2ff1a68ac4c849ee6397ebb011375. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:43.846007 11517 raft_consensus.cc:2243] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.847401 11517 raft_consensus.cc:2272] T 5c0c8b12f7474d579a8805d344efb092 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.850648 11517 tablet_replica.cc:333] T a85e6742ffca4d9c962b8613986c3896 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.851589 11517 raft_consensus.cc:2243] T a85e6742ffca4d9c962b8613986c3896 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.852177 11517 raft_consensus.cc:2272] T a85e6742ffca4d9c962b8613986c3896 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.855350 11517 tablet_replica.cc:333] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.856243 11517 raft_consensus.cc:2243] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.857334 11517 raft_consensus.cc:2272] T 311e1d9a839a418296e3f320abad6c88 P 74e2ff1a68ac4c849ee6397ebb011375 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.860325 11517 tablet_replica.cc:333] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.861248 11517 raft_consensus.cc:2243] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.861915 11517 raft_consensus.cc:2272] T 799c8e732d4d46e3b26dcd77f458dcbe P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.865331 11517 tablet_replica.cc:333] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.866304 11517 raft_consensus.cc:2243] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.866930 11517 raft_consensus.cc:2272] T cf088eed547f42aea083b31277ecece1 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.869957 11517 tablet_replica.cc:333] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.870913 11517 raft_consensus.cc:2243] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.872088 11517 raft_consensus.cc:2272] T 99478ce52e24419ea31804ba62f7a46a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.875296 11517 tablet_replica.cc:333] T 7772ca3f93fe431e8108b4e7ab98786e P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.875365 22494 raft_consensus.cc:493] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:43.875967 22494 raft_consensus.cc:515] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:43.876325 11517 raft_consensus.cc:2243] T 7772ca3f93fe431e8108b4e7ab98786e P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.877182 11517 raft_consensus.cc:2272] T 7772ca3f93fe431e8108b4e7ab98786e P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.879637 22494 leader_election.cc:290] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:43.880713 11517 tablet_replica.cc:333] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.881772 11517 raft_consensus.cc:2243] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.881927 21473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad13ab5b32d74d80990d10ba376cc2c7" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:43.882437 11517 raft_consensus.cc:2272] T d31c45d90b684fb18b03156866654665 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.885447 11517 tablet_replica.cc:333] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
W20260709 13:22:43.885697 21363 leader_election.cc:336] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111)
I20260709 13:22:43.886150 21363 leader_election.cc:304] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636
I20260709 13:22:43.886438 11517 raft_consensus.cc:2243] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.887696 11517 raft_consensus.cc:2272] T d2bf69be9e9240b5ac99adc49258432e P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.888361 22494 raft_consensus.cc:2749] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:22:43.891173 11517 tablet_replica.cc:333] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.892424 11517 raft_consensus.cc:2243] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.893163 11517 raft_consensus.cc:2272] T 34c6754d9a8e41ceb1fb416dcbbf0234 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.896405 11517 tablet_replica.cc:333] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.897315 11517 raft_consensus.cc:2243] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.897961 11517 raft_consensus.cc:2272] T ad13ab5b32d74d80990d10ba376cc2c7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.898416 22494 raft_consensus.cc:493] T dcf55c037e6c48bc9d73f87a54b26f92 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:43.899039 22494 raft_consensus.cc:515] T dcf55c037e6c48bc9d73f87a54b26f92 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:43.900894 22494 leader_election.cc:290] T dcf55c037e6c48bc9d73f87a54b26f92 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 43cd181efe6b4b2fb79fd3e997a401ea (127.11.63.68:36863), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
I20260709 13:22:43.901048 11517 tablet_replica.cc:333] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.902160 11517 raft_consensus.cc:2243] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.903383 11517 raft_consensus.cc:2272] T 3f3bf699b02c46c78d330dc7f9d3314b P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.904583 21473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcf55c037e6c48bc9d73f87a54b26f92" candidate_uuid: "74e2ff1a68ac4c849ee6397ebb011375" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" is_pre_election: true
I20260709 13:22:43.906596 11517 tablet_replica.cc:333] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
W20260709 13:22:43.907234 21439 consensus_peers.cc:597] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Couldn't send request to peer 74e2ff1a68ac4c849ee6397ebb011375. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:43.907583 11517 raft_consensus.cc:2243] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.908254 11517 raft_consensus.cc:2272] T 8188c780f29f41218f57313c157dae63 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.911499 11517 tablet_replica.cc:333] T 6558bd3928cd424fa13c61523335998a P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.912413 11517 raft_consensus.cc:2243] T 6558bd3928cd424fa13c61523335998a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.913033 11517 raft_consensus.cc:2272] T 6558bd3928cd424fa13c61523335998a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:43.913640 21363 leader_election.cc:336] T dcf55c037e6c48bc9d73f87a54b26f92 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111)
I20260709 13:22:43.914187 21363 leader_election.cc:304] T dcf55c037e6c48bc9d73f87a54b26f92 P 74e2ff1a68ac4c849ee6397ebb011375 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 74e2ff1a68ac4c849ee6397ebb011375; no voters: 43cd181efe6b4b2fb79fd3e997a401ea, ec879c0528c44465916efc6c33dde636
I20260709 13:22:43.915037 22450 raft_consensus.cc:2749] T dcf55c037e6c48bc9d73f87a54b26f92 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:22:43.916102 11517 tablet_replica.cc:333] T cdc82236ff024407ae854c96dd8c89ec P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.917207 11517 raft_consensus.cc:2243] T cdc82236ff024407ae854c96dd8c89ec P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.917923 11517 raft_consensus.cc:2272] T cdc82236ff024407ae854c96dd8c89ec P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.921401 11517 tablet_replica.cc:333] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.922413 11517 raft_consensus.cc:2243] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.923054 11517 raft_consensus.cc:2272] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.926322 11517 tablet_replica.cc:333] T 49acbb28158c4780add00f91662769da P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
W20260709 13:22:43.926949 21439 consensus_peers.cc:597] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Couldn't send request to peer 74e2ff1a68ac4c849ee6397ebb011375. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:43.927227 11517 raft_consensus.cc:2243] T 49acbb28158c4780add00f91662769da P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.927976 11517 raft_consensus.cc:2272] T 49acbb28158c4780add00f91662769da P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.931114 11517 tablet_replica.cc:333] T 5957ec3bcdcf40838d52d01a2a65281f P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.932036 11517 raft_consensus.cc:2243] T 5957ec3bcdcf40838d52d01a2a65281f P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.932657 11517 raft_consensus.cc:2272] T 5957ec3bcdcf40838d52d01a2a65281f P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.935662 11517 tablet_replica.cc:333] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.936548 11517 raft_consensus.cc:2243] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.937674 11517 raft_consensus.cc:2272] T d7c665892b494d9db72239758708b947 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.940846 11517 tablet_replica.cc:333] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
W20260709 13:22:43.940846 21439 consensus_peers.cc:597] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Couldn't send request to peer 74e2ff1a68ac4c849ee6397ebb011375. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:43.941841 11517 raft_consensus.cc:2243] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.942909 11517 raft_consensus.cc:2272] T 51766d5a86a6498b8e30249efa0ac990 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.945936 11517 tablet_replica.cc:333] T 7a5e0169610b4059877434f37effb058 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.946485 22382 raft_consensus.cc:493] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:43.946803 11517 raft_consensus.cc:2243] T 7a5e0169610b4059877434f37effb058 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.947623 11517 raft_consensus.cc:2272] T 7a5e0169610b4059877434f37effb058 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.947449 22382 raft_consensus.cc:515] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:43.950870 11517 tablet_replica.cc:333] T 37a3c02685464cf1af805559b46c4356 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.951822 11517 raft_consensus.cc:2243] T 37a3c02685464cf1af805559b46c4356 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.952394 11517 raft_consensus.cc:2272] T 37a3c02685464cf1af805559b46c4356 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.952749 22382 leader_election.cc:290] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:43.955402 11517 tablet_replica.cc:333] T d275cc2417214c6f80ff88e7fc3916b7 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.956370 11517 raft_consensus.cc:2243] T d275cc2417214c6f80ff88e7fc3916b7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.956970 11517 raft_consensus.cc:2272] T d275cc2417214c6f80ff88e7fc3916b7 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:43.958508 21438 leader_election.cc:336] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Network error: Client connection negotiation failed: client connection to 127.11.63.66:38825: connect: Connection refused (error 111)
I20260709 13:22:43.959925 11517 tablet_replica.cc:333] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.960974 11517 raft_consensus.cc:2243] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:22:43.961903 21437 leader_election.cc:336] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111)
I20260709 13:22:43.962083 11517 raft_consensus.cc:2272] T 2d5ddbaefa064931963bc1e27ac99903 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.962312 21437 leader_election.cc:304] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:43.963130 22382 raft_consensus.cc:2749] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:22:43.965436 11517 tablet_replica.cc:333] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.966419 11517 raft_consensus.cc:2243] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.967049 11517 raft_consensus.cc:2272] T d37dadd42cd04be5a0e0b339df630a04 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.970098 11517 tablet_replica.cc:333] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.970953 11517 raft_consensus.cc:2243] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.971552 11517 raft_consensus.cc:2272] T 18a79b2b6d2c4326ab9e4332e0d7406f P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.974440 11517 tablet_replica.cc:333] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.975337 11517 raft_consensus.cc:2243] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.975967 11517 raft_consensus.cc:2272] T 01967f3f14be4ecaa1cc2469a50fc156 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.979168 11517 tablet_replica.cc:333] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.980053 11517 raft_consensus.cc:2243] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.981046 11517 raft_consensus.cc:2272] T 665861bf917f4d42aa1252c374a656ea P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.984112 11517 tablet_replica.cc:333] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.984969 11517 raft_consensus.cc:2243] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.985630 11517 raft_consensus.cc:2272] T a169fb713efc445fb1f0338c7c11203a P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.988685 11517 tablet_replica.cc:333] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.989516 11517 raft_consensus.cc:2243] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.990167 11517 raft_consensus.cc:2272] T 228f7ae94b2848ecb12f93717756ba8b P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.993116 11517 tablet_replica.cc:333] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.993979 11517 raft_consensus.cc:2243] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:43.995048 11517 raft_consensus.cc:2272] T 8f2a6063e86146e8ba5f721d593edf0a P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:43.998205 11517 tablet_replica.cc:333] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:43.999053 11517 raft_consensus.cc:2243] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:43.999635 11517 raft_consensus.cc:2272] T e532ca31c67547998ba2fb09244573b3 P 74e2ff1a68ac4c849ee6397ebb011375 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.002693 11517 tablet_replica.cc:333] T dcf55c037e6c48bc9d73f87a54b26f92 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:44.003372 11517 raft_consensus.cc:2243] T dcf55c037e6c48bc9d73f87a54b26f92 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.003934 11517 raft_consensus.cc:2272] T dcf55c037e6c48bc9d73f87a54b26f92 P 74e2ff1a68ac4c849ee6397ebb011375 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.006816 11517 tablet_replica.cc:333] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375: stopping tablet replica
I20260709 13:22:44.007512 11517 raft_consensus.cc:2243] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.008646 11517 raft_consensus.cc:2272] T 347c76112a0e430ca7d607278f20c811 P 74e2ff1a68ac4c849ee6397ebb011375 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.026170 22382 raft_consensus.cc:493] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:44.026749 22382 raft_consensus.cc:515] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:44.028584 22382 leader_election.cc:290] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
W20260709 13:22:44.033960 21438 leader_election.cc:336] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Network error: Client connection negotiation failed: client connection to 127.11.63.66:38825: connect: Connection refused (error 111)
W20260709 13:22:44.034759 21437 leader_election.cc:336] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111)
I20260709 13:22:44.035135 21437 leader_election.cc:304] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:44.035907 22382 raft_consensus.cc:2749] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:22:44.065946 22382 raft_consensus.cc:493] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:44.066421 22382 raft_consensus.cc:515] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:44.068037 22382 leader_election.cc:290] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
W20260709 13:22:44.073457 21438 leader_election.cc:336] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Network error: Client connection negotiation failed: client connection to 127.11.63.66:38825: connect: Connection refused (error 111)
W20260709 13:22:44.079777 21437 leader_election.cc:336] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111)
W20260709 13:22:44.079896 21439 consensus_peers.cc:597] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea -> Peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Couldn't send request to peer 74e2ff1a68ac4c849ee6397ebb011375. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:44.080240 21437 leader_election.cc:304] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:44.081393 22382 raft_consensus.cc:2749] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:22:44.091841 22382 raft_consensus.cc:493] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:44.092778 22382 raft_consensus.cc:515] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:44.095798 22382 leader_election.cc:290] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
W20260709 13:22:44.099627 21437 leader_election.cc:336] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111)
W20260709 13:22:44.100294 21438 leader_election.cc:336] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Network error: Client connection negotiation failed: client connection to 127.11.63.66:38825: connect: Connection refused (error 111)
I20260709 13:22:44.100612 21438 leader_election.cc:304] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:44.101218 22382 raft_consensus.cc:2749] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:22:44.110635 11517 tablet_server.cc:196] TabletServer@127.11.63.67:0 shutdown complete.
I20260709 13:22:44.124804 22382 raft_consensus.cc:493] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:44.125372 22382 raft_consensus.cc:515] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:44.127588 22382 leader_election.cc:290] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
W20260709 13:22:44.133936 21439 leader_election.cc:336] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Network error: Client connection negotiation failed: client connection to 127.11.63.67:43181: connect: Connection refused (error 111)
W20260709 13:22:44.134862 21437 leader_election.cc:336] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111)
I20260709 13:22:44.135253 21437 leader_election.cc:304] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636
I20260709 13:22:44.136025 22436 raft_consensus.cc:2749] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:22:44.159050 22382 raft_consensus.cc:493] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ec879c0528c44465916efc6c33dde636)
I20260709 13:22:44.159682 22382 raft_consensus.cc:515] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } }
I20260709 13:22:44.161526 22382 leader_election.cc:290] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181), ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963)
W20260709 13:22:44.166559 21439 leader_election.cc:336] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Network error: Client connection negotiation failed: client connection to 127.11.63.67:43181: connect: Connection refused (error 111)
W20260709 13:22:44.167244 21437 leader_election.cc:336] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111)
I20260709 13:22:44.167603 21437 leader_election.cc:304] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 74e2ff1a68ac4c849ee6397ebb011375, ec879c0528c44465916efc6c33dde636
I20260709 13:22:44.168213 22382 raft_consensus.cc:2749] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:22:44.216728 11517 tablet_server.cc:179] TabletServer@127.11.63.68:0 shutting down...
I20260709 13:22:44.289284 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:44.290386 11517 tablet_replica.cc:333] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.291152 11517 raft_consensus.cc:2243] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.292606 11517 raft_consensus.cc:2272] T 0413c4b049a8412b9b3b8a36bb5d67fd P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.296584 11517 tablet_replica.cc:333] T 665861bf917f4d42aa1252c374a656ea P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.297490 11517 raft_consensus.cc:2243] T 665861bf917f4d42aa1252c374a656ea P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.297552 22492 raft_consensus.cc:493] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5109d6d6d9404ba39c53508a81fe9f1c)
I20260709 13:22:44.298090 11517 raft_consensus.cc:2272] T 665861bf917f4d42aa1252c374a656ea P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.298118 22492 raft_consensus.cc:515] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } } peers { permanent_uuid: "74e2ff1a68ac4c849ee6397ebb011375" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 43181 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } }
I20260709 13:22:44.300424 22492 leader_election.cc:290] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825), 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181)
I20260709 13:22:44.301213 11517 tablet_replica.cc:333] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.302174 11517 raft_consensus.cc:2243] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.303164 11517 raft_consensus.cc:2272] T 7acd3d8dbc204f82b3f0f75ea8b68c57 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.306172 11517 tablet_replica.cc:333] T 9805db96a0ab4ac3a686eb8ec904fb24 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.306986 11517 raft_consensus.cc:2243] T 9805db96a0ab4ac3a686eb8ec904fb24 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.307641 11517 raft_consensus.cc:2272] T 9805db96a0ab4ac3a686eb8ec904fb24 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.310952 11517 tablet_replica.cc:333] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
W20260709 13:22:44.311452 21438 leader_election.cc:336] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Network error: Client connection negotiation failed: client connection to 127.11.63.66:38825: connect: Connection refused (error 111)
I20260709 13:22:44.311849 11517 raft_consensus.cc:2243] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:22:44.312075 21439 leader_election.cc:336] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 74e2ff1a68ac4c849ee6397ebb011375 (127.11.63.67:43181): Network error: Client connection negotiation failed: client connection to 127.11.63.67:43181: connect: Connection refused (error 111)
I20260709 13:22:44.312525 21439 leader_election.cc:304] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, 74e2ff1a68ac4c849ee6397ebb011375
I20260709 13:22:44.312690 11517 raft_consensus.cc:2272] T 73fe3abd2b2647d384d0e147d1932c0a P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.313266 22436 raft_consensus.cc:2749] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:22:44.315914 11517 tablet_replica.cc:333] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.316836 11517 raft_consensus.cc:2243] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.317880 11517 raft_consensus.cc:2272] T 1fff6b36d1984d38b6f791e0ac15f56b P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.320139 11517 tablet_replica.cc:333] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.320757 11517 raft_consensus.cc:2243] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.321631 11517 raft_consensus.cc:2272] T 450b7111fe014a51b6e801869913ec52 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.323638 11517 tablet_replica.cc:333] T 033f24b2a37f421cbea89f5c5c61d0a3 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.324239 11517 raft_consensus.cc:2243] T 033f24b2a37f421cbea89f5c5c61d0a3 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.324779 11517 raft_consensus.cc:2272] T 033f24b2a37f421cbea89f5c5c61d0a3 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.326833 11517 tablet_replica.cc:333] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.327389 11517 raft_consensus.cc:2243] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.328303 11517 raft_consensus.cc:2272] T f4e1cba0e20343aeb61c31c917dd1f50 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.330304 11517 tablet_replica.cc:333] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.330900 11517 raft_consensus.cc:2243] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.331683 11517 raft_consensus.cc:2272] T 7a5e0169610b4059877434f37effb058 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.333698 11517 tablet_replica.cc:333] T d2bf69be9e9240b5ac99adc49258432e P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.334264 11517 raft_consensus.cc:2243] T d2bf69be9e9240b5ac99adc49258432e P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.334728 11517 raft_consensus.cc:2272] T d2bf69be9e9240b5ac99adc49258432e P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.336593 11517 tablet_replica.cc:333] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.337157 11517 raft_consensus.cc:2243] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.337675 11517 raft_consensus.cc:2272] T ad13ab5b32d74d80990d10ba376cc2c7 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.339816 11517 tablet_replica.cc:333] T 799c8e732d4d46e3b26dcd77f458dcbe P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.340378 11517 raft_consensus.cc:2243] T 799c8e732d4d46e3b26dcd77f458dcbe P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.340770 11517 raft_consensus.cc:2272] T 799c8e732d4d46e3b26dcd77f458dcbe P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.342748 11517 tablet_replica.cc:333] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.343309 11517 raft_consensus.cc:2243] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.344103 11517 raft_consensus.cc:2272] T bc40f11dce5540d1a056e9bacba075e2 P 43cd181efe6b4b2fb79fd3e997a401ea [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.346168 11517 tablet_replica.cc:333] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.346740 11517 raft_consensus.cc:2243] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.347189 11517 raft_consensus.cc:2272] T 05b6856cbfc84013bdfa6979b0366468 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.349082 11517 tablet_replica.cc:333] T 5c0c8b12f7474d579a8805d344efb092 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.350034 11517 raft_consensus.cc:2243] T 5c0c8b12f7474d579a8805d344efb092 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.350519 11517 raft_consensus.cc:2272] T 5c0c8b12f7474d579a8805d344efb092 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.352640 11517 tablet_replica.cc:333] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.353240 11517 raft_consensus.cc:2243] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.353775 11517 raft_consensus.cc:2272] T a1bf171a09b745ec963d790f1eefe432 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.356462 11517 tablet_replica.cc:333] T 3f3bf699b02c46c78d330dc7f9d3314b P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.357086 11517 raft_consensus.cc:2243] T 3f3bf699b02c46c78d330dc7f9d3314b P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.357597 11517 raft_consensus.cc:2272] T 3f3bf699b02c46c78d330dc7f9d3314b P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.359889 11517 tablet_replica.cc:333] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.360450 11517 raft_consensus.cc:2243] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.360924 11517 raft_consensus.cc:2272] T 8188c780f29f41218f57313c157dae63 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.363385 11517 tablet_replica.cc:333] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.363948 11517 raft_consensus.cc:2243] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.364769 11517 raft_consensus.cc:2272] T 6b569ec3d17d412c8c9087446e64bb77 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.367472 11517 tablet_replica.cc:333] T 200a5b44be554a6bb18a21a29836902e P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.368304 11517 raft_consensus.cc:2243] T 200a5b44be554a6bb18a21a29836902e P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.368896 11517 raft_consensus.cc:2272] T 200a5b44be554a6bb18a21a29836902e P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.371922 11517 tablet_replica.cc:333] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.372577 11517 raft_consensus.cc:2243] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.373026 11517 raft_consensus.cc:2272] T d2b503b6f79842efbb8a7ecd5c5851e0 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.374619 22436 raft_consensus.cc:493] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5109d6d6d9404ba39c53508a81fe9f1c)
I20260709 13:22:44.375114 22436 raft_consensus.cc:515] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec879c0528c44465916efc6c33dde636" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 35963 } } peers { permanent_uuid: "43cd181efe6b4b2fb79fd3e997a401ea" member_type: VOTER last_known_addr { host: "127.11.63.68" port: 36863 } } peers { permanent_uuid: "5109d6d6d9404ba39c53508a81fe9f1c" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 38825 } }
I20260709 13:22:44.375540 11517 tablet_replica.cc:333] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.376441 11517 raft_consensus.cc:2243] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.376945 11517 raft_consensus.cc:2272] T cf088eed547f42aea083b31277ecece1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.379740 11517 tablet_replica.cc:333] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.380649 22436 leader_election.cc:290] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963), 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825)
I20260709 13:22:44.380673 11517 raft_consensus.cc:2243] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.382001 11517 raft_consensus.cc:2272] T 1ddeabcec9b64fc68b40c70df90b80ef P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:44.382612 21437 leader_election.cc:336] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ec879c0528c44465916efc6c33dde636 (127.11.63.65:35963): Network error: Client connection negotiation failed: client connection to 127.11.63.65:35963: connect: Connection refused (error 111)
W20260709 13:22:44.383322 21438 leader_election.cc:336] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5109d6d6d9404ba39c53508a81fe9f1c (127.11.63.66:38825): Network error: Client connection negotiation failed: client connection to 127.11.63.66:38825: connect: Connection refused (error 111)
I20260709 13:22:44.383725 21438 leader_election.cc:304] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 43cd181efe6b4b2fb79fd3e997a401ea; no voters: 5109d6d6d9404ba39c53508a81fe9f1c, ec879c0528c44465916efc6c33dde636
I20260709 13:22:44.384449 22436 raft_consensus.cc:2749] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:22:44.384888 11517 tablet_replica.cc:333] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.385677 11517 raft_consensus.cc:2243] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.386132 11517 raft_consensus.cc:2272] T f301fe3cc72b4e439644a7d1617921aa P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.388168 11517 tablet_replica.cc:333] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.388727 11517 raft_consensus.cc:2243] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.389207 11517 raft_consensus.cc:2272] T d4eebc08c8f342a69d7a1248e7df69a1 P 43cd181efe6b4b2fb79fd3e997a401ea [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.391330 11517 tablet_replica.cc:333] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.391867 11517 raft_consensus.cc:2243] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.392649 11517 raft_consensus.cc:2272] T 6558bd3928cd424fa13c61523335998a P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.394676 11517 tablet_replica.cc:333] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.395220 11517 raft_consensus.cc:2243] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.395668 11517 raft_consensus.cc:2272] T a85e6742ffca4d9c962b8613986c3896 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.397611 11517 tablet_replica.cc:333] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.398159 11517 raft_consensus.cc:2243] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.398638 11517 raft_consensus.cc:2272] T 280cfe3edb3f490c8de46d0c80f6da5a P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.400702 11517 tablet_replica.cc:333] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.401235 11517 raft_consensus.cc:2243] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.402149 11517 raft_consensus.cc:2272] T e4905b7b4db2444484e9403fed337486 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.403965 11517 tablet_replica.cc:333] T 5957ec3bcdcf40838d52d01a2a65281f P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.404500 11517 raft_consensus.cc:2243] T 5957ec3bcdcf40838d52d01a2a65281f P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.404949 11517 raft_consensus.cc:2272] T 5957ec3bcdcf40838d52d01a2a65281f P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.406926 11517 tablet_replica.cc:333] T d7c665892b494d9db72239758708b947 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.407485 11517 raft_consensus.cc:2243] T d7c665892b494d9db72239758708b947 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.407922 11517 raft_consensus.cc:2272] T d7c665892b494d9db72239758708b947 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.409936 11517 tablet_replica.cc:333] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.410532 11517 raft_consensus.cc:2243] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.410928 11517 raft_consensus.cc:2272] T ff163c4709bd4437bd70e20440389d62 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.412756 11517 tablet_replica.cc:333] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.413264 11517 raft_consensus.cc:2243] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.413698 11517 raft_consensus.cc:2272] T cdc82236ff024407ae854c96dd8c89ec P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.416152 11517 tablet_replica.cc:333] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.416703 11517 raft_consensus.cc:2243] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.417145 11517 raft_consensus.cc:2272] T 97dbdec3b8bc40898dcd64ba19a4ec5d P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.419782 11517 tablet_replica.cc:333] T 99478ce52e24419ea31804ba62f7a46a P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.420526 11517 raft_consensus.cc:2243] T 99478ce52e24419ea31804ba62f7a46a P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.421049 11517 raft_consensus.cc:2272] T 99478ce52e24419ea31804ba62f7a46a P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.423691 11517 tablet_replica.cc:333] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.424429 11517 raft_consensus.cc:2243] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.425197 11517 raft_consensus.cc:2272] T 7772ca3f93fe431e8108b4e7ab98786e P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.427778 11517 tablet_replica.cc:333] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.428543 11517 raft_consensus.cc:2243] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.429448 11517 raft_consensus.cc:2272] T 5fec8ff4bc5a4d558263e38ae75ece75 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.431739 11517 tablet_replica.cc:333] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.432260 11517 raft_consensus.cc:2243] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.432699 11517 raft_consensus.cc:2272] T dcf55c037e6c48bc9d73f87a54b26f92 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.434746 11517 tablet_replica.cc:333] T 347c76112a0e430ca7d607278f20c811 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.435246 11517 raft_consensus.cc:2243] T 347c76112a0e430ca7d607278f20c811 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.435674 11517 raft_consensus.cc:2272] T 347c76112a0e430ca7d607278f20c811 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.437538 11517 tablet_replica.cc:333] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.438090 11517 raft_consensus.cc:2243] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.438903 11517 raft_consensus.cc:2272] T 37a3c02685464cf1af805559b46c4356 P 43cd181efe6b4b2fb79fd3e997a401ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.440815 11517 tablet_replica.cc:333] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.441362 11517 raft_consensus.cc:2243] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:44.441819 11517 raft_consensus.cc:2272] T 2d5ddbaefa064931963bc1e27ac99903 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.443753 11517 tablet_replica.cc:333] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.444227 11517 raft_consensus.cc:2243] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.444986 11517 raft_consensus.cc:2272] T d37dadd42cd04be5a0e0b339df630a04 P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.447162 11517 tablet_replica.cc:333] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea: stopping tablet replica
I20260709 13:22:44.447768 11517 raft_consensus.cc:2243] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.448578 11517 raft_consensus.cc:2272] T 18a79b2b6d2c4326ab9e4332e0d7406f P 43cd181efe6b4b2fb79fd3e997a401ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.523904 11517 tablet_server.cc:196] TabletServer@127.11.63.68:0 shutdown complete.
I20260709 13:22:44.596586 11517 master.cc:562] Master@127.11.63.126:41481 shutting down...
I20260709 13:22:44.624548 11517 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:44.625139 11517 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:44.625485 11517 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3c0b86f1b1314d79b7547279c55b079e: stopping tablet replica
W20260709 13:22:45.644968 11517 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 21180)
W20260709 13:22:46.645409 11517 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 21180)
I20260709 13:22:47.219170 11517 master.cc:584] Master@127.11.63.126:41481 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (34484 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:22:47.310757 11517 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.11.63.126:39291
I20260709 13:22:47.311882 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:47.318686 22499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:47.318936 22498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:47.319588 22501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:47.320474 11517 server_base.cc:1061] running on GCE node
I20260709 13:22:47.321822 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:47.322085 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:47.322258 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603367322238 us; error 0 us; skew 500 ppm
I20260709 13:22:47.322894 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:47.325832 11517 webserver.cc:533] Webserver started at http://127.11.63.126:39707/ using document root <none> and password file <none>
I20260709 13:22:47.326453 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:47.326691 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:47.327019 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:47.328307 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603019296230-11517-0/minicluster-data/master-0-root/instance:
uuid: "97b2d587e4124950b102670d518a0996"
format_stamp: "Formatted at 2026-07-09 13:22:47 on dist-test-slave-qv7r"
I20260709 13:22:47.333714 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:22:47.338045 22506 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:47.338955 11517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.002s
I20260709 13:22:47.339251 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603019296230-11517-0/minicluster-data/master-0-root
uuid: "97b2d587e4124950b102670d518a0996"
format_stamp: "Formatted at 2026-07-09 13:22:47 on dist-test-slave-qv7r"
I20260709 13:22:47.339488 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603019296230-11517-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603019296230-11517-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603019296230-11517-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:47.357733 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:47.358947 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:47.401830 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.126:39291
I20260709 13:22:47.402020 22557 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.126:39291 every 8 connection(s)
I20260709 13:22:47.405884 22558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.417595 22558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996: Bootstrap starting.
I20260709 13:22:47.422695 22558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:47.427279 22558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996: No bootstrap required, opened a new log
I20260709 13:22:47.429499 22558 raft_consensus.cc:359] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97b2d587e4124950b102670d518a0996" member_type: VOTER }
I20260709 13:22:47.429903 22558 raft_consensus.cc:385] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:47.430090 22558 raft_consensus.cc:740] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b2d587e4124950b102670d518a0996, State: Initialized, Role: FOLLOWER
I20260709 13:22:47.430694 22558 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97b2d587e4124950b102670d518a0996" member_type: VOTER }
I20260709 13:22:47.431196 22558 raft_consensus.cc:399] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:22:47.431421 22558 raft_consensus.cc:493] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:22:47.431653 22558 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:47.436609 22558 raft_consensus.cc:515] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97b2d587e4124950b102670d518a0996" member_type: VOTER }
I20260709 13:22:47.437093 22558 leader_election.cc:304] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97b2d587e4124950b102670d518a0996; no voters: 
I20260709 13:22:47.438243 22558 leader_election.cc:290] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:22:47.438714 22561 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:47.440196 22561 raft_consensus.cc:697] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996 [term 1 LEADER]: Becoming Leader. State: Replica: 97b2d587e4124950b102670d518a0996, State: Running, Role: LEADER
I20260709 13:22:47.440802 22561 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97b2d587e4124950b102670d518a0996" member_type: VOTER }
I20260709 13:22:47.441601 22558 sys_catalog.cc:565] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:22:47.445338 22562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97b2d587e4124950b102670d518a0996" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97b2d587e4124950b102670d518a0996" member_type: VOTER } }
I20260709 13:22:47.445549 22563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97b2d587e4124950b102670d518a0996. Latest consensus state: current_term: 1 leader_uuid: "97b2d587e4124950b102670d518a0996" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97b2d587e4124950b102670d518a0996" member_type: VOTER } }
I20260709 13:22:47.446273 22562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996 [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:47.446349 22563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996 [sys.catalog]: This master's current role is: LEADER
I20260709 13:22:47.454497 22567 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:22:47.459544 22567 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:22:47.460765 11517 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:22:47.469763 22567 catalog_manager.cc:1383] Generated new cluster ID: 3900fa2ccf5a42bbb4b77dfc208df488
I20260709 13:22:47.470088 22567 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:22:47.487480 22567 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:22:47.488973 22567 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:22:47.503216 22567 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996: Generated new TSK 0
I20260709 13:22:47.503916 22567 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:22:47.527664 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:47.534597 22579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:47.535655 22580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:47.538244 11517 server_base.cc:1061] running on GCE node
W20260709 13:22:47.538697 22582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:47.539719 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:47.539997 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:47.540156 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603367540144 us; error 0 us; skew 500 ppm
I20260709 13:22:47.540819 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:47.543614 11517 webserver.cc:533] Webserver started at http://127.11.63.65:42533/ using document root <none> and password file <none>
I20260709 13:22:47.544148 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:47.544338 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:47.544638 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:47.545888 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603019296230-11517-0/minicluster-data/ts-0-root/instance:
uuid: "55427c197ec4424580678498fbc616a2"
format_stamp: "Formatted at 2026-07-09 13:22:47 on dist-test-slave-qv7r"
I20260709 13:22:47.550638 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:22:47.554555 22587 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:47.555645 11517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:22:47.556010 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603019296230-11517-0/minicluster-data/ts-0-root
uuid: "55427c197ec4424580678498fbc616a2"
format_stamp: "Formatted at 2026-07-09 13:22:47 on dist-test-slave-qv7r"
I20260709 13:22:47.556293 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603019296230-11517-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603019296230-11517-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603019296230-11517-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:47.574257 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:47.575582 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:47.577209 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:47.579741 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:47.579974 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:47.580230 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:47.580425 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:47.625000 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.65:36635
I20260709 13:22:47.625082 22649 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.65:36635 every 8 connection(s)
I20260709 13:22:47.630647 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:47.638628 22654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:47.640407 22655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:47.642262 11517 server_base.cc:1061] running on GCE node
W20260709 13:22:47.643738 22657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:47.644822 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:47.645087 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:47.645283 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603367645259 us; error 0 us; skew 500 ppm
I20260709 13:22:47.646003 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:47.648813 11517 webserver.cc:533] Webserver started at http://127.11.63.66:36709/ using document root <none> and password file <none>
I20260709 13:22:47.649403 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:47.649650 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:47.649752 22650 heartbeater.cc:344] Connected to a master server at 127.11.63.126:39291
I20260709 13:22:47.649935 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:47.650362 22650 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:47.651162 22650 heartbeater.cc:507] Master 127.11.63.126:39291 requested a full tablet report, sending...
I20260709 13:22:47.651257 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603019296230-11517-0/minicluster-data/ts-1-root/instance:
uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
format_stamp: "Formatted at 2026-07-09 13:22:47 on dist-test-slave-qv7r"
I20260709 13:22:47.653484 22523 ts_manager.cc:194] Registered new tserver with Master: 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635)
I20260709 13:22:47.656090 22523 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41320
I20260709 13:22:47.657531 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:22:47.661363 22662 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:47.662484 11517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:22:47.662843 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603019296230-11517-0/minicluster-data/ts-1-root
uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
format_stamp: "Formatted at 2026-07-09 13:22:47 on dist-test-slave-qv7r"
I20260709 13:22:47.663146 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603019296230-11517-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603019296230-11517-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603019296230-11517-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:47.677111 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:47.678550 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:47.680192 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:47.682888 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:47.683162 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:47.683444 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:47.683640 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:47.730592 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.66:37617
I20260709 13:22:47.730708 22724 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.66:37617 every 8 connection(s)
I20260709 13:22:47.735993 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:47.744580 22728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:47.746990 22729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:47.747128 22731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:47.748044 11517 server_base.cc:1061] running on GCE node
I20260709 13:22:47.749145 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:47.749396 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:47.749787 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603367749764 us; error 0 us; skew 500 ppm
I20260709 13:22:47.750391 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:47.750970 22725 heartbeater.cc:344] Connected to a master server at 127.11.63.126:39291
I20260709 13:22:47.751346 22725 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:47.752071 22725 heartbeater.cc:507] Master 127.11.63.126:39291 requested a full tablet report, sending...
I20260709 13:22:47.753621 11517 webserver.cc:533] Webserver started at http://127.11.63.67:45615/ using document root <none> and password file <none>
I20260709 13:22:47.754233 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:47.754441 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:22:47.754295 22523 ts_manager.cc:194] Registered new tserver with Master: f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617)
I20260709 13:22:47.754973 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:22:47.756225 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603019296230-11517-0/minicluster-data/ts-2-root/instance:
uuid: "c79ef076f43f4a73884174a86a6a10c1"
format_stamp: "Formatted at 2026-07-09 13:22:47 on dist-test-slave-qv7r"
I20260709 13:22:47.756186 22523 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41332
I20260709 13:22:47.761267 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:22:47.765367 22736 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:47.766388 11517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:22:47.766768 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603019296230-11517-0/minicluster-data/ts-2-root
uuid: "c79ef076f43f4a73884174a86a6a10c1"
format_stamp: "Formatted at 2026-07-09 13:22:47 on dist-test-slave-qv7r"
I20260709 13:22:47.767055 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603019296230-11517-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603019296230-11517-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603019296230-11517-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:47.791627 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:47.793008 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:47.794713 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:47.797380 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:22:47.797674 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:22:47.797976 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:22:47.798194 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:47.847419 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.67:34633
I20260709 13:22:47.847618 22798 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.67:34633 every 8 connection(s)
I20260709 13:22:47.862666 22799 heartbeater.cc:344] Connected to a master server at 127.11.63.126:39291
I20260709 13:22:47.863129 22799 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:47.864136 22799 heartbeater.cc:507] Master 127.11.63.126:39291 requested a full tablet report, sending...
I20260709 13:22:47.866447 22523 ts_manager.cc:194] Registered new tserver with Master: c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:47.866801 11517 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016012374s
I20260709 13:22:47.868490 22523 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41342
I20260709 13:22:47.896623 22523 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41346:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:22:47.899657 22523 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:22:47.978307 22764 tablet_service.cc:1511] Processing CreateTablet for tablet 4a44aff4c84e4f05a00639dc531ecca5 (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:22:47.979807 22764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a44aff4c84e4f05a00639dc531ecca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.982457 22763 tablet_service.cc:1511] Processing CreateTablet for tablet 11ccb054ff314be7a47b158bf4d141b6 (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:22:47.984002 22763 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11ccb054ff314be7a47b158bf4d141b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.986668 22762 tablet_service.cc:1511] Processing CreateTablet for tablet 2214925267994c84b73383ae827170d5 (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:22:47.988118 22762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2214925267994c84b73383ae827170d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.992161 22761 tablet_service.cc:1511] Processing CreateTablet for tablet 8e8e668ce13b473186eed6b7c92dbf8c (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:22:47.993654 22761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e8e668ce13b473186eed6b7c92dbf8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.993310 22760 tablet_service.cc:1511] Processing CreateTablet for tablet add99f7fe29440ab82c13df2e1bc541d (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:22:47.994868 22760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet add99f7fe29440ab82c13df2e1bc541d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:47.998345 22759 tablet_service.cc:1511] Processing CreateTablet for tablet 28c206f085d9400c8978877b15ad9058 (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:22:47.999804 22759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28c206f085d9400c8978877b15ad9058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.004341 22758 tablet_service.cc:1511] Processing CreateTablet for tablet a93e444ff9ec4a578e573e970e281e0a (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:22:48.005924 22758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a93e444ff9ec4a578e573e970e281e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.020037 22614 tablet_service.cc:1511] Processing CreateTablet for tablet 11ccb054ff314be7a47b158bf4d141b6 (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:22:48.022979 22615 tablet_service.cc:1511] Processing CreateTablet for tablet 4a44aff4c84e4f05a00639dc531ecca5 (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:22:48.023587 22613 tablet_service.cc:1511] Processing CreateTablet for tablet 2214925267994c84b73383ae827170d5 (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:22:48.025417 22612 tablet_service.cc:1511] Processing CreateTablet for tablet 8e8e668ce13b473186eed6b7c92dbf8c (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:22:48.027388 22611 tablet_service.cc:1511] Processing CreateTablet for tablet add99f7fe29440ab82c13df2e1bc541d (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:22:48.027691 22608 tablet_service.cc:1511] Processing CreateTablet for tablet 5cbcb0fb6f4e46b991774ca89f45da59 (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:22:48.029246 22610 tablet_service.cc:1511] Processing CreateTablet for tablet 28c206f085d9400c8978877b15ad9058 (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:22:48.031322 22609 tablet_service.cc:1511] Processing CreateTablet for tablet a93e444ff9ec4a578e573e970e281e0a (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:22:48.032784 22757 tablet_service.cc:1511] Processing CreateTablet for tablet 5cbcb0fb6f4e46b991774ca89f45da59 (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:22:48.034201 22757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5cbcb0fb6f4e46b991774ca89f45da59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.032836 22614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11ccb054ff314be7a47b158bf4d141b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.041990 22608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5cbcb0fb6f4e46b991774ca89f45da59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.045974 22609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a93e444ff9ec4a578e573e970e281e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.047322 22607 tablet_service.cc:1511] Processing CreateTablet for tablet b2d10d445a81457f817ac6b4bfcb77c7 (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:22:48.048761 22607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2d10d445a81457f817ac6b4bfcb77c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.049379 22613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2214925267994c84b73383ae827170d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.050379 22689 tablet_service.cc:1511] Processing CreateTablet for tablet 11ccb054ff314be7a47b158bf4d141b6 (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:22:48.050779 22690 tablet_service.cc:1511] Processing CreateTablet for tablet 4a44aff4c84e4f05a00639dc531ecca5 (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:22:48.051836 22689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11ccb054ff314be7a47b158bf4d141b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.052783 22684 tablet_service.cc:1511] Processing CreateTablet for tablet a93e444ff9ec4a578e573e970e281e0a (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:22:48.054293 22690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a44aff4c84e4f05a00639dc531ecca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.054800 22687 tablet_service.cc:1511] Processing CreateTablet for tablet 8e8e668ce13b473186eed6b7c92dbf8c (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:22:48.053179 22610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28c206f085d9400c8978877b15ad9058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.055804 22684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a93e444ff9ec4a578e573e970e281e0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.056671 22612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e8e668ce13b473186eed6b7c92dbf8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.055368 22683 tablet_service.cc:1511] Processing CreateTablet for tablet 5cbcb0fb6f4e46b991774ca89f45da59 (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:22:48.057144 22686 tablet_service.cc:1511] Processing CreateTablet for tablet add99f7fe29440ab82c13df2e1bc541d (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:22:48.057998 22683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5cbcb0fb6f4e46b991774ca89f45da59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.045325 22814 tablet_bootstrap.cc:492] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1: Bootstrap starting.
I20260709 13:22:48.058228 22611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet add99f7fe29440ab82c13df2e1bc541d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.059545 22615 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a44aff4c84e4f05a00639dc531ecca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.059620 22685 tablet_service.cc:1511] Processing CreateTablet for tablet 28c206f085d9400c8978877b15ad9058 (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:22:48.062321 22688 tablet_service.cc:1511] Processing CreateTablet for tablet 2214925267994c84b73383ae827170d5 (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:22:48.061467 22685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28c206f085d9400c8978877b15ad9058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.064489 22688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2214925267994c84b73383ae827170d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.065106 22686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet add99f7fe29440ab82c13df2e1bc541d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.067660 22687 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e8e668ce13b473186eed6b7c92dbf8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.077350 22814 tablet_bootstrap.cc:654] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.098538 22682 tablet_service.cc:1511] Processing CreateTablet for tablet b2d10d445a81457f817ac6b4bfcb77c7 (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:22:48.100165 22682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2d10d445a81457f817ac6b4bfcb77c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.101047 22762 tablet_service.cc:1511] Processing CreateTablet for tablet b2d10d445a81457f817ac6b4bfcb77c7 (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:22:48.102425 22762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2d10d445a81457f817ac6b4bfcb77c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.109705 22606 tablet_service.cc:1511] Processing CreateTablet for tablet 7505614384fc47b19d23973ba804ea5a (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:22:48.111239 22606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7505614384fc47b19d23973ba804ea5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.121963 22681 tablet_service.cc:1511] Processing CreateTablet for tablet 7505614384fc47b19d23973ba804ea5a (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:22:48.120168 22814 tablet_bootstrap.cc:492] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1: No bootstrap required, opened a new log
I20260709 13:22:48.123679 22681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7505614384fc47b19d23973ba804ea5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.123817 22814 ts_tablet_manager.cc:1403] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1: Time spent bootstrapping tablet: real 0.079s	user 0.011s	sys 0.003s
I20260709 13:22:48.138262 22815 tablet_bootstrap.cc:492] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69: Bootstrap starting.
I20260709 13:22:48.141206 22764 tablet_service.cc:1511] Processing CreateTablet for tablet 7505614384fc47b19d23973ba804ea5a (DEFAULT_TABLE table=test-workload [id=6b39b074f8764d6ea176e2652d3c44a2]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:22:48.142567 22764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7505614384fc47b19d23973ba804ea5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:22:48.157681 22817 tablet_bootstrap.cc:492] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2: Bootstrap starting.
I20260709 13:22:48.163729 22814 raft_consensus.cc:359] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } }
I20260709 13:22:48.164556 22814 raft_consensus.cc:385] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.164898 22814 raft_consensus.cc:740] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.165627 22814 consensus_queue.cc:260] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } }
I20260709 13:22:48.205752 22817 tablet_bootstrap.cc:654] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.215162 22799 heartbeater.cc:499] Master 127.11.63.126:39291 was elected leader, sending a full tablet report...
I20260709 13:22:48.215795 22814 ts_tablet_manager.cc:1434] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1: Time spent starting tablet: real 0.092s	user 0.030s	sys 0.018s
I20260709 13:22:48.216666 22815 tablet_bootstrap.cc:654] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.216876 22814 tablet_bootstrap.cc:492] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1: Bootstrap starting.
I20260709 13:22:48.223584 22814 tablet_bootstrap.cc:654] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.230756 22817 tablet_bootstrap.cc:492] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2: No bootstrap required, opened a new log
I20260709 13:22:48.231451 22817 ts_tablet_manager.cc:1403] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2: Time spent bootstrapping tablet: real 0.074s	user 0.008s	sys 0.008s
I20260709 13:22:48.235309 22814 tablet_bootstrap.cc:492] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1: No bootstrap required, opened a new log
I20260709 13:22:48.236027 22814 ts_tablet_manager.cc:1403] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.005s
I20260709 13:22:48.235297 22817 raft_consensus.cc:359] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.236662 22817 raft_consensus.cc:385] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.237112 22817 raft_consensus.cc:740] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55427c197ec4424580678498fbc616a2, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.238242 22817 consensus_queue.cc:260] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.240974 22815 tablet_bootstrap.cc:492] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69: No bootstrap required, opened a new log
I20260709 13:22:48.243161 22650 heartbeater.cc:499] Master 127.11.63.126:39291 was elected leader, sending a full tablet report...
I20260709 13:22:48.243800 22817 ts_tablet_manager.cc:1434] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.004s
I20260709 13:22:48.243420 22814 raft_consensus.cc:359] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.244230 22814 raft_consensus.cc:385] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.244535 22814 raft_consensus.cc:740] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.244880 22817 tablet_bootstrap.cc:492] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2: Bootstrap starting.
I20260709 13:22:48.245404 22814 consensus_queue.cc:260] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.247970 22814 ts_tablet_manager.cc:1434] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:22:48.249046 22814 tablet_bootstrap.cc:492] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1: Bootstrap starting.
I20260709 13:22:48.251497 22817 tablet_bootstrap.cc:654] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.251940 22815 ts_tablet_manager.cc:1403] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69: Time spent bootstrapping tablet: real 0.114s	user 0.011s	sys 0.022s
I20260709 13:22:48.255765 22815 raft_consensus.cc:359] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.256520 22814 tablet_bootstrap.cc:654] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.256552 22815 raft_consensus.cc:385] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.258747 22815 raft_consensus.cc:740] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.259745 22815 consensus_queue.cc:260] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.262310 22725 heartbeater.cc:499] Master 127.11.63.126:39291 was elected leader, sending a full tablet report...
I20260709 13:22:48.265358 22815 ts_tablet_manager.cc:1434] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 13:22:48.266673 22815 tablet_bootstrap.cc:492] T 28c206f085d9400c8978877b15ad9058 P f7e755f96eda4b9ba37aa77b183f2a69: Bootstrap starting.
I20260709 13:22:48.273615 22815 tablet_bootstrap.cc:654] T 28c206f085d9400c8978877b15ad9058 P f7e755f96eda4b9ba37aa77b183f2a69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.273931 22814 tablet_bootstrap.cc:492] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1: No bootstrap required, opened a new log
I20260709 13:22:48.274482 22814 ts_tablet_manager.cc:1403] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.006s
I20260709 13:22:48.277457 22814 raft_consensus.cc:359] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.278216 22814 raft_consensus.cc:385] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.278509 22814 raft_consensus.cc:740] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.279556 22814 consensus_queue.cc:260] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.281134 22817 tablet_bootstrap.cc:492] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2: No bootstrap required, opened a new log
I20260709 13:22:48.281217 22815 tablet_bootstrap.cc:492] T 28c206f085d9400c8978877b15ad9058 P f7e755f96eda4b9ba37aa77b183f2a69: No bootstrap required, opened a new log
I20260709 13:22:48.281813 22817 ts_tablet_manager.cc:1403] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2: Time spent bootstrapping tablet: real 0.037s	user 0.017s	sys 0.007s
I20260709 13:22:48.281925 22815 ts_tablet_manager.cc:1403] T 28c206f085d9400c8978877b15ad9058 P f7e755f96eda4b9ba37aa77b183f2a69: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.002s
I20260709 13:22:48.285231 22815 raft_consensus.cc:359] T 28c206f085d9400c8978877b15ad9058 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.286068 22815 raft_consensus.cc:385] T 28c206f085d9400c8978877b15ad9058 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.286401 22815 raft_consensus.cc:740] T 28c206f085d9400c8978877b15ad9058 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.284973 22817 raft_consensus.cc:359] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } }
I20260709 13:22:48.287000 22817 raft_consensus.cc:385] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.287405 22817 raft_consensus.cc:740] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55427c197ec4424580678498fbc616a2, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.287354 22815 consensus_queue.cc:260] T 28c206f085d9400c8978877b15ad9058 P f7e755f96eda4b9ba37aa77b183f2a69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.288105 22814 ts_tablet_manager.cc:1434] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:22:48.288589 22817 consensus_queue.cc:260] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } }
I20260709 13:22:48.289695 22814 tablet_bootstrap.cc:492] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1: Bootstrap starting.
I20260709 13:22:48.289868 22815 ts_tablet_manager.cc:1434] T 28c206f085d9400c8978877b15ad9058 P f7e755f96eda4b9ba37aa77b183f2a69: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:22:48.291016 22815 tablet_bootstrap.cc:492] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69: Bootstrap starting.
I20260709 13:22:48.296222 22817 ts_tablet_manager.cc:1434] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:22:48.296533 22814 tablet_bootstrap.cc:654] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.297580 22815 tablet_bootstrap.cc:654] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.300102 22822 raft_consensus.cc:493] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.300772 22822 raft_consensus.cc:515] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } }
I20260709 13:22:48.301380 22817 tablet_bootstrap.cc:492] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2: Bootstrap starting.
I20260709 13:22:48.308516 22817 tablet_bootstrap.cc:654] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.315374 22822 leader_election.cc:290] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633), f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617)
I20260709 13:22:48.323508 22814 tablet_bootstrap.cc:492] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1: No bootstrap required, opened a new log
I20260709 13:22:48.324427 22814 ts_tablet_manager.cc:1403] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1: Time spent bootstrapping tablet: real 0.035s	user 0.018s	sys 0.011s
I20260709 13:22:48.325807 22815 tablet_bootstrap.cc:492] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69: No bootstrap required, opened a new log
I20260709 13:22:48.326344 22815 ts_tablet_manager.cc:1403] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.008s
I20260709 13:22:48.328800 22814 raft_consensus.cc:359] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.330054 22814 raft_consensus.cc:385] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.330463 22814 raft_consensus.cc:740] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.331374 22814 consensus_queue.cc:260] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.332931 22817 tablet_bootstrap.cc:492] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2: No bootstrap required, opened a new log
I20260709 13:22:48.329245 22815 raft_consensus.cc:359] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.333695 22817 ts_tablet_manager.cc:1403] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2: Time spent bootstrapping tablet: real 0.033s	user 0.013s	sys 0.014s
I20260709 13:22:48.333914 22815 raft_consensus.cc:385] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.334391 22815 raft_consensus.cc:740] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.335299 22815 consensus_queue.cc:260] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.336704 22817 raft_consensus.cc:359] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.337544 22817 raft_consensus.cc:385] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.337853 22815 ts_tablet_manager.cc:1434] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:22:48.337922 22817 raft_consensus.cc:740] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55427c197ec4424580678498fbc616a2, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.339135 22815 tablet_bootstrap.cc:492] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69: Bootstrap starting.
I20260709 13:22:48.339453 22700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a44aff4c84e4f05a00639dc531ecca5" candidate_uuid: "55427c197ec4424580678498fbc616a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" is_pre_election: true
I20260709 13:22:48.339178 22817 consensus_queue.cc:260] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
W20260709 13:22:48.341246 22588 leader_election.cc:343] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:48.343528 22774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a44aff4c84e4f05a00639dc531ecca5" candidate_uuid: "55427c197ec4424580678498fbc616a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1" is_pre_election: true
I20260709 13:22:48.344362 22774 raft_consensus.cc:2468] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55427c197ec4424580678498fbc616a2 in term 0.
I20260709 13:22:48.345719 22818 raft_consensus.cc:493] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.346155 22815 tablet_bootstrap.cc:654] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.346230 22818 raft_consensus.cc:515] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.347543 22591 leader_election.cc:304] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 55427c197ec4424580678498fbc616a2, c79ef076f43f4a73884174a86a6a10c1; no voters: f7e755f96eda4b9ba37aa77b183f2a69
I20260709 13:22:48.348470 22822 raft_consensus.cc:2804] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:48.348840 22822 raft_consensus.cc:493] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:48.349177 22822 raft_consensus.cc:3060] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.346174 22817 ts_tablet_manager.cc:1434] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 13:22:48.333707 22814 ts_tablet_manager.cc:1434] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:22:48.351025 22817 tablet_bootstrap.cc:492] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2: Bootstrap starting.
W20260709 13:22:48.353515 22800 tablet.cc:2406] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:22:48.354681 22814 tablet_bootstrap.cc:492] T 2214925267994c84b73383ae827170d5 P c79ef076f43f4a73884174a86a6a10c1: Bootstrap starting.
I20260709 13:22:48.365514 22818 leader_election.cc:290] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617)
I20260709 13:22:48.368443 22625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ccb054ff314be7a47b158bf4d141b6" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55427c197ec4424580678498fbc616a2" is_pre_election: true
W20260709 13:22:48.370489 22739 leader_election.cc:343] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:48.371265 22818 raft_consensus.cc:493] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.371793 22818 raft_consensus.cc:515] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.374786 22822 raft_consensus.cc:515] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } }
I20260709 13:22:48.379735 22822 leader_election.cc:290] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 election: Requested vote from peers c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633), f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617)
I20260709 13:22:48.375170 22814 tablet_bootstrap.cc:654] T 2214925267994c84b73383ae827170d5 P c79ef076f43f4a73884174a86a6a10c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.380051 22818 leader_election.cc:290] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617)
I20260709 13:22:48.376909 22817 tablet_bootstrap.cc:654] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.382622 22700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a44aff4c84e4f05a00639dc531ecca5" candidate_uuid: "55427c197ec4424580678498fbc616a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
W20260709 13:22:48.384562 22588 leader_election.cc:343] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:48.387161 22774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a44aff4c84e4f05a00639dc531ecca5" candidate_uuid: "55427c197ec4424580678498fbc616a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
I20260709 13:22:48.388206 22774 raft_consensus.cc:3060] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.392710 22815 tablet_bootstrap.cc:492] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69: No bootstrap required, opened a new log
I20260709 13:22:48.393285 22815 ts_tablet_manager.cc:1403] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69: Time spent bootstrapping tablet: real 0.054s	user 0.010s	sys 0.004s
I20260709 13:22:48.396493 22815 raft_consensus.cc:359] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.398456 22625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e8e668ce13b473186eed6b7c92dbf8c" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55427c197ec4424580678498fbc616a2" is_pre_election: true
I20260709 13:22:48.398967 22774 raft_consensus.cc:2468] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55427c197ec4424580678498fbc616a2 in term 1.
I20260709 13:22:48.397181 22815 raft_consensus.cc:385] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:22:48.400571 22739 leader_election.cc:343] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:48.400691 22815 raft_consensus.cc:740] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.401616 22815 consensus_queue.cc:260] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.402261 22591 leader_election.cc:304] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 55427c197ec4424580678498fbc616a2, c79ef076f43f4a73884174a86a6a10c1; no voters: f7e755f96eda4b9ba37aa77b183f2a69
I20260709 13:22:48.402626 22814 tablet_bootstrap.cc:492] T 2214925267994c84b73383ae827170d5 P c79ef076f43f4a73884174a86a6a10c1: No bootstrap required, opened a new log
I20260709 13:22:48.403224 22814 ts_tablet_manager.cc:1403] T 2214925267994c84b73383ae827170d5 P c79ef076f43f4a73884174a86a6a10c1: Time spent bootstrapping tablet: real 0.049s	user 0.015s	sys 0.005s
I20260709 13:22:48.404007 22700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ccb054ff314be7a47b158bf4d141b6" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" is_pre_election: true
I20260709 13:22:48.404848 22822 raft_consensus.cc:2804] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:48.405654 22832 raft_consensus.cc:493] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.406286 22832 raft_consensus.cc:515] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
W20260709 13:22:48.405844 22737 leader_election.cc:343] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:48.407938 22699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e8e668ce13b473186eed6b7c92dbf8c" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" is_pre_election: true
I20260709 13:22:48.408182 22737 leader_election.cc:304] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1; no voters: 55427c197ec4424580678498fbc616a2, f7e755f96eda4b9ba37aa77b183f2a69
I20260709 13:22:48.408459 22832 leader_election.cc:290] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617)
I20260709 13:22:48.408689 22699 raft_consensus.cc:2468] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 0.
I20260709 13:22:48.409089 22832 raft_consensus.cc:2749] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:48.409890 22737 leader_election.cc:304] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 55427c197ec4424580678498fbc616a2
I20260709 13:22:48.410432 22700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c206f085d9400c8978877b15ad9058" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" is_pre_election: true
I20260709 13:22:48.411098 22700 raft_consensus.cc:2468] T 28c206f085d9400c8978877b15ad9058 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 0.
I20260709 13:22:48.411985 22814 raft_consensus.cc:359] T 2214925267994c84b73383ae827170d5 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.412303 22832 raft_consensus.cc:2804] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:48.413582 22814 raft_consensus.cc:385] T 2214925267994c84b73383ae827170d5 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.413647 22832 raft_consensus.cc:493] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:48.413971 22814 raft_consensus.cc:740] T 2214925267994c84b73383ae827170d5 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.414136 22832 raft_consensus.cc:3060] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.414805 22814 consensus_queue.cc:260] T 2214925267994c84b73383ae827170d5 P c79ef076f43f4a73884174a86a6a10c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.416796 22815 ts_tablet_manager.cc:1434] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 13:22:48.417071 22814 ts_tablet_manager.cc:1434] T 2214925267994c84b73383ae827170d5 P c79ef076f43f4a73884174a86a6a10c1: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.000s
I20260709 13:22:48.417930 22815 tablet_bootstrap.cc:492] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69: Bootstrap starting.
I20260709 13:22:48.418133 22814 tablet_bootstrap.cc:492] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1: Bootstrap starting.
I20260709 13:22:48.407984 22817 tablet_bootstrap.cc:492] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2: No bootstrap required, opened a new log
I20260709 13:22:48.422171 22817 ts_tablet_manager.cc:1403] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2: Time spent bootstrapping tablet: real 0.071s	user 0.017s	sys 0.011s
I20260709 13:22:48.412140 22625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c206f085d9400c8978877b15ad9058" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55427c197ec4424580678498fbc616a2" is_pre_election: true
I20260709 13:22:48.413108 22737 leader_election.cc:304] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 
I20260709 13:22:48.423837 22818 raft_consensus.cc:2804] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:48.424265 22818 raft_consensus.cc:493] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:48.424629 22818 raft_consensus.cc:3060] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.425696 22814 tablet_bootstrap.cc:654] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.429411 22822 raft_consensus.cc:697] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [term 1 LEADER]: Becoming Leader. State: Replica: 55427c197ec4424580678498fbc616a2, State: Running, Role: LEADER
I20260709 13:22:48.430431 22822 consensus_queue.cc:237] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } }
W20260709 13:22:48.434226 22739 leader_election.cc:343] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:48.425271 22817 raft_consensus.cc:359] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.434962 22817 raft_consensus.cc:385] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.436239 22817 raft_consensus.cc:740] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55427c197ec4424580678498fbc616a2, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.436586 22832 raft_consensus.cc:515] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.438235 22814 tablet_bootstrap.cc:492] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1: No bootstrap required, opened a new log
I20260709 13:22:48.438884 22814 ts_tablet_manager.cc:1403] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.009s
I20260709 13:22:48.440452 22625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e8e668ce13b473186eed6b7c92dbf8c" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55427c197ec4424580678498fbc616a2"
I20260709 13:22:48.441066 22700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e8e668ce13b473186eed6b7c92dbf8c" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
I20260709 13:22:48.435817 22818 raft_consensus.cc:515] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.441782 22700 raft_consensus.cc:3060] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:48.442112 22739 leader_election.cc:343] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:48.442591 22814 raft_consensus.cc:359] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.443526 22814 raft_consensus.cc:385] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.443867 22814 raft_consensus.cc:740] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.444451 22818 leader_election.cc:290] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617)
I20260709 13:22:48.441213 22832 leader_election.cc:290] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617)
I20260709 13:22:48.437108 22817 consensus_queue.cc:260] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.445043 22625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c206f085d9400c8978877b15ad9058" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55427c197ec4424580678498fbc616a2"
I20260709 13:22:48.444880 22814 consensus_queue.cc:260] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.444664 22699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c206f085d9400c8978877b15ad9058" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
I20260709 13:22:48.446297 22699 raft_consensus.cc:3060] T 28c206f085d9400c8978877b15ad9058 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:22:48.446588 22739 leader_election.cc:343] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:48.447196 22817 ts_tablet_manager.cc:1434] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:22:48.447371 22814 ts_tablet_manager.cc:1434] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:22:48.448356 22814 tablet_bootstrap.cc:492] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1: Bootstrap starting.
I20260709 13:22:48.448357 22817 tablet_bootstrap.cc:492] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2: Bootstrap starting.
W20260709 13:22:48.453680 22571 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:22:48.457494 22814 tablet_bootstrap.cc:654] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.459335 22817 tablet_bootstrap.cc:654] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.461764 22699 raft_consensus.cc:2468] T 28c206f085d9400c8978877b15ad9058 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 1.
I20260709 13:22:48.463570 22737 leader_election.cc:304] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 55427c197ec4424580678498fbc616a2
I20260709 13:22:48.464802 22832 raft_consensus.cc:2804] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:48.465299 22832 raft_consensus.cc:697] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 [term 1 LEADER]: Becoming Leader. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Running, Role: LEADER
I20260709 13:22:48.466171 22832 consensus_queue.cc:237] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.440994 22815 tablet_bootstrap.cc:654] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.473307 22817 tablet_bootstrap.cc:492] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2: No bootstrap required, opened a new log
I20260709 13:22:48.474181 22817 ts_tablet_manager.cc:1403] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2: Time spent bootstrapping tablet: real 0.026s	user 0.020s	sys 0.000s
I20260709 13:22:48.476727 22522 catalog_manager.cc:5697] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 55427c197ec4424580678498fbc616a2 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "55427c197ec4424580678498fbc616a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:48.478670 22817 raft_consensus.cc:359] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.485121 22817 raft_consensus.cc:385] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.485594 22817 raft_consensus.cc:740] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55427c197ec4424580678498fbc616a2, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.486608 22817 consensus_queue.cc:260] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.488582 22520 catalog_manager.cc:5697] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "c79ef076f43f4a73884174a86a6a10c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:48.462278 22700 raft_consensus.cc:2468] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 1.
I20260709 13:22:48.492019 22737 leader_election.cc:304] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 55427c197ec4424580678498fbc616a2
I20260709 13:22:48.493239 22832 raft_consensus.cc:2804] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:48.493723 22832 raft_consensus.cc:697] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 1 LEADER]: Becoming Leader. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Running, Role: LEADER
I20260709 13:22:48.494167 22815 tablet_bootstrap.cc:492] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69: No bootstrap required, opened a new log
I20260709 13:22:48.494547 22832 consensus_queue.cc:237] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.494935 22815 ts_tablet_manager.cc:1403] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69: Time spent bootstrapping tablet: real 0.077s	user 0.005s	sys 0.025s
I20260709 13:22:48.489945 22817 ts_tablet_manager.cc:1434] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.004s
I20260709 13:22:48.496704 22817 tablet_bootstrap.cc:492] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2: Bootstrap starting.
I20260709 13:22:48.508577 22817 tablet_bootstrap.cc:654] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.509497 22815 raft_consensus.cc:359] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.510613 22815 raft_consensus.cc:385] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.511085 22815 raft_consensus.cc:740] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.510210 22814 tablet_bootstrap.cc:492] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1: No bootstrap required, opened a new log
I20260709 13:22:48.512578 22814 ts_tablet_manager.cc:1403] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1: Time spent bootstrapping tablet: real 0.064s	user 0.017s	sys 0.038s
I20260709 13:22:48.516759 22814 raft_consensus.cc:359] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.517786 22814 raft_consensus.cc:385] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.518232 22814 raft_consensus.cc:740] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.517912 22815 consensus_queue.cc:260] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.519196 22814 consensus_queue.cc:260] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.521025 22815 ts_tablet_manager.cc:1434] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:22:48.522424 22814 ts_tablet_manager.cc:1434] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 13:22:48.525022 22815 tablet_bootstrap.cc:492] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69: Bootstrap starting.
I20260709 13:22:48.525022 22814 tablet_bootstrap.cc:492] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1: Bootstrap starting.
I20260709 13:22:48.530457 22817 tablet_bootstrap.cc:492] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2: No bootstrap required, opened a new log
I20260709 13:22:48.531191 22817 ts_tablet_manager.cc:1403] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2: Time spent bootstrapping tablet: real 0.035s	user 0.018s	sys 0.004s
I20260709 13:22:48.533541 22814 tablet_bootstrap.cc:654] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.535180 22815 tablet_bootstrap.cc:654] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.534775 22817 raft_consensus.cc:359] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.535723 22817 raft_consensus.cc:385] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.536125 22817 raft_consensus.cc:740] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55427c197ec4424580678498fbc616a2, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.537175 22817 consensus_queue.cc:260] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.540454 22817 ts_tablet_manager.cc:1434] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.003s
I20260709 13:22:48.542073 22817 tablet_bootstrap.cc:492] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2: Bootstrap starting.
I20260709 13:22:48.545658 22814 tablet_bootstrap.cc:492] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1: No bootstrap required, opened a new log
I20260709 13:22:48.546200 22814 ts_tablet_manager.cc:1403] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.008s
I20260709 13:22:48.549495 22814 raft_consensus.cc:359] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.550510 22814 raft_consensus.cc:385] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.550462 22822 raft_consensus.cc:493] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.551235 22814 raft_consensus.cc:740] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.551560 22822 raft_consensus.cc:515] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.552600 22814 consensus_queue.cc:260] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.555341 22814 ts_tablet_manager.cc:1434] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:22:48.555373 22824 raft_consensus.cc:493] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.556134 22700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbcb0fb6f4e46b991774ca89f45da59" candidate_uuid: "55427c197ec4424580678498fbc616a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" is_pre_election: true
I20260709 13:22:48.556221 22824 raft_consensus.cc:515] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.544752 22522 catalog_manager.cc:5697] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "c79ef076f43f4a73884174a86a6a10c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:48.556962 22814 tablet_bootstrap.cc:492] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1: Bootstrap starting.
I20260709 13:22:48.557013 22700 raft_consensus.cc:2468] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55427c197ec4424580678498fbc616a2 in term 0.
I20260709 13:22:48.558506 22815 tablet_bootstrap.cc:492] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69: No bootstrap required, opened a new log
I20260709 13:22:48.558947 22588 leader_election.cc:304] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55427c197ec4424580678498fbc616a2, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 
I20260709 13:22:48.559016 22815 ts_tablet_manager.cc:1403] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.003s
I20260709 13:22:48.560505 22836 raft_consensus.cc:2804] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:48.563917 22836 raft_consensus.cc:493] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:48.564321 22836 raft_consensus.cc:3060] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.562088 22824 leader_election.cc:290] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:48.564750 22815 raft_consensus.cc:359] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.565611 22815 raft_consensus.cc:385] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.566262 22815 raft_consensus.cc:740] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.566795 22817 tablet_bootstrap.cc:654] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.570845 22822 leader_election.cc:290] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:48.572991 22774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbcb0fb6f4e46b991774ca89f45da59" candidate_uuid: "55427c197ec4424580678498fbc616a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1" is_pre_election: true
I20260709 13:22:48.573781 22814 tablet_bootstrap.cc:654] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.573851 22774 raft_consensus.cc:2468] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55427c197ec4424580678498fbc616a2 in term 0.
I20260709 13:22:48.578794 22836 raft_consensus.cc:515] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.569764 22815 consensus_queue.cc:260] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.582919 22774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbcb0fb6f4e46b991774ca89f45da59" candidate_uuid: "55427c197ec4424580678498fbc616a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
I20260709 13:22:48.583648 22774 raft_consensus.cc:3060] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.586750 22815 ts_tablet_manager.cc:1434] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.003s
I20260709 13:22:48.588382 22815 tablet_bootstrap.cc:492] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69: Bootstrap starting.
I20260709 13:22:48.594606 22774 raft_consensus.cc:2468] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55427c197ec4424580678498fbc616a2 in term 1.
I20260709 13:22:48.596164 22591 leader_election.cc:304] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55427c197ec4424580678498fbc616a2, c79ef076f43f4a73884174a86a6a10c1; no voters: 
I20260709 13:22:48.597167 22822 raft_consensus.cc:2804] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:48.597826 22814 tablet_bootstrap.cc:492] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1: No bootstrap required, opened a new log
I20260709 13:22:48.598402 22814 ts_tablet_manager.cc:1403] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1: Time spent bootstrapping tablet: real 0.042s	user 0.020s	sys 0.007s
I20260709 13:22:48.598850 22822 raft_consensus.cc:697] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [term 1 LEADER]: Becoming Leader. State: Replica: 55427c197ec4424580678498fbc616a2, State: Running, Role: LEADER
I20260709 13:22:48.598407 22815 tablet_bootstrap.cc:654] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.600344 22822 consensus_queue.cc:237] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.601701 22814 raft_consensus.cc:359] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.602522 22814 raft_consensus.cc:385] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.602857 22814 raft_consensus.cc:740] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.603365 22855 raft_consensus.cc:493] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.603864 22814 consensus_queue.cc:260] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.606384 22832 raft_consensus.cc:493] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.606925 22832 raft_consensus.cc:515] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.610637 22832 leader_election.cc:290] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617)
I20260709 13:22:48.610772 22700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d10d445a81457f817ac6b4bfcb77c7" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" is_pre_election: true
I20260709 13:22:48.611788 22700 raft_consensus.cc:2468] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 0.
I20260709 13:22:48.613047 22814 ts_tablet_manager.cc:1434] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:22:48.613942 22737 leader_election.cc:304] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 
I20260709 13:22:48.614979 22814 tablet_bootstrap.cc:492] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1: Bootstrap starting.
I20260709 13:22:48.615453 22832 raft_consensus.cc:2804] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:48.616184 22832 raft_consensus.cc:493] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:48.603905 22855 raft_consensus.cc:515] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.618117 22832 raft_consensus.cc:3060] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.619154 22815 tablet_bootstrap.cc:492] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69: No bootstrap required, opened a new log
I20260709 13:22:48.619747 22815 ts_tablet_manager.cc:1403] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.011s
I20260709 13:22:48.620046 22822 raft_consensus.cc:493] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.620491 22855 leader_election.cc:290] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:48.620657 22822 raft_consensus.cc:515] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.623040 22815 raft_consensus.cc:359] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.623867 22815 raft_consensus.cc:385] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.624240 22815 raft_consensus.cc:740] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.625047 22815 consensus_queue.cc:260] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.625500 22773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d10d445a81457f817ac6b4bfcb77c7" candidate_uuid: "55427c197ec4424580678498fbc616a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1" is_pre_election: true
I20260709 13:22:48.623265 22822 leader_election.cc:290] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:48.626657 22772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a93e444ff9ec4a578e573e970e281e0a" candidate_uuid: "55427c197ec4424580678498fbc616a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1" is_pre_election: true
I20260709 13:22:48.627333 22772 raft_consensus.cc:2468] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55427c197ec4424580678498fbc616a2 in term 0.
I20260709 13:22:48.627413 22625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d10d445a81457f817ac6b4bfcb77c7" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55427c197ec4424580678498fbc616a2" is_pre_election: true
I20260709 13:22:48.627986 22815 ts_tablet_manager.cc:1434] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:22:48.628427 22625 raft_consensus.cc:2468] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 0.
I20260709 13:22:48.629119 22815 tablet_bootstrap.cc:492] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69: Bootstrap starting.
I20260709 13:22:48.629447 22814 tablet_bootstrap.cc:654] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.631351 22591 leader_election.cc:304] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55427c197ec4424580678498fbc616a2, c79ef076f43f4a73884174a86a6a10c1; no voters: 
I20260709 13:22:48.632611 22822 raft_consensus.cc:2804] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:48.633108 22822 raft_consensus.cc:493] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:48.633644 22822 raft_consensus.cc:3060] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.640664 22815 tablet_bootstrap.cc:654] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.642014 22814 tablet_bootstrap.cc:492] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1: No bootstrap required, opened a new log
I20260709 13:22:48.642628 22814 ts_tablet_manager.cc:1403] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.004s
I20260709 13:22:48.643321 22836 leader_election.cc:290] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 election: Requested vote from peers f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:48.647043 22699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d10d445a81457f817ac6b4bfcb77c7" candidate_uuid: "55427c197ec4424580678498fbc616a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" is_pre_election: true
I20260709 13:22:48.647553 22698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a93e444ff9ec4a578e573e970e281e0a" candidate_uuid: "55427c197ec4424580678498fbc616a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" is_pre_election: true
I20260709 13:22:48.646088 22700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbcb0fb6f4e46b991774ca89f45da59" candidate_uuid: "55427c197ec4424580678498fbc616a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
I20260709 13:22:48.648263 22698 raft_consensus.cc:2468] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55427c197ec4424580678498fbc616a2 in term 0.
I20260709 13:22:48.648396 22625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a93e444ff9ec4a578e573e970e281e0a" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55427c197ec4424580678498fbc616a2" is_pre_election: true
I20260709 13:22:48.647727 22699 raft_consensus.cc:2468] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55427c197ec4424580678498fbc616a2 in term 0.
I20260709 13:22:48.648552 22700 raft_consensus.cc:3060] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.650761 22832 raft_consensus.cc:515] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.651221 22588 leader_election.cc:304] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55427c197ec4424580678498fbc616a2, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 
I20260709 13:22:48.656666 22815 tablet_bootstrap.cc:492] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69: No bootstrap required, opened a new log
I20260709 13:22:48.646042 22814 raft_consensus.cc:359] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.657238 22814 raft_consensus.cc:385] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.657675 22814 raft_consensus.cc:740] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.658762 22773 raft_consensus.cc:2393] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 55427c197ec4424580678498fbc616a2 in current term 1: Already voted for candidate c79ef076f43f4a73884174a86a6a10c1 in this term.
I20260709 13:22:48.658990 22523 catalog_manager.cc:5697] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 55427c197ec4424580678498fbc616a2 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "55427c197ec4424580678498fbc616a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:48.661711 22814 consensus_queue.cc:260] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.662627 22822 raft_consensus.cc:515] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.664093 22625 raft_consensus.cc:2393] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f7e755f96eda4b9ba37aa77b183f2a69 in current term 1: Already voted for candidate 55427c197ec4424580678498fbc616a2 in this term.
I20260709 13:22:48.664291 22699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d10d445a81457f817ac6b4bfcb77c7" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
I20260709 13:22:48.664723 22814 ts_tablet_manager.cc:1434] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:22:48.665041 22699 raft_consensus.cc:3060] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.657222 22815 ts_tablet_manager.cc:1403] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69: Time spent bootstrapping tablet: real 0.028s	user 0.009s	sys 0.008s
I20260709 13:22:48.665580 22822 leader_election.cc:290] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 election: Requested vote from peers f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:48.669004 22815 raft_consensus.cc:359] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.669865 22815 raft_consensus.cc:385] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.670202 22815 raft_consensus.cc:740] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.671016 22815 consensus_queue.cc:260] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.659906 22832 leader_election.cc:290] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617)
I20260709 13:22:48.672904 22700 raft_consensus.cc:2468] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55427c197ec4424580678498fbc616a2 in term 1.
I20260709 13:22:48.673307 22815 ts_tablet_manager.cc:1434] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:22:48.674523 22815 tablet_bootstrap.cc:492] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69: Bootstrap starting.
I20260709 13:22:48.666373 22822 raft_consensus.cc:2804] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:48.676424 22822 raft_consensus.cc:493] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:48.676864 22822 raft_consensus.cc:3060] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.677850 22773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a93e444ff9ec4a578e573e970e281e0a" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1" is_pre_election: true
I20260709 13:22:48.678659 22773 raft_consensus.cc:2468] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 0.
I20260709 13:22:48.680060 22666 leader_election.cc:304] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 
I20260709 13:22:48.680187 22855 raft_consensus.cc:493] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.680800 22855 raft_consensus.cc:515] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.681035 22824 raft_consensus.cc:2804] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:48.681847 22824 raft_consensus.cc:493] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:48.682236 22824 raft_consensus.cc:3060] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.683197 22855 leader_election.cc:290] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:48.686214 22822 raft_consensus.cc:515] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.689239 22822 leader_election.cc:290] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 election: Requested vote from peers f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:48.681177 22815 tablet_bootstrap.cc:654] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.691664 22817 tablet_bootstrap.cc:492] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2: No bootstrap required, opened a new log
I20260709 13:22:48.691720 22700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a93e444ff9ec4a578e573e970e281e0a" candidate_uuid: "55427c197ec4424580678498fbc616a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
I20260709 13:22:48.692420 22817 ts_tablet_manager.cc:1403] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2: Time spent bootstrapping tablet: real 0.151s	user 0.028s	sys 0.026s
I20260709 13:22:48.692780 22773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a93e444ff9ec4a578e573e970e281e0a" candidate_uuid: "55427c197ec4424580678498fbc616a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
I20260709 13:22:48.693401 22773 raft_consensus.cc:3060] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.696082 22817 raft_consensus.cc:359] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.696792 22817 raft_consensus.cc:385] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.697108 22817 raft_consensus.cc:740] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55427c197ec4424580678498fbc616a2, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.698051 22815 tablet_bootstrap.cc:492] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69: No bootstrap required, opened a new log
I20260709 13:22:48.697893 22817 consensus_queue.cc:260] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.698249 22832 raft_consensus.cc:493] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.698602 22815 ts_tablet_manager.cc:1403] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69: Time spent bootstrapping tablet: real 0.024s	user 0.007s	sys 0.007s
I20260709 13:22:48.699545 22824 raft_consensus.cc:515] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.700378 22832 raft_consensus.cc:515] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.702467 22824 leader_election.cc:290] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 1 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:48.702841 22832 leader_election.cc:290] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617)
I20260709 13:22:48.703693 22700 raft_consensus.cc:2393] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 55427c197ec4424580678498fbc616a2 in current term 1: Already voted for candidate f7e755f96eda4b9ba37aa77b183f2a69 in this term.
I20260709 13:22:48.703384 22815 raft_consensus.cc:359] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } }
I20260709 13:22:48.704226 22815 raft_consensus.cc:385] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.704540 22815 raft_consensus.cc:740] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.705209 22774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a93e444ff9ec4a578e573e970e281e0a" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
I20260709 13:22:48.707154 22817 ts_tablet_manager.cc:1434] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.001s
I20260709 13:22:48.707144 22697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "add99f7fe29440ab82c13df2e1bc541d" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" is_pre_election: true
I20260709 13:22:48.707990 22697 raft_consensus.cc:2468] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 0.
I20260709 13:22:48.708104 22625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d10d445a81457f817ac6b4bfcb77c7" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55427c197ec4424580678498fbc616a2"
I20260709 13:22:48.708333 22817 tablet_bootstrap.cc:492] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2: Bootstrap starting.
I20260709 13:22:48.708650 22773 raft_consensus.cc:2468] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55427c197ec4424580678498fbc616a2 in term 1.
I20260709 13:22:48.708956 22625 raft_consensus.cc:2393] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c79ef076f43f4a73884174a86a6a10c1 in current term 1: Already voted for candidate 55427c197ec4424580678498fbc616a2 in this term.
I20260709 13:22:48.689291 22699 raft_consensus.cc:2468] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 1.
I20260709 13:22:48.710500 22698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c206f085d9400c8978877b15ad9058" candidate_uuid: "55427c197ec4424580678498fbc616a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" is_pre_election: true
I20260709 13:22:48.710979 22774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28c206f085d9400c8978877b15ad9058" candidate_uuid: "55427c197ec4424580678498fbc616a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1" is_pre_election: true
I20260709 13:22:48.711601 22698 raft_consensus.cc:2393] T 28c206f085d9400c8978877b15ad9058 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 55427c197ec4424580678498fbc616a2 in current term 1: Already voted for candidate c79ef076f43f4a73884174a86a6a10c1 in this term.
I20260709 13:22:48.711503 22591 leader_election.cc:304] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55427c197ec4424580678498fbc616a2, c79ef076f43f4a73884174a86a6a10c1; no voters: 
I20260709 13:22:48.710470 22625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "add99f7fe29440ab82c13df2e1bc541d" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55427c197ec4424580678498fbc616a2" is_pre_election: true
I20260709 13:22:48.713668 22836 raft_consensus.cc:2804] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:22:48.713799 22739 leader_election.cc:343] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:48.715006 22737 leader_election.cc:304] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 55427c197ec4424580678498fbc616a2
I20260709 13:22:48.715875 22832 raft_consensus.cc:2804] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:48.715920 22698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d10d445a81457f817ac6b4bfcb77c7" candidate_uuid: "55427c197ec4424580678498fbc616a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
I20260709 13:22:48.716454 22832 raft_consensus.cc:493] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:48.716583 22817 tablet_bootstrap.cc:654] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.716908 22832 raft_consensus.cc:3060] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.717002 22698 raft_consensus.cc:2393] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 55427c197ec4424580678498fbc616a2 in current term 1: Already voted for candidate c79ef076f43f4a73884174a86a6a10c1 in this term.
I20260709 13:22:48.715099 22836 raft_consensus.cc:697] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 1 LEADER]: Becoming Leader. State: Replica: 55427c197ec4424580678498fbc616a2, State: Running, Role: LEADER
I20260709 13:22:48.722939 22737 leader_election.cc:304] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 55427c197ec4424580678498fbc616a2
I20260709 13:22:48.724193 22818 raft_consensus.cc:2804] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:48.725495 22625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a93e444ff9ec4a578e573e970e281e0a" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55427c197ec4424580678498fbc616a2"
I20260709 13:22:48.727241 22818 raft_consensus.cc:697] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [term 1 LEADER]: Becoming Leader. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Running, Role: LEADER
I20260709 13:22:48.725486 22832 raft_consensus.cc:515] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.705267 22815 consensus_queue.cc:260] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } }
I20260709 13:22:48.728328 22818 consensus_queue.cc:237] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.730278 22832 leader_election.cc:290] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617)
I20260709 13:22:48.731076 22625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "add99f7fe29440ab82c13df2e1bc541d" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55427c197ec4424580678498fbc616a2"
I20260709 13:22:48.731532 22773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d10d445a81457f817ac6b4bfcb77c7" candidate_uuid: "55427c197ec4424580678498fbc616a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
I20260709 13:22:48.732245 22698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "add99f7fe29440ab82c13df2e1bc541d" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
W20260709 13:22:48.732709 22739 leader_election.cc:343] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:22:48.733934 22815 ts_tablet_manager.cc:1434] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.002s
I20260709 13:22:48.733016 22698 raft_consensus.cc:3060] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.734989 22588 leader_election.cc:304] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55427c197ec4424580678498fbc616a2; no voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69
I20260709 13:22:48.736874 22822 raft_consensus.cc:3060] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.735199 22815 tablet_bootstrap.cc:492] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69: Bootstrap starting.
I20260709 13:22:48.745522 22815 tablet_bootstrap.cc:654] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.748517 22822 raft_consensus.cc:2749] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:48.750479 22588 leader_election.cc:304] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55427c197ec4424580678498fbc616a2; no voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69
I20260709 13:22:48.723161 22836 consensus_queue.cc:237] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.754369 22665 leader_election.cc:304] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f7e755f96eda4b9ba37aa77b183f2a69; no voters: 55427c197ec4424580678498fbc616a2, c79ef076f43f4a73884174a86a6a10c1
I20260709 13:22:48.758705 22824 raft_consensus.cc:2749] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:22:48.760300 22822 raft_consensus.cc:2749] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:22:48.764959 22817 tablet_bootstrap.cc:492] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2: No bootstrap required, opened a new log
I20260709 13:22:48.765686 22817 ts_tablet_manager.cc:1403] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2: Time spent bootstrapping tablet: real 0.058s	user 0.032s	sys 0.023s
I20260709 13:22:48.768708 22817 raft_consensus.cc:359] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.769390 22817 raft_consensus.cc:385] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.769660 22817 raft_consensus.cc:740] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55427c197ec4424580678498fbc616a2, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.770326 22817 consensus_queue.cc:260] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.772043 22817 ts_tablet_manager.cc:1434] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.003s
I20260709 13:22:48.772516 22522 catalog_manager.cc:5697] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "c79ef076f43f4a73884174a86a6a10c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:48.775487 22817 tablet_bootstrap.cc:492] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2: Bootstrap starting.
I20260709 13:22:48.776139 22698 raft_consensus.cc:2468] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 1.
I20260709 13:22:48.777642 22737 leader_election.cc:304] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 55427c197ec4424580678498fbc616a2
I20260709 13:22:48.778923 22523 catalog_manager.cc:5697] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 55427c197ec4424580678498fbc616a2 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "55427c197ec4424580678498fbc616a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:48.781648 22815 tablet_bootstrap.cc:492] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69: No bootstrap required, opened a new log
I20260709 13:22:48.782150 22818 raft_consensus.cc:2804] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:48.782852 22817 tablet_bootstrap.cc:654] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.783862 22818 raft_consensus.cc:697] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [term 1 LEADER]: Becoming Leader. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Running, Role: LEADER
I20260709 13:22:48.784940 22818 consensus_queue.cc:237] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.782200 22815 ts_tablet_manager.cc:1403] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69: Time spent bootstrapping tablet: real 0.047s	user 0.006s	sys 0.016s
I20260709 13:22:48.789093 22815 raft_consensus.cc:359] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.789868 22815 raft_consensus.cc:385] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.790194 22815 raft_consensus.cc:740] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.791054 22815 consensus_queue.cc:260] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.801198 22824 raft_consensus.cc:493] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.801777 22824 raft_consensus.cc:515] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.804318 22824 leader_election.cc:290] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:48.805603 22817 tablet_bootstrap.cc:492] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2: No bootstrap required, opened a new log
I20260709 13:22:48.806232 22817 ts_tablet_manager.cc:1403] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2: Time spent bootstrapping tablet: real 0.031s	user 0.017s	sys 0.011s
I20260709 13:22:48.806391 22625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2214925267994c84b73383ae827170d5" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55427c197ec4424580678498fbc616a2" is_pre_election: true
I20260709 13:22:48.807126 22773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2214925267994c84b73383ae827170d5" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1" is_pre_election: true
I20260709 13:22:48.807678 22815 ts_tablet_manager.cc:1434] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:22:48.808049 22773 raft_consensus.cc:2468] T 2214925267994c84b73383ae827170d5 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 0.
I20260709 13:22:48.809517 22666 leader_election.cc:304] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 
I20260709 13:22:48.807183 22625 raft_consensus.cc:2468] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 0.
I20260709 13:22:48.809626 22817 raft_consensus.cc:359] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.810706 22817 raft_consensus.cc:385] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.811069 22817 raft_consensus.cc:740] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55427c197ec4424580678498fbc616a2, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.810225 22523 catalog_manager.cc:5697] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67). New cstate: current_term: 1 leader_uuid: "c79ef076f43f4a73884174a86a6a10c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:48.811573 22824 raft_consensus.cc:2804] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:48.812093 22824 raft_consensus.cc:493] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:48.812583 22824 raft_consensus.cc:3060] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.812098 22817 consensus_queue.cc:260] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.818379 22817 ts_tablet_manager.cc:1434] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:22:48.819531 22817 tablet_bootstrap.cc:492] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2: Bootstrap starting.
I20260709 13:22:48.821019 22824 raft_consensus.cc:515] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.821296 22836 consensus_queue.cc:1048] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:48.826009 22824 leader_election.cc:290] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 1 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:48.826382 22817 tablet_bootstrap.cc:654] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2: Neither blocks nor log segments found. Creating new log.
I20260709 13:22:48.826624 22774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2214925267994c84b73383ae827170d5" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
I20260709 13:22:48.827287 22774 raft_consensus.cc:3060] T 2214925267994c84b73383ae827170d5 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.829348 22625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2214925267994c84b73383ae827170d5" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55427c197ec4424580678498fbc616a2"
I20260709 13:22:48.830017 22625 raft_consensus.cc:3060] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.835634 22774 raft_consensus.cc:2468] T 2214925267994c84b73383ae827170d5 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 1.
I20260709 13:22:48.836763 22824 raft_consensus.cc:493] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.837304 22666 leader_election.cc:304] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 
I20260709 13:22:48.837585 22824 raft_consensus.cc:515] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.840471 22824 leader_election.cc:290] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:48.841462 22824 raft_consensus.cc:2804] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:48.841737 22624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7505614384fc47b19d23973ba804ea5a" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55427c197ec4424580678498fbc616a2" is_pre_election: true
I20260709 13:22:48.842888 22624 raft_consensus.cc:2468] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 0.
I20260709 13:22:48.844559 22665 leader_election.cc:304] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55427c197ec4424580678498fbc616a2, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 
I20260709 13:22:48.847980 22871 raft_consensus.cc:2804] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:48.849660 22774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7505614384fc47b19d23973ba804ea5a" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1" is_pre_election: true
I20260709 13:22:48.850880 22625 raft_consensus.cc:2468] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 1.
I20260709 13:22:48.852114 22871 raft_consensus.cc:493] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:48.852522 22871 raft_consensus.cc:3060] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.842147 22824 raft_consensus.cc:697] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 LEADER]: Becoming Leader. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Running, Role: LEADER
I20260709 13:22:48.853772 22817 tablet_bootstrap.cc:492] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2: No bootstrap required, opened a new log
I20260709 13:22:48.854283 22817 ts_tablet_manager.cc:1403] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.017s
I20260709 13:22:48.854517 22824 consensus_queue.cc:237] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.856920 22817 raft_consensus.cc:359] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.857651 22817 raft_consensus.cc:385] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:22:48.858043 22774 raft_consensus.cc:2468] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 0.
I20260709 13:22:48.858150 22817 raft_consensus.cc:740] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55427c197ec4424580678498fbc616a2, State: Initialized, Role: FOLLOWER
I20260709 13:22:48.860061 22817 consensus_queue.cc:260] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.862232 22817 ts_tablet_manager.cc:1434] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:22:48.865260 22871 raft_consensus.cc:515] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.871483 22625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7505614384fc47b19d23973ba804ea5a" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55427c197ec4424580678498fbc616a2"
I20260709 13:22:48.872313 22773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7505614384fc47b19d23973ba804ea5a" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
I20260709 13:22:48.872455 22625 raft_consensus.cc:3060] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.872999 22773 raft_consensus.cc:3060] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.875752 22521 catalog_manager.cc:5697] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69 reported cstate change: term changed from 0 to 1, leader changed from <none> to f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:48.880977 22773 raft_consensus.cc:2468] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 1.
I20260709 13:22:48.882242 22666 leader_election.cc:304] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 
I20260709 13:22:48.883170 22871 leader_election.cc:290] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 1 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:48.883713 22871 raft_consensus.cc:2804] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:48.884167 22871 raft_consensus.cc:697] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 LEADER]: Becoming Leader. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Running, Role: LEADER
I20260709 13:22:48.885021 22871 consensus_queue.cc:237] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.888729 22855 raft_consensus.cc:493] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:48.889276 22855 raft_consensus.cc:515] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.891469 22855 leader_election.cc:290] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:48.895119 22698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ccb054ff314be7a47b158bf4d141b6" candidate_uuid: "55427c197ec4424580678498fbc616a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" is_pre_election: true
I20260709 13:22:48.895803 22698 raft_consensus.cc:2468] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55427c197ec4424580678498fbc616a2 in term 0.
I20260709 13:22:48.897315 22588 leader_election.cc:304] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55427c197ec4424580678498fbc616a2, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 
I20260709 13:22:48.898937 22625 raft_consensus.cc:2468] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 1.
I20260709 13:22:48.900146 22698 raft_consensus.cc:3060] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.901970 22773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ccb054ff314be7a47b158bf4d141b6" candidate_uuid: "55427c197ec4424580678498fbc616a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1" is_pre_election: true
I20260709 13:22:48.902925 22773 raft_consensus.cc:2468] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55427c197ec4424580678498fbc616a2 in term 0.
I20260709 13:22:48.904508 22822 raft_consensus.cc:2804] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:22:48.905120 22822 raft_consensus.cc:493] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:48.905481 22822 raft_consensus.cc:3060] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.912880 22832 consensus_queue.cc:1048] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:48.914917 22822 raft_consensus.cc:515] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.917256 22822 leader_election.cc:290] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 election: Requested vote from peers f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:48.915166 22521 catalog_manager.cc:5697] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 reported cstate change: term changed from 0 to 1, leader changed from <none> to f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:48.919281 22774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ccb054ff314be7a47b158bf4d141b6" candidate_uuid: "55427c197ec4424580678498fbc616a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
I20260709 13:22:48.919776 22822 consensus_queue.cc:1048] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:48.920249 22774 raft_consensus.cc:3060] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.922242 22698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ccb054ff314be7a47b158bf4d141b6" candidate_uuid: "55427c197ec4424580678498fbc616a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
I20260709 13:22:48.922935 22698 raft_consensus.cc:3060] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.929993 22624 raft_consensus.cc:3060] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:48.936971 22774 raft_consensus.cc:2468] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55427c197ec4424580678498fbc616a2 in term 1.
I20260709 13:22:48.938308 22591 leader_election.cc:304] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55427c197ec4424580678498fbc616a2, c79ef076f43f4a73884174a86a6a10c1; no voters: 
I20260709 13:22:48.946648 22832 consensus_queue.cc:1048] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:48.948336 22822 raft_consensus.cc:2804] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:22:48.949083 22822 raft_consensus.cc:697] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [term 1 LEADER]: Becoming Leader. State: Replica: 55427c197ec4424580678498fbc616a2, State: Running, Role: LEADER
I20260709 13:22:48.950309 22822 consensus_queue.cc:237] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:48.956092 22863 consensus_queue.cc:1048] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:48.958949 22698 raft_consensus.cc:2468] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55427c197ec4424580678498fbc616a2 in term 1.
I20260709 13:22:48.967239 22863 consensus_queue.cc:1048] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:48.967935 22523 catalog_manager.cc:5697] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 55427c197ec4424580678498fbc616a2 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "55427c197ec4424580678498fbc616a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:48.998219 22822 consensus_queue.cc:1048] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:49.008733 22856 consensus_queue.cc:1048] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:49.029066 22856 raft_consensus.cc:493] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:49.029620 22856 raft_consensus.cc:515] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:49.031442 22856 leader_election.cc:290] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:49.032366 22699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "add99f7fe29440ab82c13df2e1bc541d" candidate_uuid: "55427c197ec4424580678498fbc616a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" is_pre_election: true
I20260709 13:22:49.032562 22774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "add99f7fe29440ab82c13df2e1bc541d" candidate_uuid: "55427c197ec4424580678498fbc616a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1" is_pre_election: true
I20260709 13:22:49.033074 22699 raft_consensus.cc:2393] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 55427c197ec4424580678498fbc616a2 in current term 1: Already voted for candidate c79ef076f43f4a73884174a86a6a10c1 in this term.
I20260709 13:22:49.034149 22588 leader_election.cc:304] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 55427c197ec4424580678498fbc616a2; no voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69
I20260709 13:22:49.034937 22856 raft_consensus.cc:3060] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:22:49.040817 22856 raft_consensus.cc:2749] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:22:49.103999 11517 ts_manager.cc:295] Set tserver state for 55427c197ec4424580678498fbc616a2 to MAINTENANCE_MODE
I20260709 13:22:49.105042 11517 tablet_server.cc:179] TabletServer@127.11.63.65:0 shutting down...
I20260709 13:22:49.128427 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:49.129631 11517 tablet_replica.cc:333] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2: stopping tablet replica
I20260709 13:22:49.130368 11517 raft_consensus.cc:2243] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:49.130785 11517 raft_consensus.cc:2272] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:49.133162 11517 tablet_replica.cc:333] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2: stopping tablet replica
I20260709 13:22:49.133816 11517 raft_consensus.cc:2243] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:49.134287 11517 raft_consensus.cc:2272] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:49.136356 11517 tablet_replica.cc:333] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2: stopping tablet replica
I20260709 13:22:49.136898 11517 raft_consensus.cc:2243] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:49.137270 11517 raft_consensus.cc:2272] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:49.139942 11517 tablet_replica.cc:333] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2: stopping tablet replica
I20260709 13:22:49.140641 11517 raft_consensus.cc:2243] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:49.141120 11517 raft_consensus.cc:2272] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:49.143296 11517 tablet_replica.cc:333] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2: stopping tablet replica
I20260709 13:22:49.144062 11517 raft_consensus.cc:2243] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:49.143968 22856 consensus_queue.cc:1048] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:22:49.144965 11517 pending_rounds.cc:70] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2: Trying to abort 1 pending ops.
I20260709 13:22:49.145247 11517 pending_rounds.cc:77] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305639399233126400 op_type: NO_OP noop_request { }
I20260709 13:22:49.145676 11517 raft_consensus.cc:2889] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:22:49.146036 11517 raft_consensus.cc:2272] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:49.146528 22739 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 180 similar messages]
I20260709 13:22:49.149518 11517 tablet_replica.cc:333] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2: stopping tablet replica
I20260709 13:22:49.150553 11517 raft_consensus.cc:2243] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:49.151386 11517 pending_rounds.cc:70] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2: Trying to abort 1 pending ops.
I20260709 13:22:49.151707 11517 pending_rounds.cc:77] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305639398430842880 op_type: NO_OP noop_request { }
I20260709 13:22:49.152135 11517 raft_consensus.cc:2889] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:22:49.152515 11517 raft_consensus.cc:2272] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:49.155117 11517 tablet_replica.cc:333] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2: stopping tablet replica
I20260709 13:22:49.156174 11517 raft_consensus.cc:2243] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:49.157097 11517 raft_consensus.cc:2272] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:49.158538 22739 consensus_peers.cc:597] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:49.159919 11517 tablet_replica.cc:333] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2: stopping tablet replica
I20260709 13:22:49.160733 11517 raft_consensus.cc:2243] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:49.161218 11517 raft_consensus.cc:2272] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:49.163810 11517 tablet_replica.cc:333] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2: stopping tablet replica
I20260709 13:22:49.164357 11517 raft_consensus.cc:2243] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:49.165207 11517 raft_consensus.cc:2272] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:49.167575 11517 tablet_replica.cc:333] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2: stopping tablet replica
I20260709 13:22:49.168078 11517 raft_consensus.cc:2243] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:49.168495 11517 raft_consensus.cc:2272] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:49.188036 11517 tablet_server.cc:196] TabletServer@127.11.63.65:0 shutdown complete.
I20260709 13:22:49.290162 22863 consensus_queue.cc:1048] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:22:49.306151 22739 consensus_peers.cc:597] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:49.316097 22863 consensus_queue.cc:1048] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:22:49.323468 22871 consensus_queue.cc:1048] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:22:49.334970 22739 consensus_peers.cc:597] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:49.348531 22665 consensus_peers.cc:597] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:49.360421 22824 consensus_queue.cc:1048] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:22:49.377043 22665 consensus_peers.cc:597] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:49.454846 22571 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:22:49.573807 22739 consensus_peers.cc:597] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:50.278122 22871 raft_consensus.cc:493] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:50.278620 22871 raft_consensus.cc:515] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:50.280544 22871 leader_election.cc:290] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:50.281659 22774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a93e444ff9ec4a578e573e970e281e0a" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1" is_pre_election: true
W20260709 13:22:50.285375 22665 leader_election.cc:336] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111)
I20260709 13:22:50.285889 22665 leader_election.cc:304] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f7e755f96eda4b9ba37aa77b183f2a69; no voters: 55427c197ec4424580678498fbc616a2, c79ef076f43f4a73884174a86a6a10c1
I20260709 13:22:50.286579 22871 raft_consensus.cc:2749] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:22:50.364087 22818 raft_consensus.cc:493] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 55427c197ec4424580678498fbc616a2)
I20260709 13:22:50.364792 22818 raft_consensus.cc:515] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } }
I20260709 13:22:50.366753 22818 leader_election.cc:290] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617), 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635)
I20260709 13:22:50.367777 22697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a44aff4c84e4f05a00639dc531ecca5" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" is_pre_election: true
W20260709 13:22:50.370631 22739 leader_election.cc:336] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111)
I20260709 13:22:50.371105 22739 leader_election.cc:304] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1; no voters: 55427c197ec4424580678498fbc616a2, f7e755f96eda4b9ba37aa77b183f2a69
I20260709 13:22:50.371833 22818 raft_consensus.cc:2749] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:22:50.423650 22871 raft_consensus.cc:493] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 55427c197ec4424580678498fbc616a2)
I20260709 13:22:50.424340 22871 raft_consensus.cc:515] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } }
I20260709 13:22:50.426241 22871 leader_election.cc:290] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633), 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635)
I20260709 13:22:50.427179 22774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a44aff4c84e4f05a00639dc531ecca5" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1" is_pre_election: true
I20260709 13:22:50.427892 22774 raft_consensus.cc:2468] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 1.
I20260709 13:22:50.428911 22666 leader_election.cc:304] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 
I20260709 13:22:50.429596 22871 raft_consensus.cc:2804] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:50.429926 22871 raft_consensus.cc:493] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 55427c197ec4424580678498fbc616a2)
I20260709 13:22:50.430224 22871 raft_consensus.cc:3060] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:22:50.430680 22665 leader_election.cc:336] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111)
I20260709 13:22:50.436830 22871 raft_consensus.cc:515] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } }
I20260709 13:22:50.438619 22871 leader_election.cc:290] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 election: Requested vote from peers c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633), 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635)
I20260709 13:22:50.439570 22774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a44aff4c84e4f05a00639dc531ecca5" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
I20260709 13:22:50.440135 22774 raft_consensus.cc:3060] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:22:50.442245 22665 leader_election.cc:336] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111)
I20260709 13:22:50.445487 22774 raft_consensus.cc:2468] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 2.
I20260709 13:22:50.446615 22666 leader_election.cc:304] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 55427c197ec4424580678498fbc616a2
I20260709 13:22:50.447285 22871 raft_consensus.cc:2804] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:50.447681 22871 raft_consensus.cc:697] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 LEADER]: Becoming Leader. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Running, Role: LEADER
I20260709 13:22:50.448488 22871 consensus_queue.cc:237] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } }
W20260709 13:22:50.455802 22571 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:50.456928 22522 catalog_manager.cc:5697] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 reported cstate change: term changed from 1 to 2, leader changed from 55427c197ec4424580678498fbc616a2 (127.11.63.65) to f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66). New cstate: current_term: 2 leader_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:50.508922 22895 raft_consensus.cc:493] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 55427c197ec4424580678498fbc616a2)
I20260709 13:22:50.509431 22895 raft_consensus.cc:515] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:50.511485 22895 leader_election.cc:290] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:50.512744 22774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbcb0fb6f4e46b991774ca89f45da59" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1" is_pre_election: true
I20260709 13:22:50.513242 22818 raft_consensus.cc:493] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 55427c197ec4424580678498fbc616a2)
I20260709 13:22:50.513792 22818 raft_consensus.cc:515] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:50.515120 22774 raft_consensus.cc:2468] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 1.
I20260709 13:22:50.515853 22818 leader_election.cc:290] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617)
W20260709 13:22:50.516470 22665 leader_election.cc:336] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111)
I20260709 13:22:50.517535 22666 leader_election.cc:304] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 55427c197ec4424580678498fbc616a2
I20260709 13:22:50.517634 22699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbcb0fb6f4e46b991774ca89f45da59" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" is_pre_election: true
I20260709 13:22:50.518573 22699 raft_consensus.cc:2468] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 1.
I20260709 13:22:50.519022 22895 raft_consensus.cc:2804] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:50.519444 22895 raft_consensus.cc:493] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 55427c197ec4424580678498fbc616a2)
I20260709 13:22:50.519806 22737 leader_election.cc:304] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 
I20260709 13:22:50.519829 22895 raft_consensus.cc:3060] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:50.520664 22818 raft_consensus.cc:2804] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:22:50.520998 22739 leader_election.cc:336] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111)
I20260709 13:22:50.521070 22818 raft_consensus.cc:493] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 55427c197ec4424580678498fbc616a2)
I20260709 13:22:50.521646 22818 raft_consensus.cc:3060] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:50.525415 22895 raft_consensus.cc:515] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:50.526713 22818 raft_consensus.cc:515] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:50.527215 22895 leader_election.cc:290] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:50.528321 22774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbcb0fb6f4e46b991774ca89f45da59" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
I20260709 13:22:50.529093 22774 raft_consensus.cc:2393] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f7e755f96eda4b9ba37aa77b183f2a69 in current term 2: Already voted for candidate c79ef076f43f4a73884174a86a6a10c1 in this term.
I20260709 13:22:50.529467 22818 leader_election.cc:290] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 2 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617)
I20260709 13:22:50.529871 22699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbcb0fb6f4e46b991774ca89f45da59" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
I20260709 13:22:50.530560 22699 raft_consensus.cc:2393] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c79ef076f43f4a73884174a86a6a10c1 in current term 2: Already voted for candidate f7e755f96eda4b9ba37aa77b183f2a69 in this term.
W20260709 13:22:50.532048 22665 leader_election.cc:336] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111)
I20260709 13:22:50.532474 22665 leader_election.cc:304] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f7e755f96eda4b9ba37aa77b183f2a69; no voters: 55427c197ec4424580678498fbc616a2, c79ef076f43f4a73884174a86a6a10c1
W20260709 13:22:50.532720 22739 leader_election.cc:336] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111)
I20260709 13:22:50.533092 22895 raft_consensus.cc:2749] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:22:50.533190 22739 leader_election.cc:304] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1; no voters: 55427c197ec4424580678498fbc616a2, f7e755f96eda4b9ba37aa77b183f2a69
I20260709 13:22:50.534058 22818 raft_consensus.cc:2749] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:22:50.648767 22818 raft_consensus.cc:493] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 55427c197ec4424580678498fbc616a2)
I20260709 13:22:50.649221 22818 raft_consensus.cc:515] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:50.650951 22818 leader_election.cc:290] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617)
I20260709 13:22:50.652298 22699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a93e444ff9ec4a578e573e970e281e0a" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" is_pre_election: true
I20260709 13:22:50.653101 22699 raft_consensus.cc:2468] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 1.
I20260709 13:22:50.654193 22737 leader_election.cc:304] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 
I20260709 13:22:50.655042 22818 raft_consensus.cc:2804] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:50.655349 22818 raft_consensus.cc:493] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 55427c197ec4424580678498fbc616a2)
W20260709 13:22:50.655411 22739 leader_election.cc:336] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111)
I20260709 13:22:50.655663 22818 raft_consensus.cc:3060] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:50.660811 22818 raft_consensus.cc:515] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:50.662490 22818 leader_election.cc:290] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 2 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617)
I20260709 13:22:50.663568 22699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a93e444ff9ec4a578e573e970e281e0a" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
I20260709 13:22:50.664147 22699 raft_consensus.cc:3060] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:22:50.666083 22739 leader_election.cc:336] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111)
I20260709 13:22:50.669292 22699 raft_consensus.cc:2468] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 2.
I20260709 13:22:50.670423 22737 leader_election.cc:304] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 55427c197ec4424580678498fbc616a2
I20260709 13:22:50.671258 22818 raft_consensus.cc:2804] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:50.671742 22818 raft_consensus.cc:697] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 2 LEADER]: Becoming Leader. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Running, Role: LEADER
I20260709 13:22:50.672601 22818 consensus_queue.cc:237] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:50.680552 22522 catalog_manager.cc:5697] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 reported cstate change: term changed from 1 to 2, leader changed from 55427c197ec4424580678498fbc616a2 (127.11.63.65) to c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67). New cstate: current_term: 2 leader_uuid: "c79ef076f43f4a73884174a86a6a10c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:50.734468 22895 raft_consensus.cc:493] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:50.735914 22895 raft_consensus.cc:515] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:50.738520 22895 leader_election.cc:290] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:50.740998 22774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ccb054ff314be7a47b158bf4d141b6" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1" is_pre_election: true
I20260709 13:22:50.742605 22774 raft_consensus.cc:2468] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 1.
W20260709 13:22:50.744699 22665 leader_election.cc:336] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111)
I20260709 13:22:50.750452 22666 leader_election.cc:304] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 55427c197ec4424580678498fbc616a2
I20260709 13:22:50.754809 22895 raft_consensus.cc:2804] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:22:50.755213 22895 raft_consensus.cc:493] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:50.755606 22895 raft_consensus.cc:3060] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:50.764117 22895 raft_consensus.cc:515] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:50.766492 22895 leader_election.cc:290] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:50.768085 22774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ccb054ff314be7a47b158bf4d141b6" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
I20260709 13:22:50.769101 22774 raft_consensus.cc:3060] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:22:50.771409 22665 leader_election.cc:336] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111)
I20260709 13:22:50.780342 22774 raft_consensus.cc:2468] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 2.
I20260709 13:22:50.781780 22666 leader_election.cc:304] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 55427c197ec4424580678498fbc616a2
I20260709 13:22:50.782541 22895 raft_consensus.cc:2804] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:50.783044 22895 raft_consensus.cc:697] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 LEADER]: Becoming Leader. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Running, Role: LEADER
I20260709 13:22:50.784245 22895 consensus_queue.cc:237] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:50.794277 22522 catalog_manager.cc:5697] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 reported cstate change: term changed from 1 to 2, leader changed from 55427c197ec4424580678498fbc616a2 (127.11.63.65) to f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66). New cstate: current_term: 2 leader_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: UNKNOWN } } }
W20260709 13:22:50.879491 22665 consensus_peers.cc:597] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:50.895057 22774 raft_consensus.cc:1275] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Refusing update from remote peer f7e755f96eda4b9ba37aa77b183f2a69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:50.896646 22895 consensus_queue.cc:1048] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:22:51.207607 22699 raft_consensus.cc:1275] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Refusing update from remote peer c79ef076f43f4a73884174a86a6a10c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:51.209168 22832 consensus_queue.cc:1048] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:22:51.225059 22739 consensus_peers.cc:597] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:51.266588 22665 consensus_peers.cc:597] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:51.391031 22774 raft_consensus.cc:1275] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Refusing update from remote peer f7e755f96eda4b9ba37aa77b183f2a69: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:22:51.392304 22895 consensus_queue.cc:1048] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:22:51.457384 22571 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:22:51.750528 22739 consensus_peers.cc:597] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:51.777794 22665 consensus_peers.cc:597] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:51.964183 22739 consensus_peers.cc:597] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:52.018060 22665 consensus_peers.cc:597] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:52.031554 22739 consensus_peers.cc:597] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:52.070956 22739 consensus_peers.cc:597] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:22:52.099159 22909 raft_consensus.cc:493] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:22:52.099786 22909 raft_consensus.cc:515] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:52.102604 22909 leader_election.cc:290] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617)
I20260709 13:22:52.103891 22699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbcb0fb6f4e46b991774ca89f45da59" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" is_pre_election: true
I20260709 13:22:52.104667 22699 raft_consensus.cc:2468] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 2.
I20260709 13:22:52.106133 22737 leader_election.cc:304] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 
W20260709 13:22:52.106747 22739 leader_election.cc:336] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111)
I20260709 13:22:52.107059 22909 raft_consensus.cc:2804] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:22:52.107461 22909 raft_consensus.cc:493] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:22:52.107789 22909 raft_consensus.cc:3060] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:52.114559 22909 raft_consensus.cc:515] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:52.116291 22909 leader_election.cc:290] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 3 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617)
I20260709 13:22:52.117617 22699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbcb0fb6f4e46b991774ca89f45da59" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
I20260709 13:22:52.118307 22699 raft_consensus.cc:3060] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:22:52.120563 22739 leader_election.cc:336] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111)
I20260709 13:22:52.125361 22699 raft_consensus.cc:2468] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 3.
I20260709 13:22:52.126695 22737 leader_election.cc:304] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 55427c197ec4424580678498fbc616a2
I20260709 13:22:52.127393 22909 raft_consensus.cc:2804] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:52.128330 22909 raft_consensus.cc:697] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 3 LEADER]: Becoming Leader. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Running, Role: LEADER
I20260709 13:22:52.129482 22909 consensus_queue.cc:237] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:52.136731 22523 catalog_manager.cc:5697] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 reported cstate change: term changed from 1 to 3, leader changed from 55427c197ec4424580678498fbc616a2 (127.11.63.65) to c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67). New cstate: current_term: 3 leader_uuid: "c79ef076f43f4a73884174a86a6a10c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: HEALTHY } } }
W20260709 13:22:52.458593 22571 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:22:52.537041 22739 consensus_peers.cc:597] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:52.635988 22699 raft_consensus.cc:1275] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 FOLLOWER]: Refusing update from remote peer c79ef076f43f4a73884174a86a6a10c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:22:52.637733 22909 consensus_queue.cc:1048] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:22:53.459647 22571 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:22:53.623567 22739 consensus_peers.cc:597] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:53.628279 22665 consensus_peers.cc:597] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:53.643153 22665 consensus_peers.cc:597] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:22:54.150628 22665 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111) [suppressed 95 similar messages]
W20260709 13:22:54.153853 22665 consensus_peers.cc:597] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:54.208372 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:22:54.216749 22924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:54.217113 22925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:22:54.220860 22927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:22:54.224056 11517 server_base.cc:1061] running on GCE node
I20260709 13:22:54.225852 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:22:54.226143 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:22:54.226338 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603374226315 us; error 0 us; skew 500 ppm
I20260709 13:22:54.226962 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:22:54.229739 11517 webserver.cc:533] Webserver started at http://127.11.63.65:42533/ using document root <none> and password file <none>
I20260709 13:22:54.230311 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:22:54.230541 11517 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:22:54.234680 11517 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.000s	sys 0.004s
I20260709 13:22:54.239100 22932 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:22:54.240056 11517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:22:54.240326 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603019296230-11517-0/minicluster-data/ts-0-root
uuid: "55427c197ec4424580678498fbc616a2"
format_stamp: "Formatted at 2026-07-09 13:22:47 on dist-test-slave-qv7r"
I20260709 13:22:54.240643 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603019296230-11517-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603019296230-11517-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783603019296230-11517-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:22:54.259563 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:22:54.260900 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:22:54.263077 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:22:54.267750 22939 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 13:22:54.290789 22739 consensus_peers.cc:597] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:54.320788 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:22:54.321218 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.054s	user 0.001s	sys 0.001s
I20260709 13:22:54.321543 11517 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:22:54.326534 22939 tablet_bootstrap.cc:492] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2: Bootstrap starting.
I20260709 13:22:54.343253 22939 tablet_bootstrap.cc:492] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:22:54.344305 22939 tablet_bootstrap.cc:492] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2: Bootstrap complete.
I20260709 13:22:54.344962 22939 ts_tablet_manager.cc:1403] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.008s
I20260709 13:22:54.347826 22939 raft_consensus.cc:359] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.348778 22939 raft_consensus.cc:740] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55427c197ec4424580678498fbc616a2, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.349552 22939 consensus_queue.cc:260] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.352353 22939 ts_tablet_manager.cc:1434] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:22:54.353210 22939 tablet_bootstrap.cc:492] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2: Bootstrap starting.
I20260709 13:22:54.367743 22939 tablet_bootstrap.cc:492] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:54.368603 22939 tablet_bootstrap.cc:492] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2: Bootstrap complete.
I20260709 13:22:54.369303 22939 ts_tablet_manager.cc:1403] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2: Time spent bootstrapping tablet: real 0.016s	user 0.008s	sys 0.004s
I20260709 13:22:54.372021 22939 raft_consensus.cc:359] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.372740 22939 raft_consensus.cc:740] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55427c197ec4424580678498fbc616a2, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.373488 22939 consensus_queue.cc:260] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.375479 22939 ts_tablet_manager.cc:1434] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:22:54.376279 22939 tablet_bootstrap.cc:492] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2: Bootstrap starting.
I20260709 13:22:54.382776 11517 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:22:54.383163 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.062s	user 0.057s	sys 0.004s
I20260709 13:22:54.394817 22939 tablet_bootstrap.cc:492] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:54.396317 22939 tablet_bootstrap.cc:492] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2: Bootstrap complete.
I20260709 13:22:54.397099 22939 ts_tablet_manager.cc:1403] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.004s
I20260709 13:22:54.400625 22939 raft_consensus.cc:359] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.401535 22939 raft_consensus.cc:740] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55427c197ec4424580678498fbc616a2, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.402480 22939 consensus_queue.cc:260] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.405148 22939 ts_tablet_manager.cc:1434] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.005s
I20260709 13:22:54.406196 22939 tablet_bootstrap.cc:492] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2: Bootstrap starting.
W20260709 13:22:54.411264 22739 consensus_peers.cc:597] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:22:54.413995 22665 consensus_peers.cc:597] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:22:54.423535 22939 tablet_bootstrap.cc:492] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:22:54.424790 22939 tablet_bootstrap.cc:492] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2: Bootstrap complete.
I20260709 13:22:54.425577 22939 ts_tablet_manager.cc:1403] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.004s
I20260709 13:22:54.428928 22939 raft_consensus.cc:359] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.429962 22939 raft_consensus.cc:740] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55427c197ec4424580678498fbc616a2, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.430876 22939 consensus_queue.cc:260] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.433040 22939 ts_tablet_manager.cc:1434] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:22:54.434019 22939 tablet_bootstrap.cc:492] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2: Bootstrap starting.
I20260709 13:22:54.446192 23000 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.65:36635 every 8 connection(s)
I20260709 13:22:54.446251 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.65:36635
I20260709 13:22:54.449973 22939 tablet_bootstrap.cc:492] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:54.450991 22939 tablet_bootstrap.cc:492] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2: Bootstrap complete.
I20260709 13:22:54.451658 22939 ts_tablet_manager.cc:1403] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.005s
I20260709 13:22:54.452977 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:54.455369 22939 raft_consensus.cc:359] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.456364 22939 raft_consensus.cc:740] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55427c197ec4424580678498fbc616a2, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.457293 22939 consensus_queue.cc:260] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.459280 22939 ts_tablet_manager.cc:1434] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
W20260709 13:22:54.460807 22571 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:54.461618 22939 tablet_bootstrap.cc:492] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2: Bootstrap starting.
I20260709 13:22:54.486837 22939 tablet_bootstrap.cc:492] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:54.487824 22939 tablet_bootstrap.cc:492] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2: Bootstrap complete.
I20260709 13:22:54.488373 22939 ts_tablet_manager.cc:1403] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.009s
I20260709 13:22:54.491321 22939 raft_consensus.cc:359] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } }
I20260709 13:22:54.492045 22939 raft_consensus.cc:740] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55427c197ec4424580678498fbc616a2, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.492789 22939 consensus_queue.cc:260] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } }
I20260709 13:22:54.494956 22939 ts_tablet_manager.cc:1434] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.001s
I20260709 13:22:54.494714 22773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5cbcb0fb6f4e46b991774ca89f45da59"
dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
mode: GRACEFUL
new_leader_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
 from {username='slave'} at 127.0.0.1:41188
I20260709 13:22:54.495543 22773 raft_consensus.cc:606] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 3 LEADER]: Received request to transfer leadership to TS f7e755f96eda4b9ba37aa77b183f2a69
I20260709 13:22:54.495752 22939 tablet_bootstrap.cc:492] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2: Bootstrap starting.
I20260709 13:22:54.496325 23002 heartbeater.cc:344] Connected to a master server at 127.11.63.126:39291
I20260709 13:22:54.496788 23002 heartbeater.cc:461] Registering TS with master...
I20260709 13:22:54.497743 23002 heartbeater.cc:507] Master 127.11.63.126:39291 requested a full tablet report, sending...
I20260709 13:22:54.506300 22523 ts_manager.cc:194] Re-registered known tserver with Master: 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635)
I20260709 13:22:54.509527 22976 raft_consensus.cc:3060] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:54.512444 22939 tablet_bootstrap.cc:492] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:54.513528 22939 tablet_bootstrap.cc:492] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2: Bootstrap complete.
I20260709 13:22:54.514254 22939 ts_tablet_manager.cc:1403] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.003s
I20260709 13:22:54.516669 22699 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7505614384fc47b19d23973ba804ea5a"
dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
mode: GRACEFUL
new_leader_uuid: "c79ef076f43f4a73884174a86a6a10c1"
 from {username='slave'} at 127.0.0.1:37866
I20260709 13:22:54.517326 22699 raft_consensus.cc:606] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 LEADER]: Received request to transfer leadership to TS c79ef076f43f4a73884174a86a6a10c1
I20260709 13:22:54.517059 22939 raft_consensus.cc:359] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.517889 22939 raft_consensus.cc:740] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55427c197ec4424580678498fbc616a2, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.518652 22939 consensus_queue.cc:260] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.519804 22523 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55522
I20260709 13:22:54.520779 22939 ts_tablet_manager.cc:1434] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:22:54.521670 22939 tablet_bootstrap.cc:492] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2: Bootstrap starting.
I20260709 13:22:54.520779 22773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a93e444ff9ec4a578e573e970e281e0a"
dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
mode: GRACEFUL
new_leader_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
 from {username='slave'} at 127.0.0.1:41188
I20260709 13:22:54.522621 22773 raft_consensus.cc:606] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 2 LEADER]: Received request to transfer leadership to TS f7e755f96eda4b9ba37aa77b183f2a69
I20260709 13:22:54.523312 23002 heartbeater.cc:499] Master 127.11.63.126:39291 was elected leader, sending a full tablet report...
I20260709 13:22:54.525398 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:22:54.526095 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:54.526986 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:54.544602 22939 tablet_bootstrap.cc:492] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:54.545750 22939 tablet_bootstrap.cc:492] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2: Bootstrap complete.
I20260709 13:22:54.546419 22939 ts_tablet_manager.cc:1403] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.008s
I20260709 13:22:54.548419 22939 raft_consensus.cc:359] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.548894 22939 raft_consensus.cc:740] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55427c197ec4424580678498fbc616a2, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.549625 22939 consensus_queue.cc:260] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.551478 22939 ts_tablet_manager.cc:1434] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:22:54.552100 22939 tablet_bootstrap.cc:492] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2: Bootstrap starting.
I20260709 13:22:54.562577 22939 tablet_bootstrap.cc:492] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:54.563356 22939 tablet_bootstrap.cc:492] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2: Bootstrap complete.
I20260709 13:22:54.563943 22939 ts_tablet_manager.cc:1403] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:22:54.565903 22939 raft_consensus.cc:359] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.566413 22939 raft_consensus.cc:740] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55427c197ec4424580678498fbc616a2, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.567057 22939 consensus_queue.cc:260] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.568634 22939 ts_tablet_manager.cc:1434] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:22:54.569546 22939 tablet_bootstrap.cc:492] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2: Bootstrap starting.
I20260709 13:22:54.581602 22939 tablet_bootstrap.cc:492] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:22:54.582329 22939 tablet_bootstrap.cc:492] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2: Bootstrap complete.
I20260709 13:22:54.582904 22939 ts_tablet_manager.cc:1403] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.004s
I20260709 13:22:54.584754 22939 raft_consensus.cc:359] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.585248 22939 raft_consensus.cc:740] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55427c197ec4424580678498fbc616a2, State: Initialized, Role: FOLLOWER
I20260709 13:22:54.585937 22939 consensus_queue.cc:260] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.587445 22939 ts_tablet_manager.cc:1434] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:22:54.642251 22976 raft_consensus.cc:3060] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:54.649700 22976 pending_rounds.cc:85] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2: Aborting all ops after (but not including) 0
I20260709 13:22:54.649998 22976 pending_rounds.cc:107] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:22:54.650295 22976 raft_consensus.cc:2889] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:22:54.652314 22975 raft_consensus.cc:3060] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:22:54.652951 22974 raft_consensus.cc:3060] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:54.748283 23024 raft_consensus.cc:993] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1: : Instructing follower f7e755f96eda4b9ba37aa77b183f2a69 to start an election
I20260709 13:22:54.748996 22922 raft_consensus.cc:1081] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 2 LEADER]: Signalling peer f7e755f96eda4b9ba37aa77b183f2a69 to start an election
I20260709 13:22:54.750682 22922 raft_consensus.cc:993] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1: : Instructing follower f7e755f96eda4b9ba37aa77b183f2a69 to start an election
I20260709 13:22:54.751261 23025 raft_consensus.cc:1081] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 3 LEADER]: Signalling peer f7e755f96eda4b9ba37aa77b183f2a69 to start an election
I20260709 13:22:54.751363 22697 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a93e444ff9ec4a578e573e970e281e0a"
dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
 from {username='slave'} at 127.0.0.1:45770
I20260709 13:22:54.752228 22697 raft_consensus.cc:493] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:54.752668 22697 raft_consensus.cc:3060] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:54.753983 22699 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5cbcb0fb6f4e46b991774ca89f45da59"
dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
 from {username='slave'} at 127.0.0.1:45770
I20260709 13:22:54.754725 22699 raft_consensus.cc:493] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:54.755137 22699 raft_consensus.cc:3060] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:54.762246 22697 raft_consensus.cc:515] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.763453 22699 raft_consensus.cc:515] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.764830 22697 leader_election.cc:290] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 3 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:54.766036 22773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a93e444ff9ec4a578e573e970e281e0a" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
I20260709 13:22:54.766225 22699 leader_election.cc:290] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 4 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:54.766692 22773 raft_consensus.cc:3055] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:54.766551 22975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbcb0fb6f4e46b991774ca89f45da59" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "55427c197ec4424580678498fbc616a2"
I20260709 13:22:54.767135 22773 raft_consensus.cc:740] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Running, Role: LEADER
I20260709 13:22:54.767321 22975 raft_consensus.cc:3060] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:54.768020 22773 consensus_queue.cc:260] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.769098 22773 raft_consensus.cc:3060] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:54.771198 22774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbcb0fb6f4e46b991774ca89f45da59" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
I20260709 13:22:54.771936 22774 raft_consensus.cc:3055] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:22:54.771929 22974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a93e444ff9ec4a578e573e970e281e0a" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "55427c197ec4424580678498fbc616a2"
I20260709 13:22:54.772584 22774 raft_consensus.cc:740] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Running, Role: LEADER
I20260709 13:22:54.772805 22974 raft_consensus.cc:3060] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:54.773603 22774 consensus_queue.cc:260] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.774950 22774 raft_consensus.cc:3060] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:54.775502 22975 raft_consensus.cc:2468] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 4.
I20260709 13:22:54.776816 22665 leader_election.cc:304] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55427c197ec4424580678498fbc616a2, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 
I20260709 13:22:54.777500 22895 raft_consensus.cc:2804] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:22:54.777894 22773 raft_consensus.cc:2468] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 3.
I20260709 13:22:54.779192 22895 raft_consensus.cc:697] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 4 LEADER]: Becoming Leader. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Running, Role: LEADER
I20260709 13:22:54.779452 22666 leader_election.cc:304] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 
I20260709 13:22:54.780612 23026 raft_consensus.cc:2804] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:54.780323 22895 consensus_queue.cc:237] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.783167 22774 raft_consensus.cc:2468] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 4.
I20260709 13:22:54.783895 22974 raft_consensus.cc:2468] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 3.
I20260709 13:22:54.785374 23026 raft_consensus.cc:697] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 LEADER]: Becoming Leader. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Running, Role: LEADER
I20260709 13:22:54.786595 23026 consensus_queue.cc:237] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.791250 23027 raft_consensus.cc:993] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69: : Instructing follower c79ef076f43f4a73884174a86a6a10c1 to start an election
I20260709 13:22:54.791851 23027 raft_consensus.cc:1081] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 LEADER]: Signalling peer c79ef076f43f4a73884174a86a6a10c1 to start an election
I20260709 13:22:54.792222 22520 catalog_manager.cc:5697] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 reported cstate change: term changed from 3 to 4, leader changed from c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67) to f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66). New cstate: current_term: 4 leader_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:54.793897 22774 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7505614384fc47b19d23973ba804ea5a"
dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
 from {username='slave'} at 127.0.0.1:41086
I20260709 13:22:54.794617 22774 raft_consensus.cc:493] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:54.795020 22774 raft_consensus.cc:3060] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:54.802644 22774 raft_consensus.cc:515] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.804809 22774 leader_election.cc:290] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 2 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617)
I20260709 13:22:54.806082 22974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7505614384fc47b19d23973ba804ea5a" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "55427c197ec4424580678498fbc616a2"
I20260709 13:22:54.806041 22699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7505614384fc47b19d23973ba804ea5a" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
I20260709 13:22:54.807013 22974 raft_consensus.cc:3060] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:54.807073 22699 raft_consensus.cc:3055] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:54.807581 22699 raft_consensus.cc:740] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Running, Role: LEADER
I20260709 13:22:54.808612 22699 consensus_queue.cc:260] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.810074 22699 raft_consensus.cc:3060] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:54.814070 22520 catalog_manager.cc:5697] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 reported cstate change: term changed from 2 to 3, leader changed from c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67) to f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66). New cstate: current_term: 3 leader_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:54.815881 22974 raft_consensus.cc:2468] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 2.
I20260709 13:22:54.817250 22739 leader_election.cc:304] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55427c197ec4424580678498fbc616a2, c79ef076f43f4a73884174a86a6a10c1; no voters: 
I20260709 13:22:54.818109 23025 raft_consensus.cc:2804] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:54.818603 23025 raft_consensus.cc:697] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 2 LEADER]: Becoming Leader. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Running, Role: LEADER
I20260709 13:22:54.818832 22699 raft_consensus.cc:2468] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 2.
I20260709 13:22:54.819717 23025 consensus_queue.cc:237] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:54.829700 22520 catalog_manager.cc:5697] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 reported cstate change: term changed from 1 to 2, leader changed from f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66) to c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67). New cstate: current_term: 2 leader_uuid: "c79ef076f43f4a73884174a86a6a10c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:55.203779 22974 raft_consensus.cc:1275] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 3 FOLLOWER]: Refusing update from remote peer f7e755f96eda4b9ba37aa77b183f2a69: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:55.205585 22895 consensus_queue.cc:1048] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:55.221045 22774 raft_consensus.cc:1275] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 3 FOLLOWER]: Refusing update from remote peer f7e755f96eda4b9ba37aa77b183f2a69: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:55.222851 23026 consensus_queue.cc:1048] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:55.272409 22774 raft_consensus.cc:1275] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 4 FOLLOWER]: Refusing update from remote peer f7e755f96eda4b9ba37aa77b183f2a69: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:22:55.273983 22895 consensus_queue.cc:1048] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:55.279697 22699 raft_consensus.cc:1275] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Refusing update from remote peer c79ef076f43f4a73884174a86a6a10c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:55.281092 22922 consensus_queue.cc:1048] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:55.287532 22974 raft_consensus.cc:1275] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [term 4 FOLLOWER]: Refusing update from remote peer f7e755f96eda4b9ba37aa77b183f2a69: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:22:55.290377 23041 consensus_queue.cc:1048] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:55.299572 22974 raft_consensus.cc:1275] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [term 2 FOLLOWER]: Refusing update from remote peer c79ef076f43f4a73884174a86a6a10c1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:55.302109 23025 consensus_queue.cc:1048] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:22:55.461967 22571 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:55.541530 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:55.550195 22699 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5cbcb0fb6f4e46b991774ca89f45da59"
dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
mode: GRACEFUL
new_leader_uuid: "c79ef076f43f4a73884174a86a6a10c1"
 from {username='slave'} at 127.0.0.1:37866
I20260709 13:22:55.550772 22699 raft_consensus.cc:606] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 4 LEADER]: Received request to transfer leadership to TS c79ef076f43f4a73884174a86a6a10c1
I20260709 13:22:55.553831 22773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7505614384fc47b19d23973ba804ea5a"
dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
mode: GRACEFUL
new_leader_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
 from {username='slave'} at 127.0.0.1:41188
I20260709 13:22:55.554359 22773 raft_consensus.cc:606] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 2 LEADER]: Received request to transfer leadership to TS f7e755f96eda4b9ba37aa77b183f2a69
I20260709 13:22:55.557322 22699 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a93e444ff9ec4a578e573e970e281e0a"
dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
mode: GRACEFUL
new_leader_uuid: "c79ef076f43f4a73884174a86a6a10c1"
 from {username='slave'} at 127.0.0.1:37866
I20260709 13:22:55.558003 22699 raft_consensus.cc:606] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 LEADER]: Received request to transfer leadership to TS c79ef076f43f4a73884174a86a6a10c1
I20260709 13:22:55.559228 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:22:55.559845 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:55.560410 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:55.710877 22895 raft_consensus.cc:993] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69: : Instructing follower c79ef076f43f4a73884174a86a6a10c1 to start an election
I20260709 13:22:55.711481 23026 raft_consensus.cc:1081] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 4 LEADER]: Signalling peer c79ef076f43f4a73884174a86a6a10c1 to start an election
I20260709 13:22:55.713183 22773 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5cbcb0fb6f4e46b991774ca89f45da59"
dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
 from {username='slave'} at 127.0.0.1:41086
I20260709 13:22:55.713827 22773 raft_consensus.cc:493] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:55.714421 22773 raft_consensus.cc:3060] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:22:55.719079 23041 raft_consensus.cc:993] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69: : Instructing follower c79ef076f43f4a73884174a86a6a10c1 to start an election
I20260709 13:22:55.719669 22895 raft_consensus.cc:1081] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 LEADER]: Signalling peer c79ef076f43f4a73884174a86a6a10c1 to start an election
I20260709 13:22:55.721582 22774 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a93e444ff9ec4a578e573e970e281e0a"
dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
 from {username='slave'} at 127.0.0.1:41086
I20260709 13:22:55.722211 22774 raft_consensus.cc:493] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:55.722591 22774 raft_consensus.cc:3060] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:55.722431 22773 raft_consensus.cc:515] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:55.724836 22773 leader_election.cc:290] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 5 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617)
I20260709 13:22:55.725425 22699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbcb0fb6f4e46b991774ca89f45da59" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
I20260709 13:22:55.725795 22974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbcb0fb6f4e46b991774ca89f45da59" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "55427c197ec4424580678498fbc616a2"
I20260709 13:22:55.726188 22699 raft_consensus.cc:3055] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:22:55.726414 22974 raft_consensus.cc:3060] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:22:55.726629 22699 raft_consensus.cc:740] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Running, Role: LEADER
I20260709 13:22:55.727516 22699 consensus_queue.cc:260] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:55.728753 22699 raft_consensus.cc:3060] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:22:55.730432 22774 raft_consensus.cc:515] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:55.732374 22974 raft_consensus.cc:2468] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 5.
I20260709 13:22:55.733246 22975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a93e444ff9ec4a578e573e970e281e0a" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "55427c197ec4424580678498fbc616a2"
I20260709 13:22:55.733999 22697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a93e444ff9ec4a578e573e970e281e0a" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
I20260709 13:22:55.733341 22774 leader_election.cc:290] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 4 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617)
I20260709 13:22:55.734058 22975 raft_consensus.cc:3060] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:55.734928 22739 leader_election.cc:304] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55427c197ec4424580678498fbc616a2, c79ef076f43f4a73884174a86a6a10c1; no voters: 
I20260709 13:22:55.736100 23025 raft_consensus.cc:2804] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:22:55.737241 23025 raft_consensus.cc:697] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 5 LEADER]: Becoming Leader. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Running, Role: LEADER
I20260709 13:22:55.738062 23025 consensus_queue.cc:237] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:55.734726 22697 raft_consensus.cc:3055] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:22:55.739132 22697 raft_consensus.cc:740] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Running, Role: LEADER
I20260709 13:22:55.739907 22697 consensus_queue.cc:260] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:55.740636 22699 raft_consensus.cc:2468] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 5.
I20260709 13:22:55.741276 22697 raft_consensus.cc:3060] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:55.745383 22975 raft_consensus.cc:2468] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 4.
I20260709 13:22:55.746592 22739 leader_election.cc:304] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55427c197ec4424580678498fbc616a2, c79ef076f43f4a73884174a86a6a10c1; no voters: 
I20260709 13:22:55.746497 22522 catalog_manager.cc:5697] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 reported cstate change: term changed from 4 to 5, leader changed from f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66) to c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67). New cstate: current_term: 5 leader_uuid: "c79ef076f43f4a73884174a86a6a10c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:55.747401 23025 raft_consensus.cc:2804] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:22:55.747605 22697 raft_consensus.cc:2468] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 4.
I20260709 13:22:55.748217 23025 raft_consensus.cc:697] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 4 LEADER]: Becoming Leader. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Running, Role: LEADER
I20260709 13:22:55.749058 23025 consensus_queue.cc:237] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:55.762748 22522 catalog_manager.cc:5697] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 reported cstate change: term changed from 3 to 4, leader changed from f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66) to c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67). New cstate: current_term: 4 leader_uuid: "c79ef076f43f4a73884174a86a6a10c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:55.764817 22922 raft_consensus.cc:993] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1: : Instructing follower f7e755f96eda4b9ba37aa77b183f2a69 to start an election
I20260709 13:22:55.765463 22922 raft_consensus.cc:1081] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 2 LEADER]: Signalling peer f7e755f96eda4b9ba37aa77b183f2a69 to start an election
I20260709 13:22:55.767364 22697 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7505614384fc47b19d23973ba804ea5a"
dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
 from {username='slave'} at 127.0.0.1:45770
I20260709 13:22:55.767908 22697 raft_consensus.cc:493] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:55.768371 22697 raft_consensus.cc:3060] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:55.775121 22697 raft_consensus.cc:515] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:55.777153 22697 leader_election.cc:290] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 3 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:55.778035 22975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7505614384fc47b19d23973ba804ea5a" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "55427c197ec4424580678498fbc616a2"
I20260709 13:22:55.778239 22774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7505614384fc47b19d23973ba804ea5a" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
I20260709 13:22:55.778759 22975 raft_consensus.cc:3060] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:55.778896 22774 raft_consensus.cc:3055] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:55.779314 22774 raft_consensus.cc:740] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Running, Role: LEADER
I20260709 13:22:55.780128 22774 consensus_queue.cc:260] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:55.781405 22774 raft_consensus.cc:3060] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:55.786625 22975 raft_consensus.cc:2468] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 3.
I20260709 13:22:55.788012 22665 leader_election.cc:304] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55427c197ec4424580678498fbc616a2, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 
I20260709 13:22:55.788786 22774 raft_consensus.cc:2468] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 3.
I20260709 13:22:55.788931 23041 raft_consensus.cc:2804] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:55.789544 23041 raft_consensus.cc:697] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 LEADER]: Becoming Leader. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Running, Role: LEADER
I20260709 13:22:55.790450 23041 consensus_queue.cc:237] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:55.796932 22522 catalog_manager.cc:5697] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 reported cstate change: term changed from 2 to 3, leader changed from c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67) to f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66). New cstate: current_term: 3 leader_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:56.145825 22975 raft_consensus.cc:1275] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [term 5 FOLLOWER]: Refusing update from remote peer c79ef076f43f4a73884174a86a6a10c1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:22:56.147210 23025 consensus_queue.cc:1048] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:22:56.157312 22697 raft_consensus.cc:1275] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 5 FOLLOWER]: Refusing update from remote peer c79ef076f43f4a73884174a86a6a10c1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:22:56.158981 23025 consensus_queue.cc:1048] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:22:56.195807 22974 raft_consensus.cc:1275] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [term 3 FOLLOWER]: Refusing update from remote peer f7e755f96eda4b9ba37aa77b183f2a69: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:56.197355 23041 consensus_queue.cc:1048] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:56.206095 22774 raft_consensus.cc:1275] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 3 FOLLOWER]: Refusing update from remote peer f7e755f96eda4b9ba37aa77b183f2a69: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:56.207901 23026 consensus_queue.cc:1048] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:56.283850 22974 raft_consensus.cc:1275] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 4 FOLLOWER]: Refusing update from remote peer c79ef076f43f4a73884174a86a6a10c1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:22:56.285136 23025 consensus_queue.cc:1048] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:22:56.295244 22697 raft_consensus.cc:1275] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 4 FOLLOWER]: Refusing update from remote peer c79ef076f43f4a73884174a86a6a10c1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:22:56.296945 22922 consensus_queue.cc:1048] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:22:56.463174 22571 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:56.568286 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:56.576979 22697 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7505614384fc47b19d23973ba804ea5a"
dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
mode: GRACEFUL
new_leader_uuid: "c79ef076f43f4a73884174a86a6a10c1"
 from {username='slave'} at 127.0.0.1:37866
I20260709 13:22:56.577554 22697 raft_consensus.cc:606] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 LEADER]: Received request to transfer leadership to TS c79ef076f43f4a73884174a86a6a10c1
I20260709 13:22:56.580657 22773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e8e668ce13b473186eed6b7c92dbf8c"
dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
mode: GRACEFUL
new_leader_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
 from {username='slave'} at 127.0.0.1:41188
I20260709 13:22:56.581143 22773 raft_consensus.cc:606] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 1 LEADER]: Received request to transfer leadership to TS f7e755f96eda4b9ba37aa77b183f2a69
I20260709 13:22:56.584048 22773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2d10d445a81457f817ac6b4bfcb77c7"
dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
mode: GRACEFUL
new_leader_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
 from {username='slave'} at 127.0.0.1:41188
I20260709 13:22:56.584604 22773 raft_consensus.cc:606] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [term 1 LEADER]: Received request to transfer leadership to TS f7e755f96eda4b9ba37aa77b183f2a69
I20260709 13:22:56.585852 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:22:56.586313 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:56.586835 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:56.694423 23026 raft_consensus.cc:993] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69: : Instructing follower c79ef076f43f4a73884174a86a6a10c1 to start an election
I20260709 13:22:56.695037 22895 raft_consensus.cc:1081] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 LEADER]: Signalling peer c79ef076f43f4a73884174a86a6a10c1 to start an election
I20260709 13:22:56.696544 22773 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7505614384fc47b19d23973ba804ea5a"
dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
 from {username='slave'} at 127.0.0.1:41086
I20260709 13:22:56.697160 22773 raft_consensus.cc:493] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:56.697535 22773 raft_consensus.cc:3060] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:56.702608 22773 raft_consensus.cc:515] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:56.704351 22773 leader_election.cc:290] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 4 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617)
I20260709 13:22:56.705127 22975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7505614384fc47b19d23973ba804ea5a" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "55427c197ec4424580678498fbc616a2"
I20260709 13:22:56.705312 22697 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7505614384fc47b19d23973ba804ea5a" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
I20260709 13:22:56.705962 22975 raft_consensus.cc:3060] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:56.706058 22697 raft_consensus.cc:3055] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:22:56.706588 22697 raft_consensus.cc:740] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Running, Role: LEADER
I20260709 13:22:56.707253 22697 consensus_queue.cc:260] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:56.708191 22697 raft_consensus.cc:3060] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:56.711483 22975 raft_consensus.cc:2468] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 4.
I20260709 13:22:56.712646 22739 leader_election.cc:304] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55427c197ec4424580678498fbc616a2, c79ef076f43f4a73884174a86a6a10c1; no voters: 
I20260709 13:22:56.713465 22922 raft_consensus.cc:2804] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:22:56.713726 22697 raft_consensus.cc:2468] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 4.
I20260709 13:22:56.714064 22922 raft_consensus.cc:697] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 4 LEADER]: Becoming Leader. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Running, Role: LEADER
I20260709 13:22:56.715183 22922 consensus_queue.cc:237] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:56.722078 22522 catalog_manager.cc:5697] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 reported cstate change: term changed from 3 to 4, leader changed from f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66) to c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67). New cstate: current_term: 4 leader_uuid: "c79ef076f43f4a73884174a86a6a10c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:56.795693 23025 raft_consensus.cc:993] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1: : Instructing follower f7e755f96eda4b9ba37aa77b183f2a69 to start an election
I20260709 13:22:56.796151 22922 raft_consensus.cc:1081] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [term 1 LEADER]: Signalling peer f7e755f96eda4b9ba37aa77b183f2a69 to start an election
I20260709 13:22:56.797801 22697 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2d10d445a81457f817ac6b4bfcb77c7"
dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
 from {username='slave'} at 127.0.0.1:45770
I20260709 13:22:56.798288 22697 raft_consensus.cc:493] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:56.798547 22697 raft_consensus.cc:3060] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:56.803763 22697 raft_consensus.cc:515] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:56.805366 22697 leader_election.cc:290] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:56.806321 22975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d10d445a81457f817ac6b4bfcb77c7" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "55427c197ec4424580678498fbc616a2"
I20260709 13:22:56.806411 22773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2d10d445a81457f817ac6b4bfcb77c7" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
I20260709 13:22:56.806932 22975 raft_consensus.cc:3060] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:56.807178 22773 raft_consensus.cc:3055] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:56.807564 22773 raft_consensus.cc:740] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Running, Role: LEADER
I20260709 13:22:56.808351 22773 consensus_queue.cc:260] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:56.809326 22773 raft_consensus.cc:3060] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:56.812266 22975 raft_consensus.cc:2468] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 2.
I20260709 13:22:56.813493 22665 leader_election.cc:304] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55427c197ec4424580678498fbc616a2, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 
I20260709 13:22:56.814296 22895 raft_consensus.cc:2804] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:56.814811 22895 raft_consensus.cc:697] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 LEADER]: Becoming Leader. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Running, Role: LEADER
I20260709 13:22:56.814746 22773 raft_consensus.cc:2468] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 2.
I20260709 13:22:56.815902 22895 consensus_queue.cc:237] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:56.823179 22522 catalog_manager.cc:5697] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69 reported cstate change: term changed from 1 to 2, leader changed from c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67) to f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66). New cstate: current_term: 2 leader_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:56.832762 23025 raft_consensus.cc:993] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1: : Instructing follower f7e755f96eda4b9ba37aa77b183f2a69 to start an election
I20260709 13:22:56.833285 22922 raft_consensus.cc:1081] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 1 LEADER]: Signalling peer f7e755f96eda4b9ba37aa77b183f2a69 to start an election
I20260709 13:22:56.834800 22697 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e8e668ce13b473186eed6b7c92dbf8c"
dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
 from {username='slave'} at 127.0.0.1:45770
I20260709 13:22:56.835247 22697 raft_consensus.cc:493] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:56.835510 22697 raft_consensus.cc:3060] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:56.841120 22697 raft_consensus.cc:515] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:56.842986 22697 leader_election.cc:290] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:56.843993 22975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e8e668ce13b473186eed6b7c92dbf8c" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "55427c197ec4424580678498fbc616a2"
I20260709 13:22:56.844242 22773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e8e668ce13b473186eed6b7c92dbf8c" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
I20260709 13:22:56.844770 22975 raft_consensus.cc:3060] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:56.844844 22773 raft_consensus.cc:3055] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:56.845263 22773 raft_consensus.cc:740] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Running, Role: LEADER
I20260709 13:22:56.846099 22773 consensus_queue.cc:260] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:56.847106 22773 raft_consensus.cc:3060] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:56.850816 22975 raft_consensus.cc:2468] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 2.
I20260709 13:22:56.852073 22665 leader_election.cc:304] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55427c197ec4424580678498fbc616a2, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 
I20260709 13:22:56.852248 22773 raft_consensus.cc:2468] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 2.
I20260709 13:22:56.852876 22895 raft_consensus.cc:2804] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:56.853408 22895 raft_consensus.cc:697] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 LEADER]: Becoming Leader. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Running, Role: LEADER
I20260709 13:22:56.854322 22895 consensus_queue.cc:237] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:56.861011 22522 catalog_manager.cc:5697] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 reported cstate change: term changed from 1 to 2, leader changed from c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67) to f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66). New cstate: current_term: 2 leader_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:57.223889 22975 raft_consensus.cc:1275] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [term 4 FOLLOWER]: Refusing update from remote peer c79ef076f43f4a73884174a86a6a10c1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:22:57.225348 22922 consensus_queue.cc:1048] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:22:57.225822 22697 raft_consensus.cc:1275] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 4 FOLLOWER]: Refusing update from remote peer c79ef076f43f4a73884174a86a6a10c1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:22:57.227309 22922 consensus_queue.cc:1048] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:22:57.258453 22773 raft_consensus.cc:1275] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Refusing update from remote peer f7e755f96eda4b9ba37aa77b183f2a69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:57.260133 22895 consensus_queue.cc:1048] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:57.272114 22975 raft_consensus.cc:1275] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [term 2 FOLLOWER]: Refusing update from remote peer f7e755f96eda4b9ba37aa77b183f2a69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:57.274410 22895 consensus_queue.cc:1048] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:57.317814 22974 raft_consensus.cc:1275] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2 [term 2 FOLLOWER]: Refusing update from remote peer f7e755f96eda4b9ba37aa77b183f2a69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:57.319195 22895 consensus_queue.cc:1048] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:57.328627 22773 raft_consensus.cc:1275] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Refusing update from remote peer f7e755f96eda4b9ba37aa77b183f2a69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:57.330698 23026 consensus_queue.cc:1048] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:22:57.464112 22571 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:57.595341 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:57.603830 22699 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a44aff4c84e4f05a00639dc531ecca5"
dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
mode: GRACEFUL
new_leader_uuid: "c79ef076f43f4a73884174a86a6a10c1"
 from {username='slave'} at 127.0.0.1:37866
I20260709 13:22:57.604377 22699 raft_consensus.cc:606] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 LEADER]: Received request to transfer leadership to TS c79ef076f43f4a73884174a86a6a10c1
I20260709 13:22:57.607563 22699 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e8e668ce13b473186eed6b7c92dbf8c"
dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
mode: GRACEFUL
new_leader_uuid: "c79ef076f43f4a73884174a86a6a10c1"
 from {username='slave'} at 127.0.0.1:37866
I20260709 13:22:57.608035 22699 raft_consensus.cc:606] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 LEADER]: Received request to transfer leadership to TS c79ef076f43f4a73884174a86a6a10c1
I20260709 13:22:57.610718 22773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "add99f7fe29440ab82c13df2e1bc541d"
dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
mode: GRACEFUL
new_leader_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
 from {username='slave'} at 127.0.0.1:41188
I20260709 13:22:57.611318 22773 raft_consensus.cc:606] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [term 1 LEADER]: Received request to transfer leadership to TS f7e755f96eda4b9ba37aa77b183f2a69
I20260709 13:22:57.612648 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:22:57.613180 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:57.613788 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:57.656628 23024 raft_consensus.cc:993] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1: : Instructing follower f7e755f96eda4b9ba37aa77b183f2a69 to start an election
I20260709 13:22:57.657294 23025 raft_consensus.cc:1081] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [term 1 LEADER]: Signalling peer f7e755f96eda4b9ba37aa77b183f2a69 to start an election
I20260709 13:22:57.658970 22699 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "add99f7fe29440ab82c13df2e1bc541d"
dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
 from {username='slave'} at 127.0.0.1:45770
I20260709 13:22:57.659595 22699 raft_consensus.cc:493] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:57.659947 22699 raft_consensus.cc:3060] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:57.665072 22699 raft_consensus.cc:515] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:57.666591 22699 leader_election.cc:290] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:57.667652 22974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "add99f7fe29440ab82c13df2e1bc541d" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "55427c197ec4424580678498fbc616a2"
I20260709 13:22:57.667652 22773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "add99f7fe29440ab82c13df2e1bc541d" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
I20260709 13:22:57.668365 22974 raft_consensus.cc:3060] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:57.668538 22773 raft_consensus.cc:3055] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:22:57.668977 22773 raft_consensus.cc:740] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Running, Role: LEADER
I20260709 13:22:57.669823 22773 consensus_queue.cc:260] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:57.670773 22773 raft_consensus.cc:3060] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:22:57.674459 22974 raft_consensus.cc:2468] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 2.
I20260709 13:22:57.675572 22665 leader_election.cc:304] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55427c197ec4424580678498fbc616a2, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 
I20260709 13:22:57.676043 22773 raft_consensus.cc:2468] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 2.
I20260709 13:22:57.676281 23026 raft_consensus.cc:2804] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:22:57.676864 23026 raft_consensus.cc:697] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 LEADER]: Becoming Leader. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Running, Role: LEADER
I20260709 13:22:57.677780 23026 consensus_queue.cc:237] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:57.684808 22522 catalog_manager.cc:5697] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69 reported cstate change: term changed from 1 to 2, leader changed from c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67) to f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66). New cstate: current_term: 2 leader_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:57.888744 23066 raft_consensus.cc:993] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69: : Instructing follower c79ef076f43f4a73884174a86a6a10c1 to start an election
I20260709 13:22:57.889483 23026 raft_consensus.cc:1081] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 LEADER]: Signalling peer c79ef076f43f4a73884174a86a6a10c1 to start an election
I20260709 13:22:57.891263 22773 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e8e668ce13b473186eed6b7c92dbf8c"
dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
 from {username='slave'} at 127.0.0.1:41086
I20260709 13:22:57.892107 22773 raft_consensus.cc:493] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:57.892437 22773 raft_consensus.cc:3060] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:57.897823 22773 raft_consensus.cc:515] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:57.899776 22773 leader_election.cc:290] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 3 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617)
I20260709 13:22:57.900550 22974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e8e668ce13b473186eed6b7c92dbf8c" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "55427c197ec4424580678498fbc616a2"
I20260709 13:22:57.900887 22699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e8e668ce13b473186eed6b7c92dbf8c" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
I20260709 13:22:57.901235 22974 raft_consensus.cc:3060] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:57.901520 22699 raft_consensus.cc:3055] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:57.901933 22699 raft_consensus.cc:740] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Running, Role: LEADER
I20260709 13:22:57.902626 22699 consensus_queue.cc:260] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:57.903630 22699 raft_consensus.cc:3060] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:57.906783 22974 raft_consensus.cc:2468] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 3.
I20260709 13:22:57.908120 22739 leader_election.cc:304] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55427c197ec4424580678498fbc616a2, c79ef076f43f4a73884174a86a6a10c1; no voters: 
I20260709 13:22:57.909053 23025 raft_consensus.cc:2804] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:57.909518 23025 raft_consensus.cc:697] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 3 LEADER]: Becoming Leader. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Running, Role: LEADER
I20260709 13:22:57.909565 22699 raft_consensus.cc:2468] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 3.
I20260709 13:22:57.910416 23025 consensus_queue.cc:237] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:57.916744 22522 catalog_manager.cc:5697] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 reported cstate change: term changed from 2 to 3, leader changed from f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66) to c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67). New cstate: current_term: 3 leader_uuid: "c79ef076f43f4a73884174a86a6a10c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:57.964146 23026 raft_consensus.cc:993] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69: : Instructing follower c79ef076f43f4a73884174a86a6a10c1 to start an election
I20260709 13:22:57.965044 23066 raft_consensus.cc:1081] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 LEADER]: Signalling peer c79ef076f43f4a73884174a86a6a10c1 to start an election
I20260709 13:22:57.966754 22773 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a44aff4c84e4f05a00639dc531ecca5"
dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
 from {username='slave'} at 127.0.0.1:41086
I20260709 13:22:57.967417 22773 raft_consensus.cc:493] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:57.967744 22773 raft_consensus.cc:3060] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:57.972957 22773 raft_consensus.cc:515] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } }
I20260709 13:22:57.974644 22773 leader_election.cc:290] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 3 election: Requested vote from peers f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617), 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635)
I20260709 13:22:57.975692 22699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a44aff4c84e4f05a00639dc531ecca5" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
I20260709 13:22:57.976305 22699 raft_consensus.cc:3055] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:57.975728 22974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a44aff4c84e4f05a00639dc531ecca5" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "55427c197ec4424580678498fbc616a2"
I20260709 13:22:57.976593 22699 raft_consensus.cc:740] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Running, Role: LEADER
I20260709 13:22:57.976861 22974 raft_consensus.cc:3060] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:57.977424 22699 consensus_queue.cc:260] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } }
I20260709 13:22:57.978529 22699 raft_consensus.cc:3060] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:57.983263 22974 raft_consensus.cc:2468] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 3.
I20260709 13:22:57.983278 22699 raft_consensus.cc:2468] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 3.
I20260709 13:22:57.984746 22739 leader_election.cc:304] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55427c197ec4424580678498fbc616a2, c79ef076f43f4a73884174a86a6a10c1; no voters: 
I20260709 13:22:57.985517 23024 raft_consensus.cc:2804] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:57.986068 23024 raft_consensus.cc:697] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 3 LEADER]: Becoming Leader. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Running, Role: LEADER
I20260709 13:22:57.987031 23024 consensus_queue.cc:237] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } }
I20260709 13:22:57.994082 22522 catalog_manager.cc:5697] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 reported cstate change: term changed from 2 to 3, leader changed from f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66) to c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67). New cstate: current_term: 3 leader_uuid: "c79ef076f43f4a73884174a86a6a10c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:58.254279 22773 raft_consensus.cc:1275] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Refusing update from remote peer f7e755f96eda4b9ba37aa77b183f2a69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:58.255748 23066 consensus_queue.cc:1048] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:58.261947 22974 raft_consensus.cc:1275] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2 [term 2 FOLLOWER]: Refusing update from remote peer f7e755f96eda4b9ba37aa77b183f2a69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:22:58.263564 22895 consensus_queue.cc:1048] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:58.315070 22699 raft_consensus.cc:1275] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 FOLLOWER]: Refusing update from remote peer c79ef076f43f4a73884174a86a6a10c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:58.316427 23024 consensus_queue.cc:1048] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:58.326129 22974 raft_consensus.cc:1275] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2 [term 3 FOLLOWER]: Refusing update from remote peer c79ef076f43f4a73884174a86a6a10c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:58.327850 23024 consensus_queue.cc:1048] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:58.372087 22699 raft_consensus.cc:1275] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 FOLLOWER]: Refusing update from remote peer c79ef076f43f4a73884174a86a6a10c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:58.373661 23024 consensus_queue.cc:1048] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:22:58.384645 22974 raft_consensus.cc:1275] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [term 3 FOLLOWER]: Refusing update from remote peer c79ef076f43f4a73884174a86a6a10c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:22:58.386037 23025 consensus_queue.cc:1048] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:22:58.465142 22571 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:58.620851 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:58.628255 22699 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "11ccb054ff314be7a47b158bf4d141b6"
dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
mode: GRACEFUL
new_leader_uuid: "c79ef076f43f4a73884174a86a6a10c1"
 from {username='slave'} at 127.0.0.1:37866
I20260709 13:22:58.628777 22699 raft_consensus.cc:606] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 LEADER]: Received request to transfer leadership to TS c79ef076f43f4a73884174a86a6a10c1
I20260709 13:22:58.631461 22773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a44aff4c84e4f05a00639dc531ecca5"
dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
mode: GRACEFUL
new_leader_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
 from {username='slave'} at 127.0.0.1:41188
I20260709 13:22:58.631951 22773 raft_consensus.cc:606] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 3 LEADER]: Received request to transfer leadership to TS f7e755f96eda4b9ba37aa77b183f2a69
I20260709 13:22:58.634723 22773 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7505614384fc47b19d23973ba804ea5a"
dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
mode: GRACEFUL
new_leader_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
 from {username='slave'} at 127.0.0.1:41188
I20260709 13:22:58.635208 22773 raft_consensus.cc:606] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 4 LEADER]: Received request to transfer leadership to TS f7e755f96eda4b9ba37aa77b183f2a69
I20260709 13:22:58.636248 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:22:58.636855 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:22:58.637446 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:58.770588 23066 raft_consensus.cc:993] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69: : Instructing follower c79ef076f43f4a73884174a86a6a10c1 to start an election
I20260709 13:22:58.771116 23066 raft_consensus.cc:1081] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 LEADER]: Signalling peer c79ef076f43f4a73884174a86a6a10c1 to start an election
I20260709 13:22:58.772892 22773 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "11ccb054ff314be7a47b158bf4d141b6"
dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
 from {username='slave'} at 127.0.0.1:41086
I20260709 13:22:58.773401 22773 raft_consensus.cc:493] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:58.773759 22773 raft_consensus.cc:3060] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:58.778826 22773 raft_consensus.cc:515] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:58.780344 22773 leader_election.cc:290] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 3 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617)
I20260709 13:22:58.781389 22699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ccb054ff314be7a47b158bf4d141b6" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
I20260709 13:22:58.781365 22974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11ccb054ff314be7a47b158bf4d141b6" candidate_uuid: "c79ef076f43f4a73884174a86a6a10c1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "55427c197ec4424580678498fbc616a2"
I20260709 13:22:58.782367 22699 raft_consensus.cc:3055] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:22:58.782384 22974 raft_consensus.cc:3060] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:58.782897 22699 raft_consensus.cc:740] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Running, Role: LEADER
I20260709 13:22:58.783789 22699 consensus_queue.cc:260] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:58.785066 22699 raft_consensus.cc:3060] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:22:58.790148 22974 raft_consensus.cc:2468] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 3.
I20260709 13:22:58.791744 22699 raft_consensus.cc:2468] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c79ef076f43f4a73884174a86a6a10c1 in term 3.
I20260709 13:22:58.792165 22739 leader_election.cc:304] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55427c197ec4424580678498fbc616a2, c79ef076f43f4a73884174a86a6a10c1; no voters: 
I20260709 13:22:58.793040 23084 raft_consensus.cc:2804] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:22:58.793479 23084 raft_consensus.cc:697] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [term 3 LEADER]: Becoming Leader. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Running, Role: LEADER
I20260709 13:22:58.794337 23084 consensus_queue.cc:237] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:58.801177 22522 catalog_manager.cc:5697] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 reported cstate change: term changed from 2 to 3, leader changed from f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66) to c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67). New cstate: current_term: 3 leader_uuid: "c79ef076f43f4a73884174a86a6a10c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: HEALTHY } } }
I20260709 13:22:58.864992 23081 raft_consensus.cc:993] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1: : Instructing follower f7e755f96eda4b9ba37aa77b183f2a69 to start an election
I20260709 13:22:58.865761 23084 raft_consensus.cc:1081] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 4 LEADER]: Signalling peer f7e755f96eda4b9ba37aa77b183f2a69 to start an election
I20260709 13:22:58.867740 22699 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7505614384fc47b19d23973ba804ea5a"
dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
 from {username='slave'} at 127.0.0.1:45770
I20260709 13:22:58.868418 22699 raft_consensus.cc:493] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:58.868978 22699 raft_consensus.cc:3060] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:22:58.874617 22699 raft_consensus.cc:515] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:58.876119 22699 leader_election.cc:290] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 5 election: Requested vote from peers 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635), c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633)
I20260709 13:22:58.877153 22975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7505614384fc47b19d23973ba804ea5a" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "55427c197ec4424580678498fbc616a2"
I20260709 13:22:58.877161 22773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7505614384fc47b19d23973ba804ea5a" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
I20260709 13:22:58.877815 22975 raft_consensus.cc:3060] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:22:58.877897 22773 raft_consensus.cc:3055] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:22:58.878309 22773 raft_consensus.cc:740] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Running, Role: LEADER
I20260709 13:22:58.878894 22773 consensus_queue.cc:260] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:58.879745 22773 raft_consensus.cc:3060] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:22:58.884164 22975 raft_consensus.cc:2468] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 5.
I20260709 13:22:58.884781 22773 raft_consensus.cc:2468] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 5.
I20260709 13:22:58.886044 22666 leader_election.cc:304] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c79ef076f43f4a73884174a86a6a10c1, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 
I20260709 13:22:58.886823 23092 raft_consensus.cc:2804] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:22:58.887360 23092 raft_consensus.cc:697] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 5 LEADER]: Becoming Leader. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Running, Role: LEADER
I20260709 13:22:58.888116 23092 consensus_queue.cc:237] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } }
I20260709 13:22:58.895357 22523 catalog_manager.cc:5697] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 reported cstate change: term changed from 4 to 5, leader changed from c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67) to f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66). New cstate: current_term: 5 leader_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:58.978464 23081 raft_consensus.cc:993] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1: : Instructing follower f7e755f96eda4b9ba37aa77b183f2a69 to start an election
I20260709 13:22:58.978952 23084 raft_consensus.cc:1081] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 3 LEADER]: Signalling peer f7e755f96eda4b9ba37aa77b183f2a69 to start an election
I20260709 13:22:58.980719 22699 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a44aff4c84e4f05a00639dc531ecca5"
dest_uuid: "f7e755f96eda4b9ba37aa77b183f2a69"
 from {username='slave'} at 127.0.0.1:45770
I20260709 13:22:58.981299 22699 raft_consensus.cc:493] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:22:58.981734 22699 raft_consensus.cc:3060] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:58.988586 22699 raft_consensus.cc:515] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } }
I20260709 13:22:58.990125 22699 leader_election.cc:290] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 4 election: Requested vote from peers c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67:34633), 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635)
I20260709 13:22:58.991071 22773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a44aff4c84e4f05a00639dc531ecca5" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c79ef076f43f4a73884174a86a6a10c1"
I20260709 13:22:58.991142 22975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a44aff4c84e4f05a00639dc531ecca5" candidate_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "55427c197ec4424580678498fbc616a2"
I20260709 13:22:58.991895 22773 raft_consensus.cc:3055] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:22:58.991947 22975 raft_consensus.cc:3060] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:58.992355 22773 raft_consensus.cc:740] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c79ef076f43f4a73884174a86a6a10c1, State: Running, Role: LEADER
I20260709 13:22:58.993124 22773 consensus_queue.cc:260] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } }
I20260709 13:22:58.994282 22773 raft_consensus.cc:3060] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:22:58.999018 22975 raft_consensus.cc:2468] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 4.
I20260709 13:22:59.000337 22665 leader_election.cc:304] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55427c197ec4424580678498fbc616a2, f7e755f96eda4b9ba37aa77b183f2a69; no voters: 
I20260709 13:22:59.000802 22773 raft_consensus.cc:2468] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e755f96eda4b9ba37aa77b183f2a69 in term 4.
I20260709 13:22:59.001056 23092 raft_consensus.cc:2804] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:22:59.001601 23092 raft_consensus.cc:697] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 4 LEADER]: Becoming Leader. State: Replica: f7e755f96eda4b9ba37aa77b183f2a69, State: Running, Role: LEADER
I20260709 13:22:59.002386 23092 consensus_queue.cc:237] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } }
I20260709 13:22:59.008349 22523 catalog_manager.cc:5697] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 reported cstate change: term changed from 3 to 4, leader changed from c79ef076f43f4a73884174a86a6a10c1 (127.11.63.67) to f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66). New cstate: current_term: 4 leader_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 } health_report { overall_health: UNKNOWN } } }
I20260709 13:22:59.270357 22975 raft_consensus.cc:1275] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [term 5 FOLLOWER]: Refusing update from remote peer f7e755f96eda4b9ba37aa77b183f2a69: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:22:59.271824 23092 consensus_queue.cc:1048] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:22:59.281754 22773 raft_consensus.cc:1275] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 5 FOLLOWER]: Refusing update from remote peer f7e755f96eda4b9ba37aa77b183f2a69: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:22:59.283788 23092 consensus_queue.cc:1048] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:22:59.371140 22975 raft_consensus.cc:1275] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [term 3 FOLLOWER]: Refusing update from remote peer c79ef076f43f4a73884174a86a6a10c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:22:59.372367 23084 consensus_queue.cc:1048] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:22:59.381722 22699 raft_consensus.cc:1275] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 FOLLOWER]: Refusing update from remote peer c79ef076f43f4a73884174a86a6a10c1: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:22:59.383841 23081 consensus_queue.cc:1048] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7e755f96eda4b9ba37aa77b183f2a69" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 37617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:22:59.466187 22571 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:22:59.501688 22975 raft_consensus.cc:1275] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [term 4 FOLLOWER]: Refusing update from remote peer f7e755f96eda4b9ba37aa77b183f2a69: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:22:59.503098 23086 consensus_queue.cc:1048] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55427c197ec4424580678498fbc616a2" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:22:59.512868 22773 raft_consensus.cc:1275] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 4 FOLLOWER]: Refusing update from remote peer f7e755f96eda4b9ba37aa77b183f2a69: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:22:59.516605 23086 consensus_queue.cc:1048] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c79ef076f43f4a73884174a86a6a10c1" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 34633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:22:59.644088 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:22:59.679201 11517 tablet_server.cc:179] TabletServer@127.11.63.65:36635 shutting down...
I20260709 13:22:59.701094 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:59.701934 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:59.702615 11517 raft_consensus.cc:2243] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.703081 11517 raft_consensus.cc:2272] T add99f7fe29440ab82c13df2e1bc541d P 55427c197ec4424580678498fbc616a2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.705098 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:59.705734 11517 raft_consensus.cc:2243] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.706205 11517 raft_consensus.cc:2272] T 7505614384fc47b19d23973ba804ea5a P 55427c197ec4424580678498fbc616a2 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.708578 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:59.709146 11517 raft_consensus.cc:2243] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.709636 11517 raft_consensus.cc:2272] T 8e8e668ce13b473186eed6b7c92dbf8c P 55427c197ec4424580678498fbc616a2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.711486 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:59.712049 11517 raft_consensus.cc:2243] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.712445 11517 raft_consensus.cc:2272] T 28c206f085d9400c8978877b15ad9058 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.714190 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:59.714706 11517 raft_consensus.cc:2243] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.715129 11517 raft_consensus.cc:2272] T 4a44aff4c84e4f05a00639dc531ecca5 P 55427c197ec4424580678498fbc616a2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.717267 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:59.717888 11517 raft_consensus.cc:2243] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.718302 11517 raft_consensus.cc:2272] T 2214925267994c84b73383ae827170d5 P 55427c197ec4424580678498fbc616a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.720003 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:59.720505 11517 raft_consensus.cc:2243] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.720921 11517 raft_consensus.cc:2272] T 11ccb054ff314be7a47b158bf4d141b6 P 55427c197ec4424580678498fbc616a2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.723140 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:59.723671 11517 raft_consensus.cc:2243] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.724102 11517 raft_consensus.cc:2272] T a93e444ff9ec4a578e573e970e281e0a P 55427c197ec4424580678498fbc616a2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.725867 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:59.726349 11517 raft_consensus.cc:2243] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.726764 11517 raft_consensus.cc:2272] T 5cbcb0fb6f4e46b991774ca89f45da59 P 55427c197ec4424580678498fbc616a2 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.728467 11517 tablet_replica.cc:333] stopping tablet replica
I20260709 13:22:59.728960 11517 raft_consensus.cc:2243] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.729362 11517 raft_consensus.cc:2272] T b2d10d445a81457f817ac6b4bfcb77c7 P 55427c197ec4424580678498fbc616a2 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:22:59.754396 22739 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111) [suppressed 6 similar messages]
W20260709 13:22:59.758705 22739 consensus_peers.cc:597] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:59.759166 11517 tablet_server.cc:196] TabletServer@127.11.63.65:36635 shutdown complete.
I20260709 13:22:59.779819 11517 tablet_server.cc:179] TabletServer@127.11.63.66:0 shutting down...
W20260709 13:22:59.797375 22665 consensus_peers.cc:597] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:59.808494 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:59.809408 11517 tablet_replica.cc:333] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69: stopping tablet replica
I20260709 13:22:59.810274 11517 raft_consensus.cc:2243] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:22:59.811160 11517 raft_consensus.cc:2272] T 7505614384fc47b19d23973ba804ea5a P f7e755f96eda4b9ba37aa77b183f2a69 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.813979 11517 tablet_replica.cc:333] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69: stopping tablet replica
I20260709 13:22:59.814639 11517 raft_consensus.cc:2243] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:22:59.815238 22663 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:22:59.815313 11517 raft_consensus.cc:2272] T add99f7fe29440ab82c13df2e1bc541d P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.817814 11517 tablet_replica.cc:333] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69: stopping tablet replica
I20260709 13:22:59.818434 11517 raft_consensus.cc:2243] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:59.819253 11517 raft_consensus.cc:2272] T 2214925267994c84b73383ae827170d5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.821606 11517 tablet_replica.cc:333] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69: stopping tablet replica
I20260709 13:22:59.822229 11517 raft_consensus.cc:2243] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:22:59.823021 11517 raft_consensus.cc:2272] T b2d10d445a81457f817ac6b4bfcb77c7 P f7e755f96eda4b9ba37aa77b183f2a69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.825225 11517 tablet_replica.cc:333] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69: stopping tablet replica
I20260709 13:22:59.825896 11517 raft_consensus.cc:2243] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.826375 11517 raft_consensus.cc:2272] T 8e8e668ce13b473186eed6b7c92dbf8c P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.828544 11517 tablet_replica.cc:333] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69: stopping tablet replica
I20260709 13:22:59.829135 11517 raft_consensus.cc:2243] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:22:59.829994 11517 raft_consensus.cc:2272] T 4a44aff4c84e4f05a00639dc531ecca5 P f7e755f96eda4b9ba37aa77b183f2a69 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.832465 11517 tablet_replica.cc:333] T 28c206f085d9400c8978877b15ad9058 P f7e755f96eda4b9ba37aa77b183f2a69: stopping tablet replica
I20260709 13:22:59.833076 11517 raft_consensus.cc:2243] T 28c206f085d9400c8978877b15ad9058 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.833549 11517 raft_consensus.cc:2272] T 28c206f085d9400c8978877b15ad9058 P f7e755f96eda4b9ba37aa77b183f2a69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.835693 11517 tablet_replica.cc:333] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69: stopping tablet replica
I20260709 13:22:59.836218 11517 raft_consensus.cc:2243] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.836623 11517 raft_consensus.cc:2272] T 11ccb054ff314be7a47b158bf4d141b6 P f7e755f96eda4b9ba37aa77b183f2a69 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.839084 11517 tablet_replica.cc:333] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69: stopping tablet replica
I20260709 13:22:59.839618 11517 raft_consensus.cc:2243] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.840094 11517 raft_consensus.cc:2272] T a93e444ff9ec4a578e573e970e281e0a P f7e755f96eda4b9ba37aa77b183f2a69 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.842545 11517 tablet_replica.cc:333] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69: stopping tablet replica
I20260709 13:22:59.843014 11517 raft_consensus.cc:2243] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.843437 11517 raft_consensus.cc:2272] T 5cbcb0fb6f4e46b991774ca89f45da59 P f7e755f96eda4b9ba37aa77b183f2a69 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.875525 11517 tablet_server.cc:196] TabletServer@127.11.63.66:0 shutdown complete.
W20260709 13:22:59.879096 22739 consensus_peers.cc:597] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:59.900326 11517 tablet_server.cc:179] TabletServer@127.11.63.67:0 shutting down...
W20260709 13:22:59.911770 22739 consensus_peers.cc:597] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 -> Peer 55427c197ec4424580678498fbc616a2 (127.11.63.65:36635): Couldn't send request to peer 55427c197ec4424580678498fbc616a2. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36635: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:22:59.929284 22737 consensus_peers.cc:597] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 -> Peer f7e755f96eda4b9ba37aa77b183f2a69 (127.11.63.66:37617): Couldn't send request to peer f7e755f96eda4b9ba37aa77b183f2a69. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:37617: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:22:59.937055 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:22:59.937958 11517 tablet_replica.cc:333] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1: stopping tablet replica
I20260709 13:22:59.938658 11517 raft_consensus.cc:2243] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.939152 11517 raft_consensus.cc:2272] T 7505614384fc47b19d23973ba804ea5a P c79ef076f43f4a73884174a86a6a10c1 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.942164 11517 tablet_replica.cc:333] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1: stopping tablet replica
I20260709 13:22:59.942831 11517 raft_consensus.cc:2243] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:22:59.943668 11517 raft_consensus.cc:2272] T 5cbcb0fb6f4e46b991774ca89f45da59 P c79ef076f43f4a73884174a86a6a10c1 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.945859 11517 tablet_replica.cc:333] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1: stopping tablet replica
I20260709 13:22:59.946475 11517 raft_consensus.cc:2243] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.946923 11517 raft_consensus.cc:2272] T b2d10d445a81457f817ac6b4bfcb77c7 P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.949060 11517 tablet_replica.cc:333] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1: stopping tablet replica
I20260709 13:22:59.949666 11517 raft_consensus.cc:2243] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.950098 11517 raft_consensus.cc:2272] T add99f7fe29440ab82c13df2e1bc541d P c79ef076f43f4a73884174a86a6a10c1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.952183 11517 tablet_replica.cc:333] T 2214925267994c84b73383ae827170d5 P c79ef076f43f4a73884174a86a6a10c1: stopping tablet replica
I20260709 13:22:59.952771 11517 raft_consensus.cc:2243] T 2214925267994c84b73383ae827170d5 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.953306 11517 raft_consensus.cc:2272] T 2214925267994c84b73383ae827170d5 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.955500 11517 tablet_replica.cc:333] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1: stopping tablet replica
I20260709 13:22:59.956038 11517 raft_consensus.cc:2243] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:59.956833 11517 raft_consensus.cc:2272] T 8e8e668ce13b473186eed6b7c92dbf8c P c79ef076f43f4a73884174a86a6a10c1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.959025 11517 tablet_replica.cc:333] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1: stopping tablet replica
I20260709 13:22:59.959560 11517 raft_consensus.cc:2243] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:22:59.960361 11517 raft_consensus.cc:2272] T a93e444ff9ec4a578e573e970e281e0a P c79ef076f43f4a73884174a86a6a10c1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.962419 11517 tablet_replica.cc:333] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1: stopping tablet replica
I20260709 13:22:59.962934 11517 raft_consensus.cc:2243] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:22:59.963773 11517 raft_consensus.cc:2272] T 11ccb054ff314be7a47b158bf4d141b6 P c79ef076f43f4a73884174a86a6a10c1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.967089 11517 tablet_replica.cc:333] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1: stopping tablet replica
I20260709 13:22:59.967639 11517 raft_consensus.cc:2243] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:22:59.968437 11517 raft_consensus.cc:2272] T 28c206f085d9400c8978877b15ad9058 P c79ef076f43f4a73884174a86a6a10c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:22:59.970705 11517 tablet_replica.cc:333] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1: stopping tablet replica
I20260709 13:22:59.971184 11517 raft_consensus.cc:2243] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:22:59.971623 11517 raft_consensus.cc:2272] T 4a44aff4c84e4f05a00639dc531ecca5 P c79ef076f43f4a73884174a86a6a10c1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.006238 11517 tablet_server.cc:196] TabletServer@127.11.63.67:0 shutdown complete.
I20260709 13:23:00.033123 11517 master.cc:562] Master@127.11.63.126:39291 shutting down...
I20260709 13:23:00.058944 11517 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:00.059497 11517 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:00.059805 11517 tablet_replica.cc:333] T 00000000000000000000000000000000 P 97b2d587e4124950b102670d518a0996: stopping tablet replica
I20260709 13:23:00.081103 11517 master.cc:584] Master@127.11.63.126:39291 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (12818 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:23:00.130062 11517 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.11.63.126:44045
I20260709 13:23:00.131565 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:23:00.137964 23114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:00.138121 23115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:00.138577 23117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:23:00.139840 11517 server_base.cc:1061] running on GCE node
I20260709 13:23:00.140857 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:23:00.141029 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:23:00.141219 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603380141185 us; error 0 us; skew 500 ppm
I20260709 13:23:00.141826 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:23:00.144135 11517 webserver.cc:533] Webserver started at http://127.11.63.126:38717/ using document root <none> and password file <none>
I20260709 13:23:00.144637 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:23:00.144822 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:23:00.145116 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:23:00.146413 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603019296230-11517-0/minicluster-data/master-0-root/instance:
uuid: "09f31d81a9de4d3eaefd219af9031e18"
format_stamp: "Formatted at 2026-07-09 13:23:00 on dist-test-slave-qv7r"
I20260709 13:23:00.151068 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:23:00.155082 23122 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:00.155957 11517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:23:00.156276 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603019296230-11517-0/minicluster-data/master-0-root
uuid: "09f31d81a9de4d3eaefd219af9031e18"
format_stamp: "Formatted at 2026-07-09 13:23:00 on dist-test-slave-qv7r"
I20260709 13:23:00.156680 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603019296230-11517-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603019296230-11517-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603019296230-11517-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:23:00.181205 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:23:00.182403 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:23:00.222651 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.126:44045
I20260709 13:23:00.222759 23173 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.126:44045 every 8 connection(s)
I20260709 13:23:00.226599 23174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:00.237738 23174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18: Bootstrap starting.
I20260709 13:23:00.242583 23174 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:00.247135 23174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18: No bootstrap required, opened a new log
I20260709 13:23:00.249814 23174 raft_consensus.cc:359] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09f31d81a9de4d3eaefd219af9031e18" member_type: VOTER }
I20260709 13:23:00.250257 23174 raft_consensus.cc:385] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:00.250535 23174 raft_consensus.cc:740] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09f31d81a9de4d3eaefd219af9031e18, State: Initialized, Role: FOLLOWER
I20260709 13:23:00.251267 23174 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09f31d81a9de4d3eaefd219af9031e18" member_type: VOTER }
I20260709 13:23:00.251787 23174 raft_consensus.cc:399] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:23:00.252065 23174 raft_consensus.cc:493] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:23:00.252389 23174 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:00.258056 23174 raft_consensus.cc:515] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09f31d81a9de4d3eaefd219af9031e18" member_type: VOTER }
I20260709 13:23:00.258720 23174 leader_election.cc:304] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09f31d81a9de4d3eaefd219af9031e18; no voters: 
I20260709 13:23:00.260021 23174 leader_election.cc:290] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:23:00.260351 23177 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:00.261907 23177 raft_consensus.cc:697] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18 [term 1 LEADER]: Becoming Leader. State: Replica: 09f31d81a9de4d3eaefd219af9031e18, State: Running, Role: LEADER
I20260709 13:23:00.262583 23177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09f31d81a9de4d3eaefd219af9031e18" member_type: VOTER }
I20260709 13:23:00.263275 23174 sys_catalog.cc:565] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:23:00.267108 23179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09f31d81a9de4d3eaefd219af9031e18. Latest consensus state: current_term: 1 leader_uuid: "09f31d81a9de4d3eaefd219af9031e18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09f31d81a9de4d3eaefd219af9031e18" member_type: VOTER } }
I20260709 13:23:00.267153 23178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09f31d81a9de4d3eaefd219af9031e18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09f31d81a9de4d3eaefd219af9031e18" member_type: VOTER } }
I20260709 13:23:00.268260 23178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18 [sys.catalog]: This master's current role is: LEADER
I20260709 13:23:00.270300 23179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18 [sys.catalog]: This master's current role is: LEADER
I20260709 13:23:00.270432 23183 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:23:00.275393 23183 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:23:00.281659 11517 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:23:00.285404 23183 catalog_manager.cc:1383] Generated new cluster ID: 6a4a952575ed42c784eb2403a83f12dc
I20260709 13:23:00.285717 23183 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:23:00.307611 23183 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:23:00.308962 23183 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:23:00.329149 23183 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18: Generated new TSK 0
I20260709 13:23:00.329931 23183 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:23:00.348872 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:23:00.355203 23195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:00.356226 23196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:00.357409 23198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:23:00.358628 11517 server_base.cc:1061] running on GCE node
I20260709 13:23:00.359773 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:23:00.360031 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:23:00.360244 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603380360218 us; error 0 us; skew 500 ppm
I20260709 13:23:00.360817 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:23:00.363629 11517 webserver.cc:533] Webserver started at http://127.11.63.65:46657/ using document root <none> and password file <none>
I20260709 13:23:00.364174 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:23:00.364387 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:23:00.364658 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:23:00.366011 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603019296230-11517-0/minicluster-data/ts-0-root/instance:
uuid: "6ced1532c9df44abb8736460523870c8"
format_stamp: "Formatted at 2026-07-09 13:23:00 on dist-test-slave-qv7r"
I20260709 13:23:00.370628 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:23:00.374223 23203 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:00.375236 11517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 13:23:00.375552 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603019296230-11517-0/minicluster-data/ts-0-root
uuid: "6ced1532c9df44abb8736460523870c8"
format_stamp: "Formatted at 2026-07-09 13:23:00 on dist-test-slave-qv7r"
I20260709 13:23:00.375864 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603019296230-11517-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603019296230-11517-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603019296230-11517-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:23:00.400542 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:23:00.401846 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:23:00.403385 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:23:00.405781 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:23:00.406013 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:00.406260 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:23:00.406426 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:00.448796 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.65:40629
I20260709 13:23:00.448915 23265 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.65:40629 every 8 connection(s)
I20260709 13:23:00.453948 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:23:00.460806 23270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:00.461998 23271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:23:00.465246 11517 server_base.cc:1061] running on GCE node
W20260709 13:23:00.464913 23273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:23:00.467023 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:23:00.467306 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:23:00.467430 23266 heartbeater.cc:344] Connected to a master server at 127.11.63.126:44045
I20260709 13:23:00.467453 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603380467442 us; error 0 us; skew 500 ppm
I20260709 13:23:00.467979 23266 heartbeater.cc:461] Registering TS with master...
I20260709 13:23:00.468386 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:23:00.468791 23266 heartbeater.cc:507] Master 127.11.63.126:44045 requested a full tablet report, sending...
I20260709 13:23:00.471544 11517 webserver.cc:533] Webserver started at http://127.11.63.66:44007/ using document root <none> and password file <none>
I20260709 13:23:00.471550 23139 ts_manager.cc:194] Registered new tserver with Master: 6ced1532c9df44abb8736460523870c8 (127.11.63.65:40629)
I20260709 13:23:00.472470 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:23:00.472744 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:23:00.473066 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:23:00.474203 23139 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58164
I20260709 13:23:00.474642 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603019296230-11517-0/minicluster-data/ts-1-root/instance:
uuid: "1a349bcb0c8447aeb12112510e3e64c9"
format_stamp: "Formatted at 2026-07-09 13:23:00 on dist-test-slave-qv7r"
I20260709 13:23:00.479750 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:23:00.483314 23278 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:00.484206 11517 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:23:00.484462 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603019296230-11517-0/minicluster-data/ts-1-root
uuid: "1a349bcb0c8447aeb12112510e3e64c9"
format_stamp: "Formatted at 2026-07-09 13:23:00 on dist-test-slave-qv7r"
I20260709 13:23:00.484710 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603019296230-11517-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603019296230-11517-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603019296230-11517-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:23:00.502729 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:23:00.503872 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:23:00.505607 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:23:00.508098 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:23:00.508280 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:00.508450 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:23:00.508579 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:00.550707 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.66:41675
I20260709 13:23:00.550813 23340 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.66:41675 every 8 connection(s)
I20260709 13:23:00.555490 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:23:00.563449 23344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:00.565485 23345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:00.567243 23347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:23:00.568188 11517 server_base.cc:1061] running on GCE node
I20260709 13:23:00.569070 23341 heartbeater.cc:344] Connected to a master server at 127.11.63.126:44045
I20260709 13:23:00.569317 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:23:00.569505 23341 heartbeater.cc:461] Registering TS with master...
W20260709 13:23:00.569638 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:23:00.569911 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603380569894 us; error 0 us; skew 500 ppm
I20260709 13:23:00.570538 23341 heartbeater.cc:507] Master 127.11.63.126:44045 requested a full tablet report, sending...
I20260709 13:23:00.570740 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:23:00.573112 23139 ts_manager.cc:194] Registered new tserver with Master: 1a349bcb0c8447aeb12112510e3e64c9 (127.11.63.66:41675)
I20260709 13:23:00.573696 11517 webserver.cc:533] Webserver started at http://127.11.63.67:44393/ using document root <none> and password file <none>
I20260709 13:23:00.574255 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:23:00.574465 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:23:00.574735 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:23:00.574865 23139 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58172
I20260709 13:23:00.576241 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603019296230-11517-0/minicluster-data/ts-2-root/instance:
uuid: "b7b1dad8486e4856bd4f9a6b3041e567"
format_stamp: "Formatted at 2026-07-09 13:23:00 on dist-test-slave-qv7r"
I20260709 13:23:00.581045 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:23:00.584687 23352 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:00.585551 11517 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:23:00.585922 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603019296230-11517-0/minicluster-data/ts-2-root
uuid: "b7b1dad8486e4856bd4f9a6b3041e567"
format_stamp: "Formatted at 2026-07-09 13:23:00 on dist-test-slave-qv7r"
I20260709 13:23:00.586212 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603019296230-11517-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603019296230-11517-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783603019296230-11517-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:23:00.604076 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:23:00.605870 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:23:00.607399 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:23:00.609870 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:23:00.610102 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:00.610347 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:23:00.610527 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:00.653824 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.67:46203
I20260709 13:23:00.654184 23414 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.67:46203 every 8 connection(s)
I20260709 13:23:00.667980 23415 heartbeater.cc:344] Connected to a master server at 127.11.63.126:44045
I20260709 13:23:00.668531 23415 heartbeater.cc:461] Registering TS with master...
I20260709 13:23:00.669297 23415 heartbeater.cc:507] Master 127.11.63.126:44045 requested a full tablet report, sending...
I20260709 13:23:00.671315 23139 ts_manager.cc:194] Registered new tserver with Master: b7b1dad8486e4856bd4f9a6b3041e567 (127.11.63.67:46203)
I20260709 13:23:00.671757 11517 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014833059s
I20260709 13:23:00.672755 23139 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58186
I20260709 13:23:00.699255 23139 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58202:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:23:00.701534 23139 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:23:00.800873 23306 tablet_service.cc:1511] Processing CreateTablet for tablet 5c0603019f674a6787e5e8f3ac2c601f (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=91fbeae1e3024e21aea7f9e9f50444bb]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:23:00.809222 23306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c0603019f674a6787e5e8f3ac2c601f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:00.806740 23305 tablet_service.cc:1511] Processing CreateTablet for tablet 035b53728f9542d397b030aeadc4e6d8 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=91fbeae1e3024e21aea7f9e9f50444bb]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:23:00.811995 23305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 035b53728f9542d397b030aeadc4e6d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:00.817497 23380 tablet_service.cc:1511] Processing CreateTablet for tablet 5c0603019f674a6787e5e8f3ac2c601f (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=91fbeae1e3024e21aea7f9e9f50444bb]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:23:00.817904 23379 tablet_service.cc:1511] Processing CreateTablet for tablet 035b53728f9542d397b030aeadc4e6d8 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=91fbeae1e3024e21aea7f9e9f50444bb]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:23:00.819710 23380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c0603019f674a6787e5e8f3ac2c601f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:00.820088 23230 tablet_service.cc:1511] Processing CreateTablet for tablet 035b53728f9542d397b030aeadc4e6d8 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=91fbeae1e3024e21aea7f9e9f50444bb]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:23:00.822052 23230 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 035b53728f9542d397b030aeadc4e6d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:00.817680 23231 tablet_service.cc:1511] Processing CreateTablet for tablet 5c0603019f674a6787e5e8f3ac2c601f (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=91fbeae1e3024e21aea7f9e9f50444bb]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:23:00.827150 23231 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c0603019f674a6787e5e8f3ac2c601f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:00.834345 23379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 035b53728f9542d397b030aeadc4e6d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:00.851769 23431 tablet_bootstrap.cc:492] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9: Bootstrap starting.
I20260709 13:23:00.854171 23432 tablet_bootstrap.cc:492] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8: Bootstrap starting.
I20260709 13:23:00.861064 23431 tablet_bootstrap.cc:654] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:00.861266 23432 tablet_bootstrap.cc:654] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:00.878140 23431 tablet_bootstrap.cc:492] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9: No bootstrap required, opened a new log
I20260709 13:23:00.878276 23435 tablet_bootstrap.cc:492] T 035b53728f9542d397b030aeadc4e6d8 P b7b1dad8486e4856bd4f9a6b3041e567: Bootstrap starting.
I20260709 13:23:00.878718 23431 ts_tablet_manager.cc:1403] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.016s
I20260709 13:23:00.878372 23432 tablet_bootstrap.cc:492] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8: No bootstrap required, opened a new log
I20260709 13:23:00.882755 23431 raft_consensus.cc:359] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:00.884193 23431 raft_consensus.cc:385] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:00.884295 23432 ts_tablet_manager.cc:1403] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8: Time spent bootstrapping tablet: real 0.031s	user 0.005s	sys 0.019s
I20260709 13:23:00.884631 23431 raft_consensus.cc:740] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a349bcb0c8447aeb12112510e3e64c9, State: Initialized, Role: FOLLOWER
I20260709 13:23:00.886006 23431 consensus_queue.cc:260] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:00.886925 23435 tablet_bootstrap.cc:654] T 035b53728f9542d397b030aeadc4e6d8 P b7b1dad8486e4856bd4f9a6b3041e567: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:00.888486 23432 raft_consensus.cc:359] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:00.889346 23432 raft_consensus.cc:385] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:00.889694 23432 raft_consensus.cc:740] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ced1532c9df44abb8736460523870c8, State: Initialized, Role: FOLLOWER
I20260709 13:23:00.890746 23432 consensus_queue.cc:260] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:00.894129 23266 heartbeater.cc:499] Master 127.11.63.126:44045 was elected leader, sending a full tablet report...
I20260709 13:23:00.894572 23432 ts_tablet_manager.cc:1434] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.004s
I20260709 13:23:00.895920 23432 tablet_bootstrap.cc:492] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8: Bootstrap starting.
I20260709 13:23:00.895951 23341 heartbeater.cc:499] Master 127.11.63.126:44045 was elected leader, sending a full tablet report...
I20260709 13:23:00.897991 23438 raft_consensus.cc:493] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:00.898753 23438 raft_consensus.cc:515] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:00.899513 23435 tablet_bootstrap.cc:492] T 035b53728f9542d397b030aeadc4e6d8 P b7b1dad8486e4856bd4f9a6b3041e567: No bootstrap required, opened a new log
I20260709 13:23:00.900133 23435 ts_tablet_manager.cc:1403] T 035b53728f9542d397b030aeadc4e6d8 P b7b1dad8486e4856bd4f9a6b3041e567: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.004s
I20260709 13:23:00.900359 23431 ts_tablet_manager.cc:1434] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9: Time spent starting tablet: real 0.021s	user 0.014s	sys 0.002s
I20260709 13:23:00.901535 23431 tablet_bootstrap.cc:492] T 5c0603019f674a6787e5e8f3ac2c601f P 1a349bcb0c8447aeb12112510e3e64c9: Bootstrap starting.
I20260709 13:23:00.903585 23435 raft_consensus.cc:359] T 035b53728f9542d397b030aeadc4e6d8 P b7b1dad8486e4856bd4f9a6b3041e567 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:00.904376 23435 raft_consensus.cc:385] T 035b53728f9542d397b030aeadc4e6d8 P b7b1dad8486e4856bd4f9a6b3041e567 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:00.904678 23435 raft_consensus.cc:740] T 035b53728f9542d397b030aeadc4e6d8 P b7b1dad8486e4856bd4f9a6b3041e567 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7b1dad8486e4856bd4f9a6b3041e567, State: Initialized, Role: FOLLOWER
I20260709 13:23:00.908504 23438 leader_election.cc:290] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a349bcb0c8447aeb12112510e3e64c9 (127.11.63.66:41675), b7b1dad8486e4856bd4f9a6b3041e567 (127.11.63.67:46203)
I20260709 13:23:00.909008 23431 tablet_bootstrap.cc:654] T 5c0603019f674a6787e5e8f3ac2c601f P 1a349bcb0c8447aeb12112510e3e64c9: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:00.916299 23436 raft_consensus.cc:493] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:00.917232 23436 raft_consensus.cc:515] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:00.917898 23432 tablet_bootstrap.cc:654] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:00.919659 23435 consensus_queue.cc:260] T 035b53728f9542d397b030aeadc4e6d8 P b7b1dad8486e4856bd4f9a6b3041e567 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:00.935700 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "035b53728f9542d397b030aeadc4e6d8" candidate_uuid: "6ced1532c9df44abb8736460523870c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" is_pre_election: true
I20260709 13:23:00.936213 23436 leader_election.cc:290] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ced1532c9df44abb8736460523870c8 (127.11.63.65:40629), b7b1dad8486e4856bd4f9a6b3041e567 (127.11.63.67:46203)
I20260709 13:23:00.936760 23390 raft_consensus.cc:2468] T 035b53728f9542d397b030aeadc4e6d8 P b7b1dad8486e4856bd4f9a6b3041e567 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ced1532c9df44abb8736460523870c8 in term 0.
I20260709 13:23:00.941084 23207 leader_election.cc:304] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ced1532c9df44abb8736460523870c8, b7b1dad8486e4856bd4f9a6b3041e567; no voters: 
I20260709 13:23:00.942155 23438 raft_consensus.cc:2804] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:00.942551 23438 raft_consensus.cc:493] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:00.942910 23438 raft_consensus.cc:3060] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:00.944161 23431 tablet_bootstrap.cc:492] T 5c0603019f674a6787e5e8f3ac2c601f P 1a349bcb0c8447aeb12112510e3e64c9: No bootstrap required, opened a new log
I20260709 13:23:00.945001 23431 ts_tablet_manager.cc:1403] T 5c0603019f674a6787e5e8f3ac2c601f P 1a349bcb0c8447aeb12112510e3e64c9: Time spent bootstrapping tablet: real 0.044s	user 0.008s	sys 0.011s
I20260709 13:23:00.949801 23241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "035b53728f9542d397b030aeadc4e6d8" candidate_uuid: "1a349bcb0c8447aeb12112510e3e64c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ced1532c9df44abb8736460523870c8" is_pre_election: true
I20260709 13:23:00.949648 23431 raft_consensus.cc:359] T 5c0603019f674a6787e5e8f3ac2c601f P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } } peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } }
I20260709 13:23:00.950762 23431 raft_consensus.cc:385] T 5c0603019f674a6787e5e8f3ac2c601f P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:00.951160 23431 raft_consensus.cc:740] T 5c0603019f674a6787e5e8f3ac2c601f P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a349bcb0c8447aeb12112510e3e64c9, State: Initialized, Role: FOLLOWER
I20260709 13:23:00.952184 23431 consensus_queue.cc:260] T 5c0603019f674a6787e5e8f3ac2c601f P 1a349bcb0c8447aeb12112510e3e64c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } } peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } }
I20260709 13:23:00.955629 23431 ts_tablet_manager.cc:1434] T 5c0603019f674a6787e5e8f3ac2c601f P 1a349bcb0c8447aeb12112510e3e64c9: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.005s
I20260709 13:23:00.964033 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "035b53728f9542d397b030aeadc4e6d8" candidate_uuid: "6ced1532c9df44abb8736460523870c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a349bcb0c8447aeb12112510e3e64c9" is_pre_election: true
I20260709 13:23:00.964998 23316 raft_consensus.cc:2468] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ced1532c9df44abb8736460523870c8 in term 0.
I20260709 13:23:00.964670 23438 raft_consensus.cc:515] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:00.966451 23241 raft_consensus.cc:2393] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1a349bcb0c8447aeb12112510e3e64c9 in current term 1: Already voted for candidate 6ced1532c9df44abb8736460523870c8 in this term.
I20260709 13:23:00.971344 23435 ts_tablet_manager.cc:1434] T 035b53728f9542d397b030aeadc4e6d8 P b7b1dad8486e4856bd4f9a6b3041e567: Time spent starting tablet: real 0.071s	user 0.010s	sys 0.043s
I20260709 13:23:00.975301 23432 tablet_bootstrap.cc:492] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8: No bootstrap required, opened a new log
I20260709 13:23:00.976622 23432 ts_tablet_manager.cc:1403] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8: Time spent bootstrapping tablet: real 0.081s	user 0.009s	sys 0.021s
I20260709 13:23:00.978325 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "035b53728f9542d397b030aeadc4e6d8" candidate_uuid: "6ced1532c9df44abb8736460523870c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a349bcb0c8447aeb12112510e3e64c9"
I20260709 13:23:00.979031 23316 raft_consensus.cc:3060] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:00.980899 23432 raft_consensus.cc:359] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } } peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } }
I20260709 13:23:00.981874 23432 raft_consensus.cc:385] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:00.982242 23432 raft_consensus.cc:740] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ced1532c9df44abb8736460523870c8, State: Initialized, Role: FOLLOWER
I20260709 13:23:00.983338 23415 heartbeater.cc:499] Master 127.11.63.126:44045 was elected leader, sending a full tablet report...
I20260709 13:23:00.983434 23432 consensus_queue.cc:260] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } } peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } }
I20260709 13:23:00.985176 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "035b53728f9542d397b030aeadc4e6d8" candidate_uuid: "6ced1532c9df44abb8736460523870c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b1dad8486e4856bd4f9a6b3041e567"
I20260709 13:23:00.985858 23390 raft_consensus.cc:3060] T 035b53728f9542d397b030aeadc4e6d8 P b7b1dad8486e4856bd4f9a6b3041e567 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:00.989063 23435 tablet_bootstrap.cc:492] T 5c0603019f674a6787e5e8f3ac2c601f P b7b1dad8486e4856bd4f9a6b3041e567: Bootstrap starting.
I20260709 13:23:00.992089 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "035b53728f9542d397b030aeadc4e6d8" candidate_uuid: "1a349bcb0c8447aeb12112510e3e64c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" is_pre_election: true
I20260709 13:23:00.994935 23282 leader_election.cc:304] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a349bcb0c8447aeb12112510e3e64c9; no voters: 6ced1532c9df44abb8736460523870c8, b7b1dad8486e4856bd4f9a6b3041e567
I20260709 13:23:00.997104 23316 raft_consensus.cc:2468] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ced1532c9df44abb8736460523870c8 in term 1.
I20260709 13:23:00.998749 23436 raft_consensus.cc:2749] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:00.999303 23438 leader_election.cc:290] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [CANDIDATE]: Term 1 election: Requested vote from peers 1a349bcb0c8447aeb12112510e3e64c9 (127.11.63.66:41675), b7b1dad8486e4856bd4f9a6b3041e567 (127.11.63.67:46203)
I20260709 13:23:00.999541 23435 tablet_bootstrap.cc:654] T 5c0603019f674a6787e5e8f3ac2c601f P b7b1dad8486e4856bd4f9a6b3041e567: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:01.003844 23204 leader_election.cc:304] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a349bcb0c8447aeb12112510e3e64c9, 6ced1532c9df44abb8736460523870c8; no voters: 
I20260709 13:23:01.006712 23444 raft_consensus.cc:2804] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:01.007884 23432 ts_tablet_manager.cc:1434] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8: Time spent starting tablet: real 0.031s	user 0.012s	sys 0.018s
I20260709 13:23:01.009367 23444 raft_consensus.cc:697] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [term 1 LEADER]: Becoming Leader. State: Replica: 6ced1532c9df44abb8736460523870c8, State: Running, Role: LEADER
I20260709 13:23:01.010488 23444 consensus_queue.cc:237] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:01.011826 23390 raft_consensus.cc:2468] T 035b53728f9542d397b030aeadc4e6d8 P b7b1dad8486e4856bd4f9a6b3041e567 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ced1532c9df44abb8736460523870c8 in term 1.
I20260709 13:23:01.016269 23435 tablet_bootstrap.cc:492] T 5c0603019f674a6787e5e8f3ac2c601f P b7b1dad8486e4856bd4f9a6b3041e567: No bootstrap required, opened a new log
I20260709 13:23:01.017182 23435 ts_tablet_manager.cc:1403] T 5c0603019f674a6787e5e8f3ac2c601f P b7b1dad8486e4856bd4f9a6b3041e567: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.014s
I20260709 13:23:01.020428 23435 raft_consensus.cc:359] T 5c0603019f674a6787e5e8f3ac2c601f P b7b1dad8486e4856bd4f9a6b3041e567 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } } peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } }
I20260709 13:23:01.021103 23435 raft_consensus.cc:385] T 5c0603019f674a6787e5e8f3ac2c601f P b7b1dad8486e4856bd4f9a6b3041e567 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:01.021380 23435 raft_consensus.cc:740] T 5c0603019f674a6787e5e8f3ac2c601f P b7b1dad8486e4856bd4f9a6b3041e567 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7b1dad8486e4856bd4f9a6b3041e567, State: Initialized, Role: FOLLOWER
I20260709 13:23:01.022167 23435 consensus_queue.cc:260] T 5c0603019f674a6787e5e8f3ac2c601f P b7b1dad8486e4856bd4f9a6b3041e567 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } } peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } }
I20260709 13:23:01.023167 23138 catalog_manager.cc:5697] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ced1532c9df44abb8736460523870c8 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "6ced1532c9df44abb8736460523870c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:01.024314 23435 ts_tablet_manager.cc:1434] T 5c0603019f674a6787e5e8f3ac2c601f P b7b1dad8486e4856bd4f9a6b3041e567: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:23:01.128262 23444 raft_consensus.cc:493] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:01.128996 23444 raft_consensus.cc:515] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } } peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } }
I20260709 13:23:01.131258 23444 leader_election.cc:290] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a349bcb0c8447aeb12112510e3e64c9 (127.11.63.66:41675), b7b1dad8486e4856bd4f9a6b3041e567 (127.11.63.67:46203)
I20260709 13:23:01.132130 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c0603019f674a6787e5e8f3ac2c601f" candidate_uuid: "6ced1532c9df44abb8736460523870c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a349bcb0c8447aeb12112510e3e64c9" is_pre_election: true
I20260709 13:23:01.132465 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c0603019f674a6787e5e8f3ac2c601f" candidate_uuid: "6ced1532c9df44abb8736460523870c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" is_pre_election: true
I20260709 13:23:01.132898 23316 raft_consensus.cc:2468] T 5c0603019f674a6787e5e8f3ac2c601f P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ced1532c9df44abb8736460523870c8 in term 0.
I20260709 13:23:01.133279 23390 raft_consensus.cc:2468] T 5c0603019f674a6787e5e8f3ac2c601f P b7b1dad8486e4856bd4f9a6b3041e567 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ced1532c9df44abb8736460523870c8 in term 0.
I20260709 13:23:01.133965 23204 leader_election.cc:304] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a349bcb0c8447aeb12112510e3e64c9, 6ced1532c9df44abb8736460523870c8; no voters: 
I20260709 13:23:01.134914 23444 raft_consensus.cc:2804] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:01.135197 23444 raft_consensus.cc:493] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:01.135471 23444 raft_consensus.cc:3060] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:01.141731 23444 raft_consensus.cc:515] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } } peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } }
I20260709 13:23:01.143319 23444 leader_election.cc:290] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8 [CANDIDATE]: Term 1 election: Requested vote from peers 1a349bcb0c8447aeb12112510e3e64c9 (127.11.63.66:41675), b7b1dad8486e4856bd4f9a6b3041e567 (127.11.63.67:46203)
I20260709 13:23:01.144301 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c0603019f674a6787e5e8f3ac2c601f" candidate_uuid: "6ced1532c9df44abb8736460523870c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a349bcb0c8447aeb12112510e3e64c9"
I20260709 13:23:01.144562 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c0603019f674a6787e5e8f3ac2c601f" candidate_uuid: "6ced1532c9df44abb8736460523870c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b1dad8486e4856bd4f9a6b3041e567"
I20260709 13:23:01.144906 23316 raft_consensus.cc:3060] T 5c0603019f674a6787e5e8f3ac2c601f P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:01.145136 23390 raft_consensus.cc:3060] T 5c0603019f674a6787e5e8f3ac2c601f P b7b1dad8486e4856bd4f9a6b3041e567 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:01.150246 23390 raft_consensus.cc:2468] T 5c0603019f674a6787e5e8f3ac2c601f P b7b1dad8486e4856bd4f9a6b3041e567 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ced1532c9df44abb8736460523870c8 in term 1.
I20260709 13:23:01.150800 23316 raft_consensus.cc:2468] T 5c0603019f674a6787e5e8f3ac2c601f P 1a349bcb0c8447aeb12112510e3e64c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ced1532c9df44abb8736460523870c8 in term 1.
I20260709 13:23:01.151398 23207 leader_election.cc:304] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ced1532c9df44abb8736460523870c8, b7b1dad8486e4856bd4f9a6b3041e567; no voters: 
I20260709 13:23:01.152328 23444 raft_consensus.cc:2804] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:01.152805 23444 raft_consensus.cc:697] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8 [term 1 LEADER]: Becoming Leader. State: Replica: 6ced1532c9df44abb8736460523870c8, State: Running, Role: LEADER
I20260709 13:23:01.153708 23444 consensus_queue.cc:237] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } } peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } }
I20260709 13:23:01.161741 23137 catalog_manager.cc:5697] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ced1532c9df44abb8736460523870c8 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "6ced1532c9df44abb8736460523870c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:01.235809 23137 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58204:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:23:01.238361 23137 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:23:01.262593 23231 tablet_service.cc:1511] Processing CreateTablet for tablet 9ef6f66168a2459d9cd62f390d878801 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=75b9e5817b444961b6dfc52da40f382c]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:23:01.263921 23231 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ef6f66168a2459d9cd62f390d878801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:01.264037 23306 tablet_service.cc:1511] Processing CreateTablet for tablet 9ef6f66168a2459d9cd62f390d878801 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=75b9e5817b444961b6dfc52da40f382c]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:23:01.265364 23306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ef6f66168a2459d9cd62f390d878801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:01.266557 23379 tablet_service.cc:1511] Processing CreateTablet for tablet 9ef6f66168a2459d9cd62f390d878801 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=75b9e5817b444961b6dfc52da40f382c]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:23:01.267174 23230 tablet_service.cc:1511] Processing CreateTablet for tablet 186e80e1f23f4c77a1b6fd7502979a80 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=75b9e5817b444961b6dfc52da40f382c]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:23:01.267977 23379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ef6f66168a2459d9cd62f390d878801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:01.268432 23230 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 186e80e1f23f4c77a1b6fd7502979a80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:01.269753 23380 tablet_service.cc:1511] Processing CreateTablet for tablet 186e80e1f23f4c77a1b6fd7502979a80 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=75b9e5817b444961b6dfc52da40f382c]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:23:01.270982 23380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 186e80e1f23f4c77a1b6fd7502979a80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:01.273296 23305 tablet_service.cc:1511] Processing CreateTablet for tablet 186e80e1f23f4c77a1b6fd7502979a80 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=75b9e5817b444961b6dfc52da40f382c]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:23:01.274564 23305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 186e80e1f23f4c77a1b6fd7502979a80. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:23:01.276548 23187 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:23:01.286485 23432 tablet_bootstrap.cc:492] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8: Bootstrap starting.
I20260709 13:23:01.292785 23435 tablet_bootstrap.cc:492] T 9ef6f66168a2459d9cd62f390d878801 P b7b1dad8486e4856bd4f9a6b3041e567: Bootstrap starting.
I20260709 13:23:01.299897 23431 tablet_bootstrap.cc:492] T 9ef6f66168a2459d9cd62f390d878801 P 1a349bcb0c8447aeb12112510e3e64c9: Bootstrap starting.
I20260709 13:23:01.302520 23432 tablet_bootstrap.cc:654] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:01.309008 23435 tablet_bootstrap.cc:654] T 9ef6f66168a2459d9cd62f390d878801 P b7b1dad8486e4856bd4f9a6b3041e567: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:01.309489 23431 tablet_bootstrap.cc:654] T 9ef6f66168a2459d9cd62f390d878801 P 1a349bcb0c8447aeb12112510e3e64c9: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:01.312474 23432 tablet_bootstrap.cc:492] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8: No bootstrap required, opened a new log
I20260709 13:23:01.316778 23432 ts_tablet_manager.cc:1403] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.003s
I20260709 13:23:01.320665 23432 raft_consensus.cc:359] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:01.321688 23432 raft_consensus.cc:385] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:01.321724 23435 tablet_bootstrap.cc:492] T 9ef6f66168a2459d9cd62f390d878801 P b7b1dad8486e4856bd4f9a6b3041e567: No bootstrap required, opened a new log
I20260709 13:23:01.322082 23431 tablet_bootstrap.cc:492] T 9ef6f66168a2459d9cd62f390d878801 P 1a349bcb0c8447aeb12112510e3e64c9: No bootstrap required, opened a new log
I20260709 13:23:01.322372 23435 ts_tablet_manager.cc:1403] T 9ef6f66168a2459d9cd62f390d878801 P b7b1dad8486e4856bd4f9a6b3041e567: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.010s
I20260709 13:23:01.323273 23431 ts_tablet_manager.cc:1403] T 9ef6f66168a2459d9cd62f390d878801 P 1a349bcb0c8447aeb12112510e3e64c9: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.010s
I20260709 13:23:01.322090 23432 raft_consensus.cc:740] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ced1532c9df44abb8736460523870c8, State: Initialized, Role: FOLLOWER
I20260709 13:23:01.324198 23432 consensus_queue.cc:260] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:01.325949 23435 raft_consensus.cc:359] T 9ef6f66168a2459d9cd62f390d878801 P b7b1dad8486e4856bd4f9a6b3041e567 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:01.326274 23431 raft_consensus.cc:359] T 9ef6f66168a2459d9cd62f390d878801 P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:01.326757 23435 raft_consensus.cc:385] T 9ef6f66168a2459d9cd62f390d878801 P b7b1dad8486e4856bd4f9a6b3041e567 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:01.326944 23431 raft_consensus.cc:385] T 9ef6f66168a2459d9cd62f390d878801 P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:01.326988 23432 ts_tablet_manager.cc:1434] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 13:23:01.327088 23435 raft_consensus.cc:740] T 9ef6f66168a2459d9cd62f390d878801 P b7b1dad8486e4856bd4f9a6b3041e567 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7b1dad8486e4856bd4f9a6b3041e567, State: Initialized, Role: FOLLOWER
I20260709 13:23:01.327895 23431 raft_consensus.cc:740] T 9ef6f66168a2459d9cd62f390d878801 P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a349bcb0c8447aeb12112510e3e64c9, State: Initialized, Role: FOLLOWER
I20260709 13:23:01.328801 23432 tablet_bootstrap.cc:492] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8: Bootstrap starting.
I20260709 13:23:01.328774 23435 consensus_queue.cc:260] T 9ef6f66168a2459d9cd62f390d878801 P b7b1dad8486e4856bd4f9a6b3041e567 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:01.328774 23431 consensus_queue.cc:260] T 9ef6f66168a2459d9cd62f390d878801 P 1a349bcb0c8447aeb12112510e3e64c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:01.331602 23431 ts_tablet_manager.cc:1434] T 9ef6f66168a2459d9cd62f390d878801 P 1a349bcb0c8447aeb12112510e3e64c9: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:23:01.331790 23435 ts_tablet_manager.cc:1434] T 9ef6f66168a2459d9cd62f390d878801 P b7b1dad8486e4856bd4f9a6b3041e567: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:23:01.332821 23435 tablet_bootstrap.cc:492] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567: Bootstrap starting.
I20260709 13:23:01.332821 23431 tablet_bootstrap.cc:492] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9: Bootstrap starting.
I20260709 13:23:01.340695 23431 tablet_bootstrap.cc:654] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:01.340757 23432 tablet_bootstrap.cc:654] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:01.340695 23435 tablet_bootstrap.cc:654] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:01.355579 23432 tablet_bootstrap.cc:492] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8: No bootstrap required, opened a new log
I20260709 13:23:01.356113 23432 ts_tablet_manager.cc:1403] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.005s
I20260709 13:23:01.359205 23432 raft_consensus.cc:359] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:01.359972 23432 raft_consensus.cc:385] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:01.360277 23432 raft_consensus.cc:740] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ced1532c9df44abb8736460523870c8, State: Initialized, Role: FOLLOWER
I20260709 13:23:01.361207 23432 consensus_queue.cc:260] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:01.365279 23431 tablet_bootstrap.cc:492] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9: No bootstrap required, opened a new log
I20260709 13:23:01.365404 23435 tablet_bootstrap.cc:492] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567: No bootstrap required, opened a new log
I20260709 13:23:01.366003 23431 ts_tablet_manager.cc:1403] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9: Time spent bootstrapping tablet: real 0.033s	user 0.016s	sys 0.015s
I20260709 13:23:01.366111 23435 ts_tablet_manager.cc:1403] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.011s
I20260709 13:23:01.367674 23432 ts_tablet_manager.cc:1434] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.005s
I20260709 13:23:01.369356 23431 raft_consensus.cc:359] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:01.369673 23435 raft_consensus.cc:359] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:01.370186 23431 raft_consensus.cc:385] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:01.370448 23435 raft_consensus.cc:385] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:01.370558 23431 raft_consensus.cc:740] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a349bcb0c8447aeb12112510e3e64c9, State: Initialized, Role: FOLLOWER
I20260709 13:23:01.370878 23435 raft_consensus.cc:740] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7b1dad8486e4856bd4f9a6b3041e567, State: Initialized, Role: FOLLOWER
I20260709 13:23:01.371546 23431 consensus_queue.cc:260] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:01.371793 23435 consensus_queue.cc:260] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:01.374243 23435 ts_tablet_manager.cc:1434] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:23:01.375087 23431 ts_tablet_manager.cc:1434] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:23:01.383574 23444 raft_consensus.cc:493] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:01.384171 23444 raft_consensus.cc:515] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:01.386279 23444 leader_election.cc:290] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a349bcb0c8447aeb12112510e3e64c9 (127.11.63.66:41675), b7b1dad8486e4856bd4f9a6b3041e567 (127.11.63.67:46203)
I20260709 13:23:01.387218 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ef6f66168a2459d9cd62f390d878801" candidate_uuid: "6ced1532c9df44abb8736460523870c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a349bcb0c8447aeb12112510e3e64c9" is_pre_election: true
I20260709 13:23:01.387457 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ef6f66168a2459d9cd62f390d878801" candidate_uuid: "6ced1532c9df44abb8736460523870c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" is_pre_election: true
I20260709 13:23:01.388109 23390 raft_consensus.cc:2468] T 9ef6f66168a2459d9cd62f390d878801 P b7b1dad8486e4856bd4f9a6b3041e567 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ced1532c9df44abb8736460523870c8 in term 0.
I20260709 13:23:01.388049 23316 raft_consensus.cc:2468] T 9ef6f66168a2459d9cd62f390d878801 P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ced1532c9df44abb8736460523870c8 in term 0.
I20260709 13:23:01.389454 23207 leader_election.cc:304] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ced1532c9df44abb8736460523870c8, b7b1dad8486e4856bd4f9a6b3041e567; no voters: 
I20260709 13:23:01.390309 23444 raft_consensus.cc:2804] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:01.390668 23444 raft_consensus.cc:493] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:01.390966 23444 raft_consensus.cc:3060] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:01.396174 23444 raft_consensus.cc:515] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:01.397934 23444 leader_election.cc:290] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8 [CANDIDATE]: Term 1 election: Requested vote from peers 1a349bcb0c8447aeb12112510e3e64c9 (127.11.63.66:41675), b7b1dad8486e4856bd4f9a6b3041e567 (127.11.63.67:46203)
I20260709 13:23:01.398921 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ef6f66168a2459d9cd62f390d878801" candidate_uuid: "6ced1532c9df44abb8736460523870c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b1dad8486e4856bd4f9a6b3041e567"
I20260709 13:23:01.398963 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ef6f66168a2459d9cd62f390d878801" candidate_uuid: "6ced1532c9df44abb8736460523870c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a349bcb0c8447aeb12112510e3e64c9"
I20260709 13:23:01.399550 23390 raft_consensus.cc:3060] T 9ef6f66168a2459d9cd62f390d878801 P b7b1dad8486e4856bd4f9a6b3041e567 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:01.399821 23316 raft_consensus.cc:3060] T 9ef6f66168a2459d9cd62f390d878801 P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:01.399950 23436 raft_consensus.cc:493] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:01.400470 23436 raft_consensus.cc:515] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:01.402750 23436 leader_election.cc:290] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ced1532c9df44abb8736460523870c8 (127.11.63.65:40629), b7b1dad8486e4856bd4f9a6b3041e567 (127.11.63.67:46203)
I20260709 13:23:01.403532 23241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "186e80e1f23f4c77a1b6fd7502979a80" candidate_uuid: "1a349bcb0c8447aeb12112510e3e64c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ced1532c9df44abb8736460523870c8" is_pre_election: true
I20260709 13:23:01.404241 23241 raft_consensus.cc:2468] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a349bcb0c8447aeb12112510e3e64c9 in term 0.
I20260709 13:23:01.404423 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "186e80e1f23f4c77a1b6fd7502979a80" candidate_uuid: "1a349bcb0c8447aeb12112510e3e64c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" is_pre_election: true
I20260709 13:23:01.405145 23389 raft_consensus.cc:2468] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a349bcb0c8447aeb12112510e3e64c9 in term 0.
I20260709 13:23:01.405782 23280 leader_election.cc:304] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a349bcb0c8447aeb12112510e3e64c9, 6ced1532c9df44abb8736460523870c8; no voters: 
I20260709 13:23:01.406446 23390 raft_consensus.cc:2468] T 9ef6f66168a2459d9cd62f390d878801 P b7b1dad8486e4856bd4f9a6b3041e567 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ced1532c9df44abb8736460523870c8 in term 1.
I20260709 13:23:01.407054 23436 raft_consensus.cc:2804] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:01.407508 23207 leader_election.cc:304] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ced1532c9df44abb8736460523870c8, b7b1dad8486e4856bd4f9a6b3041e567; no voters: 
I20260709 13:23:01.407469 23436 raft_consensus.cc:493] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:01.407814 23316 raft_consensus.cc:2468] T 9ef6f66168a2459d9cd62f390d878801 P 1a349bcb0c8447aeb12112510e3e64c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ced1532c9df44abb8736460523870c8 in term 1.
I20260709 13:23:01.408280 23436 raft_consensus.cc:3060] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:01.408530 23444 raft_consensus.cc:2804] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:01.409803 23444 raft_consensus.cc:697] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8 [term 1 LEADER]: Becoming Leader. State: Replica: 6ced1532c9df44abb8736460523870c8, State: Running, Role: LEADER
I20260709 13:23:01.410658 23444 consensus_queue.cc:237] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:01.416812 23436 raft_consensus.cc:515] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:01.419170 23436 leader_election.cc:290] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [CANDIDATE]: Term 1 election: Requested vote from peers 6ced1532c9df44abb8736460523870c8 (127.11.63.65:40629), b7b1dad8486e4856bd4f9a6b3041e567 (127.11.63.67:46203)
I20260709 13:23:01.419567 23241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "186e80e1f23f4c77a1b6fd7502979a80" candidate_uuid: "1a349bcb0c8447aeb12112510e3e64c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ced1532c9df44abb8736460523870c8"
I20260709 13:23:01.419860 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "186e80e1f23f4c77a1b6fd7502979a80" candidate_uuid: "1a349bcb0c8447aeb12112510e3e64c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b1dad8486e4856bd4f9a6b3041e567"
I20260709 13:23:01.418978 23137 catalog_manager.cc:5697] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ced1532c9df44abb8736460523870c8 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "6ced1532c9df44abb8736460523870c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:01.420258 23241 raft_consensus.cc:3060] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:01.420518 23390 raft_consensus.cc:3060] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:01.427049 23390 raft_consensus.cc:2468] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a349bcb0c8447aeb12112510e3e64c9 in term 1.
I20260709 13:23:01.427146 23241 raft_consensus.cc:2468] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a349bcb0c8447aeb12112510e3e64c9 in term 1.
I20260709 13:23:01.428303 23282 leader_election.cc:304] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a349bcb0c8447aeb12112510e3e64c9, b7b1dad8486e4856bd4f9a6b3041e567; no voters: 
I20260709 13:23:01.429054 23436 raft_consensus.cc:2804] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:01.433966 23436 raft_consensus.cc:697] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [term 1 LEADER]: Becoming Leader. State: Replica: 1a349bcb0c8447aeb12112510e3e64c9, State: Running, Role: LEADER
I20260709 13:23:01.434751 23436 consensus_queue.cc:237] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:01.443444 23137 catalog_manager.cc:5697] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a349bcb0c8447aeb12112510e3e64c9 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "1a349bcb0c8447aeb12112510e3e64c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:01.485145 23444 consensus_queue.cc:1048] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:01.497733 23241 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "035b53728f9542d397b030aeadc4e6d8"
dest_uuid: "6ced1532c9df44abb8736460523870c8"
mode: GRACEFUL
new_leader_uuid: "1a349bcb0c8447aeb12112510e3e64c9"
 from {username='slave'} at 127.0.0.1:58546
I20260709 13:23:01.514130 23241 raft_consensus.cc:606] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [term 1 LEADER]: Received request to transfer leadership to TS 1a349bcb0c8447aeb12112510e3e64c9
I20260709 13:23:01.516607 23444 consensus_queue.cc:1048] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:01.541181 23316 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "186e80e1f23f4c77a1b6fd7502979a80"
dest_uuid: "1a349bcb0c8447aeb12112510e3e64c9"
mode: GRACEFUL
new_leader_uuid: "6ced1532c9df44abb8736460523870c8"
 from {username='slave'} at 127.0.0.1:34644
I20260709 13:23:01.541858 23316 raft_consensus.cc:606] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [term 1 LEADER]: Received request to transfer leadership to TS 6ced1532c9df44abb8736460523870c8
I20260709 13:23:01.667157 23438 consensus_queue.cc:1048] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:01.679152 23438 consensus_queue.cc:1048] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:01.879045 23438 consensus_queue.cc:1048] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:01.882977 23438 consensus_queue.cc:1048] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:01.887969 23436 consensus_queue.cc:1048] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:01.912317 23477 consensus_queue.cc:1048] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:01.925031 23436 raft_consensus.cc:993] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9: : Instructing follower 6ced1532c9df44abb8736460523870c8 to start an election
I20260709 13:23:01.925688 23477 raft_consensus.cc:1081] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [term 1 LEADER]: Signalling peer 6ced1532c9df44abb8736460523870c8 to start an election
I20260709 13:23:01.927296 23241 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "186e80e1f23f4c77a1b6fd7502979a80"
dest_uuid: "6ced1532c9df44abb8736460523870c8"
 from {username='slave'} at 127.0.0.1:59090
I20260709 13:23:01.927858 23241 raft_consensus.cc:493] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:01.928169 23241 raft_consensus.cc:3060] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:01.933472 23241 raft_consensus.cc:515] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:01.935328 23241 leader_election.cc:290] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [CANDIDATE]: Term 2 election: Requested vote from peers 1a349bcb0c8447aeb12112510e3e64c9 (127.11.63.66:41675), b7b1dad8486e4856bd4f9a6b3041e567 (127.11.63.67:46203)
I20260709 13:23:01.936216 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "186e80e1f23f4c77a1b6fd7502979a80" candidate_uuid: "6ced1532c9df44abb8736460523870c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1a349bcb0c8447aeb12112510e3e64c9"
I20260709 13:23:01.936641 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "186e80e1f23f4c77a1b6fd7502979a80" candidate_uuid: "6ced1532c9df44abb8736460523870c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b7b1dad8486e4856bd4f9a6b3041e567"
I20260709 13:23:01.936928 23316 raft_consensus.cc:3055] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:01.937299 23316 raft_consensus.cc:740] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1a349bcb0c8447aeb12112510e3e64c9, State: Running, Role: LEADER
I20260709 13:23:01.937305 23390 raft_consensus.cc:3060] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:01.938467 23316 consensus_queue.cc:260] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:01.939533 23316 raft_consensus.cc:3060] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:01.944140 23390 raft_consensus.cc:2468] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ced1532c9df44abb8736460523870c8 in term 2.
I20260709 13:23:01.945293 23207 leader_election.cc:304] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ced1532c9df44abb8736460523870c8, b7b1dad8486e4856bd4f9a6b3041e567; no voters: 
I20260709 13:23:01.946090 23316 raft_consensus.cc:2468] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ced1532c9df44abb8736460523870c8 in term 2.
I20260709 13:23:01.946161 23446 raft_consensus.cc:2804] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:01.946767 23446 raft_consensus.cc:697] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [term 2 LEADER]: Becoming Leader. State: Replica: 6ced1532c9df44abb8736460523870c8, State: Running, Role: LEADER
I20260709 13:23:01.947558 23446 consensus_queue.cc:237] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:01.954207 23137 catalog_manager.cc:5697] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 reported cstate change: term changed from 1 to 2, leader changed from 1a349bcb0c8447aeb12112510e3e64c9 (127.11.63.66) to 6ced1532c9df44abb8736460523870c8 (127.11.63.65). New cstate: current_term: 2 leader_uuid: "6ced1532c9df44abb8736460523870c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:02.072033 23444 raft_consensus.cc:993] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8: : Instructing follower 1a349bcb0c8447aeb12112510e3e64c9 to start an election
I20260709 13:23:02.072583 23444 raft_consensus.cc:1081] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [term 1 LEADER]: Signalling peer 1a349bcb0c8447aeb12112510e3e64c9 to start an election
I20260709 13:23:02.074174 23316 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "035b53728f9542d397b030aeadc4e6d8"
dest_uuid: "1a349bcb0c8447aeb12112510e3e64c9"
 from {username='slave'} at 127.0.0.1:35186
I20260709 13:23:02.074751 23316 raft_consensus.cc:493] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:02.075101 23316 raft_consensus.cc:3060] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:02.080966 23316 raft_consensus.cc:515] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:02.082875 23316 leader_election.cc:290] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 [CANDIDATE]: Term 2 election: Requested vote from peers 6ced1532c9df44abb8736460523870c8 (127.11.63.65:40629), b7b1dad8486e4856bd4f9a6b3041e567 (127.11.63.67:46203)
I20260709 13:23:02.083725 23241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "035b53728f9542d397b030aeadc4e6d8" candidate_uuid: "1a349bcb0c8447aeb12112510e3e64c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ced1532c9df44abb8736460523870c8"
I20260709 13:23:02.084075 23390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "035b53728f9542d397b030aeadc4e6d8" candidate_uuid: "1a349bcb0c8447aeb12112510e3e64c9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b7b1dad8486e4856bd4f9a6b3041e567"
I20260709 13:23:02.084365 23241 raft_consensus.cc:3055] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:02.084724 23390 raft_consensus.cc:3060] T 035b53728f9542d397b030aeadc4e6d8 P b7b1dad8486e4856bd4f9a6b3041e567 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:02.084724 23241 raft_consensus.cc:740] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ced1532c9df44abb8736460523870c8, State: Running, Role: LEADER
I20260709 13:23:02.085850 23241 consensus_queue.cc:260] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:02.086871 23241 raft_consensus.cc:3060] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:02.090885 23390 raft_consensus.cc:2468] T 035b53728f9542d397b030aeadc4e6d8 P b7b1dad8486e4856bd4f9a6b3041e567 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a349bcb0c8447aeb12112510e3e64c9 in term 2.
I20260709 13:23:02.092201 23282 leader_election.cc:304] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a349bcb0c8447aeb12112510e3e64c9, b7b1dad8486e4856bd4f9a6b3041e567; no voters: 
I20260709 13:23:02.092370 23241 raft_consensus.cc:2468] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a349bcb0c8447aeb12112510e3e64c9 in term 2.
I20260709 13:23:02.093120 23477 raft_consensus.cc:2804] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:02.093721 23477 raft_consensus.cc:697] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 [term 2 LEADER]: Becoming Leader. State: Replica: 1a349bcb0c8447aeb12112510e3e64c9, State: Running, Role: LEADER
I20260709 13:23:02.094403 23477 consensus_queue.cc:237] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:02.100389 23137 catalog_manager.cc:5697] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 reported cstate change: term changed from 1 to 2, leader changed from 6ced1532c9df44abb8736460523870c8 (127.11.63.65) to 1a349bcb0c8447aeb12112510e3e64c9 (127.11.63.66). New cstate: current_term: 2 leader_uuid: "1a349bcb0c8447aeb12112510e3e64c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:02.277412 11517 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:23:02.279264 11517 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:23:02.279726 11517 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:23:02.294718 23241 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "186e80e1f23f4c77a1b6fd7502979a80"
dest_uuid: "6ced1532c9df44abb8736460523870c8"
mode: GRACEFUL
new_leader_uuid: "b7b1dad8486e4856bd4f9a6b3041e567"
 from {username='slave'} at 127.0.0.1:58556
I20260709 13:23:02.295313 23241 raft_consensus.cc:606] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [term 2 LEADER]: Received request to transfer leadership to TS b7b1dad8486e4856bd4f9a6b3041e567
I20260709 13:23:02.296545 11517 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:23:02.386261 23390 raft_consensus.cc:1275] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567 [term 2 FOLLOWER]: Refusing update from remote peer 6ced1532c9df44abb8736460523870c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:02.387884 23444 consensus_queue.cc:1048] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:02.393515 23446 raft_consensus.cc:993] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8: : Instructing follower b7b1dad8486e4856bd4f9a6b3041e567 to start an election
I20260709 13:23:02.394037 23446 raft_consensus.cc:1081] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [term 2 LEADER]: Signalling peer b7b1dad8486e4856bd4f9a6b3041e567 to start an election
I20260709 13:23:02.395656 23390 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "186e80e1f23f4c77a1b6fd7502979a80"
dest_uuid: "b7b1dad8486e4856bd4f9a6b3041e567"
 from {username='slave'} at 127.0.0.1:53914
I20260709 13:23:02.396226 23390 raft_consensus.cc:493] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:02.396620 23390 raft_consensus.cc:3060] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:02.400440 23316 raft_consensus.cc:1275] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [term 2 FOLLOWER]: Refusing update from remote peer 6ced1532c9df44abb8736460523870c8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:02.402252 23444 consensus_queue.cc:1048] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:02.403087 23390 raft_consensus.cc:515] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:02.405150 23389 raft_consensus.cc:1240] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567 [term 3 FOLLOWER]: Rejecting Update request from peer 6ced1532c9df44abb8736460523870c8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:23:02.408866 23444 consensus_queue.cc:1059] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:02.410527 23444 raft_consensus.cc:3055] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:23:02.411041 23444 raft_consensus.cc:740] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6ced1532c9df44abb8736460523870c8, State: Running, Role: LEADER
I20260709 13:23:02.412089 23444 consensus_queue.cc:260] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:02.413775 23444 raft_consensus.cc:3060] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:02.421077 23241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "186e80e1f23f4c77a1b6fd7502979a80" candidate_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6ced1532c9df44abb8736460523870c8"
I20260709 13:23:02.425760 23390 leader_election.cc:290] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567 [CANDIDATE]: Term 3 election: Requested vote from peers 6ced1532c9df44abb8736460523870c8 (127.11.63.65:40629), 1a349bcb0c8447aeb12112510e3e64c9 (127.11.63.66:41675)
I20260709 13:23:02.434222 23241 raft_consensus.cc:2468] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7b1dad8486e4856bd4f9a6b3041e567 in term 3.
I20260709 13:23:02.436270 23354 leader_election.cc:304] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ced1532c9df44abb8736460523870c8, b7b1dad8486e4856bd4f9a6b3041e567; no voters: 
I20260709 13:23:02.437256 23441 raft_consensus.cc:2804] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:23:02.438524 23441 raft_consensus.cc:697] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567 [term 3 LEADER]: Becoming Leader. State: Replica: b7b1dad8486e4856bd4f9a6b3041e567, State: Running, Role: LEADER
I20260709 13:23:02.439453 23441 consensus_queue.cc:237] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } }
I20260709 13:23:02.441915 23316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "186e80e1f23f4c77a1b6fd7502979a80" candidate_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1a349bcb0c8447aeb12112510e3e64c9"
I20260709 13:23:02.442601 23316 raft_consensus.cc:3060] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:02.446214 23138 catalog_manager.cc:5697] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567 reported cstate change: term changed from 2 to 3, leader changed from 6ced1532c9df44abb8736460523870c8 (127.11.63.65) to b7b1dad8486e4856bd4f9a6b3041e567 (127.11.63.67). New cstate: current_term: 3 leader_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a349bcb0c8447aeb12112510e3e64c9" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 41675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:02.448084 23316 raft_consensus.cc:2468] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7b1dad8486e4856bd4f9a6b3041e567 in term 3.
I20260709 13:23:02.484428 23390 raft_consensus.cc:1275] T 035b53728f9542d397b030aeadc4e6d8 P b7b1dad8486e4856bd4f9a6b3041e567 [term 2 FOLLOWER]: Refusing update from remote peer 1a349bcb0c8447aeb12112510e3e64c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:02.485801 23477 consensus_queue.cc:1048] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b1dad8486e4856bd4f9a6b3041e567" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:02.494292 23241 raft_consensus.cc:1275] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [term 2 FOLLOWER]: Refusing update from remote peer 1a349bcb0c8447aeb12112510e3e64c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:02.496457 23490 consensus_queue.cc:1048] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ced1532c9df44abb8736460523870c8" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 40629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:02.515987 11517 tablet_server.cc:179] TabletServer@127.11.63.65:0 shutting down...
I20260709 13:23:02.545921 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:23:02.546820 11517 tablet_replica.cc:333] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8: stopping tablet replica
I20260709 13:23:02.547369 11517 raft_consensus.cc:2243] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:02.547899 11517 raft_consensus.cc:2272] T 186e80e1f23f4c77a1b6fd7502979a80 P 6ced1532c9df44abb8736460523870c8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:02.550395 11517 tablet_replica.cc:333] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8: stopping tablet replica
I20260709 13:23:02.550886 11517 raft_consensus.cc:2243] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:02.551695 11517 raft_consensus.cc:2272] T 9ef6f66168a2459d9cd62f390d878801 P 6ced1532c9df44abb8736460523870c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:02.554177 11517 tablet_replica.cc:333] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8: stopping tablet replica
I20260709 13:23:02.554658 11517 raft_consensus.cc:2243] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:02.555362 11517 raft_consensus.cc:2272] T 5c0603019f674a6787e5e8f3ac2c601f P 6ced1532c9df44abb8736460523870c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:02.557502 11517 tablet_replica.cc:333] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8: stopping tablet replica
I20260709 13:23:02.558027 11517 raft_consensus.cc:2243] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:02.558444 11517 raft_consensus.cc:2272] T 035b53728f9542d397b030aeadc4e6d8 P 6ced1532c9df44abb8736460523870c8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:02.583597 11517 tablet_server.cc:196] TabletServer@127.11.63.65:0 shutdown complete.
I20260709 13:23:02.598806 11517 tablet_server.cc:179] TabletServer@127.11.63.66:0 shutting down...
I20260709 13:23:02.626816 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:23:02.627701 11517 tablet_replica.cc:333] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9: stopping tablet replica
I20260709 13:23:02.628337 11517 raft_consensus.cc:2243] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:02.628960 11517 raft_consensus.cc:2272] T 186e80e1f23f4c77a1b6fd7502979a80 P 1a349bcb0c8447aeb12112510e3e64c9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:02.631434 11517 tablet_replica.cc:333] T 9ef6f66168a2459d9cd62f390d878801 P 1a349bcb0c8447aeb12112510e3e64c9: stopping tablet replica
I20260709 13:23:02.632006 11517 raft_consensus.cc:2243] T 9ef6f66168a2459d9cd62f390d878801 P 1a349bcb0c8447aeb12112510e3e64c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:02.632460 11517 raft_consensus.cc:2272] T 9ef6f66168a2459d9cd62f390d878801 P 1a349bcb0c8447aeb12112510e3e64c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:02.635197 11517 tablet_replica.cc:333] T 5c0603019f674a6787e5e8f3ac2c601f P 1a349bcb0c8447aeb12112510e3e64c9: stopping tablet replica
I20260709 13:23:02.635730 11517 raft_consensus.cc:2243] T 5c0603019f674a6787e5e8f3ac2c601f P 1a349bcb0c8447aeb12112510e3e64c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:02.636142 11517 raft_consensus.cc:2272] T 5c0603019f674a6787e5e8f3ac2c601f P 1a349bcb0c8447aeb12112510e3e64c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:02.638525 11517 tablet_replica.cc:333] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9: stopping tablet replica
I20260709 13:23:02.639045 11517 raft_consensus.cc:2243] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:02.640017 11517 raft_consensus.cc:2272] T 035b53728f9542d397b030aeadc4e6d8 P 1a349bcb0c8447aeb12112510e3e64c9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:02.664677 11517 tablet_server.cc:196] TabletServer@127.11.63.66:0 shutdown complete.
I20260709 13:23:02.678987 11517 tablet_server.cc:179] TabletServer@127.11.63.67:0 shutting down...
I20260709 13:23:02.701189 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:23:02.702052 11517 tablet_replica.cc:333] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567: stopping tablet replica
I20260709 13:23:02.702615 11517 raft_consensus.cc:2243] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:23:02.703351 11517 pending_rounds.cc:70] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567: Trying to abort 2 pending ops.
I20260709 13:23:02.703593 11517 pending_rounds.cc:77] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305639454481670144 op_type: NO_OP noop_request { }
I20260709 13:23:02.703907 11517 raft_consensus.cc:2889] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:23:02.704149 11517 pending_rounds.cc:77] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305639452465082368 op_type: NO_OP noop_request { }
I20260709 13:23:02.704392 11517 raft_consensus.cc:2889] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:23:02.704671 11517 raft_consensus.cc:2272] T 186e80e1f23f4c77a1b6fd7502979a80 P b7b1dad8486e4856bd4f9a6b3041e567 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:02.707230 11517 tablet_replica.cc:333] T 9ef6f66168a2459d9cd62f390d878801 P b7b1dad8486e4856bd4f9a6b3041e567: stopping tablet replica
I20260709 13:23:02.707790 11517 raft_consensus.cc:2243] T 9ef6f66168a2459d9cd62f390d878801 P b7b1dad8486e4856bd4f9a6b3041e567 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:02.708218 11517 raft_consensus.cc:2272] T 9ef6f66168a2459d9cd62f390d878801 P b7b1dad8486e4856bd4f9a6b3041e567 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:02.711045 11517 tablet_replica.cc:333] T 5c0603019f674a6787e5e8f3ac2c601f P b7b1dad8486e4856bd4f9a6b3041e567: stopping tablet replica
I20260709 13:23:02.711617 11517 raft_consensus.cc:2243] T 5c0603019f674a6787e5e8f3ac2c601f P b7b1dad8486e4856bd4f9a6b3041e567 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:02.712051 11517 raft_consensus.cc:2272] T 5c0603019f674a6787e5e8f3ac2c601f P b7b1dad8486e4856bd4f9a6b3041e567 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:02.713951 11517 tablet_replica.cc:333] T 035b53728f9542d397b030aeadc4e6d8 P b7b1dad8486e4856bd4f9a6b3041e567: stopping tablet replica
I20260709 13:23:02.714432 11517 raft_consensus.cc:2243] T 035b53728f9542d397b030aeadc4e6d8 P b7b1dad8486e4856bd4f9a6b3041e567 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:02.714862 11517 raft_consensus.cc:2272] T 035b53728f9542d397b030aeadc4e6d8 P b7b1dad8486e4856bd4f9a6b3041e567 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:02.739507 11517 tablet_server.cc:196] TabletServer@127.11.63.67:0 shutdown complete.
I20260709 13:23:02.756506 11517 master.cc:562] Master@127.11.63.126:44045 shutting down...
I20260709 13:23:02.779183 11517 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:02.779956 11517 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:02.780308 11517 tablet_replica.cc:333] T 00000000000000000000000000000000 P 09f31d81a9de4d3eaefd219af9031e18: stopping tablet replica
I20260709 13:23:02.801112 11517 master.cc:584] Master@127.11.63.126:44045 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2714 ms)
[----------] 6 tests from LeaderRebalancerTest (363369 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:23:02.843892 11517 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.11.63.126:37329
I20260709 13:23:02.845191 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:23:02.851437 23493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:02.851469 23492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:02.855139 23495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:23:02.855209 11517 server_base.cc:1061] running on GCE node
I20260709 13:23:02.856410 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:23:02.856655 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:23:02.856825 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603382856804 us; error 0 us; skew 500 ppm
I20260709 13:23:02.857350 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:23:02.860292 11517 webserver.cc:533] Webserver started at http://127.11.63.126:36103/ using document root <none> and password file <none>
I20260709 13:23:02.860822 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:23:02.861035 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:23:02.861315 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:23:02.862602 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603019296230-11517-0/minicluster-data/master-0-root/instance:
uuid: "2633b9a6b4c7452a9f983097f7ba060e"
format_stamp: "Formatted at 2026-07-09 13:23:02 on dist-test-slave-qv7r"
I20260709 13:23:02.867621 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:23:02.871991 23500 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:02.873021 11517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:23:02.873559 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603019296230-11517-0/minicluster-data/master-0-root
uuid: "2633b9a6b4c7452a9f983097f7ba060e"
format_stamp: "Formatted at 2026-07-09 13:23:02 on dist-test-slave-qv7r"
I20260709 13:23:02.873939 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603019296230-11517-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603019296230-11517-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603019296230-11517-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:23:02.883150 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:23:02.884503 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:23:02.927793 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.126:37329
I20260709 13:23:02.927906 23551 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.126:37329 every 8 connection(s)
I20260709 13:23:02.932170 23552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:02.943437 23552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e: Bootstrap starting.
I20260709 13:23:02.948254 23552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:02.953274 23552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e: No bootstrap required, opened a new log
I20260709 13:23:02.956550 23552 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2633b9a6b4c7452a9f983097f7ba060e" member_type: VOTER }
I20260709 13:23:02.956962 23552 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:02.957324 23552 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2633b9a6b4c7452a9f983097f7ba060e, State: Initialized, Role: FOLLOWER
I20260709 13:23:02.958276 23552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2633b9a6b4c7452a9f983097f7ba060e" member_type: VOTER }
I20260709 13:23:02.958961 23552 raft_consensus.cc:399] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:23:02.959256 23552 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:23:02.960004 23552 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:02.965893 23552 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2633b9a6b4c7452a9f983097f7ba060e" member_type: VOTER }
I20260709 13:23:02.966532 23552 leader_election.cc:304] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2633b9a6b4c7452a9f983097f7ba060e; no voters: 
I20260709 13:23:02.967691 23552 leader_election.cc:290] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:23:02.968140 23555 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:02.969496 23555 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e [term 1 LEADER]: Becoming Leader. State: Replica: 2633b9a6b4c7452a9f983097f7ba060e, State: Running, Role: LEADER
I20260709 13:23:02.970173 23555 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2633b9a6b4c7452a9f983097f7ba060e" member_type: VOTER }
I20260709 13:23:02.970611 23552 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:23:02.973031 23557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2633b9a6b4c7452a9f983097f7ba060e. Latest consensus state: current_term: 1 leader_uuid: "2633b9a6b4c7452a9f983097f7ba060e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2633b9a6b4c7452a9f983097f7ba060e" member_type: VOTER } }
I20260709 13:23:02.972918 23556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2633b9a6b4c7452a9f983097f7ba060e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2633b9a6b4c7452a9f983097f7ba060e" member_type: VOTER } }
I20260709 13:23:02.973716 23557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e [sys.catalog]: This master's current role is: LEADER
I20260709 13:23:02.973790 23556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e [sys.catalog]: This master's current role is: LEADER
I20260709 13:23:02.979059 23560 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:23:02.983695 23560 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:23:02.985882 11517 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:23:02.993040 23560 catalog_manager.cc:1383] Generated new cluster ID: 222246847f1049b1bdcb5f02b999a9ab
I20260709 13:23:02.993342 23560 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:23:03.013983 23560 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:23:03.015262 23560 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:23:03.031273 23560 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e: Generated new TSK 0
I20260709 13:23:03.032065 23560 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:23:03.052361 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:23:03.058913 23573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:03.060536 23574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:23:03.062626 11517 server_base.cc:1061] running on GCE node
W20260709 13:23:03.062736 23576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:23:03.063709 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:23:03.063935 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:23:03.064110 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603383064089 us; error 0 us; skew 500 ppm
I20260709 13:23:03.064640 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:23:03.067204 11517 webserver.cc:533] Webserver started at http://127.11.63.65:46675/ using document root <none> and password file <none>
I20260709 13:23:03.067747 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:23:03.067955 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:23:03.068228 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:23:03.069376 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603019296230-11517-0/minicluster-data/ts-0-root/instance:
uuid: "025b14df93f44ae39aefc3c0573cc19a"
format_stamp: "Formatted at 2026-07-09 13:23:03 on dist-test-slave-qv7r"
I20260709 13:23:03.074393 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:23:03.078120 23581 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:03.079006 11517 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.002s
I20260709 13:23:03.079267 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603019296230-11517-0/minicluster-data/ts-0-root
uuid: "025b14df93f44ae39aefc3c0573cc19a"
format_stamp: "Formatted at 2026-07-09 13:23:03 on dist-test-slave-qv7r"
I20260709 13:23:03.079521 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603019296230-11517-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603019296230-11517-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603019296230-11517-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:23:03.090034 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:23:03.091256 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:23:03.092796 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:23:03.095381 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:23:03.095597 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:03.095794 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:23:03.095948 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:03.139380 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.65:44419
I20260709 13:23:03.139472 23643 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.65:44419 every 8 connection(s)
I20260709 13:23:03.144352 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:23:03.152904 23649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:03.154131 23648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:03.154706 23651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:23:03.155880 11517 server_base.cc:1061] running on GCE node
I20260709 13:23:03.157670 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:23:03.157943 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:23:03.158121 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603383158088 us; error 0 us; skew 500 ppm
I20260709 13:23:03.158835 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:23:03.158857 23644 heartbeater.cc:344] Connected to a master server at 127.11.63.126:37329
I20260709 13:23:03.159376 23644 heartbeater.cc:461] Registering TS with master...
I20260709 13:23:03.160395 23644 heartbeater.cc:507] Master 127.11.63.126:37329 requested a full tablet report, sending...
I20260709 13:23:03.161777 11517 webserver.cc:533] Webserver started at http://127.11.63.66:37397/ using document root <none> and password file <none>
I20260709 13:23:03.162463 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:23:03.162659 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:23:03.162933 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:23:03.162891 23517 ts_manager.cc:194] Registered new tserver with Master: 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419)
I20260709 13:23:03.164407 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603019296230-11517-0/minicluster-data/ts-1-root/instance:
uuid: "b403848a6de0483c82db9cd37d1d5a2f"
format_stamp: "Formatted at 2026-07-09 13:23:03 on dist-test-slave-qv7r"
I20260709 13:23:03.165170 23517 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35602
I20260709 13:23:03.169170 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:23:03.173187 23656 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:03.174088 11517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:23:03.174569 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603019296230-11517-0/minicluster-data/ts-1-root
uuid: "b403848a6de0483c82db9cd37d1d5a2f"
format_stamp: "Formatted at 2026-07-09 13:23:03 on dist-test-slave-qv7r"
I20260709 13:23:03.174856 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603019296230-11517-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603019296230-11517-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603019296230-11517-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:23:03.184069 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:23:03.185151 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:23:03.186856 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:23:03.189079 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:23:03.189306 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:03.189550 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:23:03.189745 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:23:03.230093 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.66:36121
I20260709 13:23:03.230218 23718 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.66:36121 every 8 connection(s)
I20260709 13:23:03.235055 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:23:03.243280 23723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:03.245477 23722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:03.245791 23725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:23:03.248626 11517 server_base.cc:1061] running on GCE node
I20260709 13:23:03.248862 23719 heartbeater.cc:344] Connected to a master server at 127.11.63.126:37329
I20260709 13:23:03.249353 23719 heartbeater.cc:461] Registering TS with master...
I20260709 13:23:03.249866 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:23:03.250098 23719 heartbeater.cc:507] Master 127.11.63.126:37329 requested a full tablet report, sending...
W20260709 13:23:03.250170 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:23:03.250568 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603383250548 us; error 0 us; skew 500 ppm
I20260709 13:23:03.251441 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:23:03.252648 23517 ts_manager.cc:194] Registered new tserver with Master: b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121)
I20260709 13:23:03.254741 11517 webserver.cc:533] Webserver started at http://127.11.63.67:34019/ using document root <none> and password file <none>
I20260709 13:23:03.254850 23517 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35606
I20260709 13:23:03.255519 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:23:03.255784 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:23:03.256109 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:23:03.257349 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603019296230-11517-0/minicluster-data/ts-2-root/instance:
uuid: "11398324f1664f29be659b7034c8eedd"
format_stamp: "Formatted at 2026-07-09 13:23:03 on dist-test-slave-qv7r"
I20260709 13:23:03.261801 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:23:03.265234 23730 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:03.266153 11517 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:23:03.266402 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603019296230-11517-0/minicluster-data/ts-2-root
uuid: "11398324f1664f29be659b7034c8eedd"
format_stamp: "Formatted at 2026-07-09 13:23:03 on dist-test-slave-qv7r"
I20260709 13:23:03.266667 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603019296230-11517-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603019296230-11517-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783603019296230-11517-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:23:03.283344 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:23:03.284399 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:23:03.286041 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:23:03.288228 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:23:03.288447 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:03.288673 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:23:03.288839 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:03.330926 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.67:38311
I20260709 13:23:03.331086 23792 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.67:38311 every 8 connection(s)
I20260709 13:23:03.345127 23793 heartbeater.cc:344] Connected to a master server at 127.11.63.126:37329
I20260709 13:23:03.345638 23793 heartbeater.cc:461] Registering TS with master...
I20260709 13:23:03.346382 23793 heartbeater.cc:507] Master 127.11.63.126:37329 requested a full tablet report, sending...
I20260709 13:23:03.348474 23517 ts_manager.cc:194] Registered new tserver with Master: 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:03.348940 11517 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014822846s
I20260709 13:23:03.349957 23517 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35622
I20260709 13:23:03.375316 23517 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35634:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:23:03.378036 23517 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:23:03.445525 23684 tablet_service.cc:1511] Processing CreateTablet for tablet 91bc474b5f64410d9a9c21058a2feca0 (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:23:03.447111 23684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91bc474b5f64410d9a9c21058a2feca0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.450827 23683 tablet_service.cc:1511] Processing CreateTablet for tablet 84909b8ded8d40518c017313ef8bfeb8 (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:23:03.452543 23683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84909b8ded8d40518c017313ef8bfeb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.454844 23682 tablet_service.cc:1511] Processing CreateTablet for tablet 10e65bc4fc654cd6a302b592b8b3a733 (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:23:03.456296 23682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10e65bc4fc654cd6a302b592b8b3a733. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.465425 23609 tablet_service.cc:1511] Processing CreateTablet for tablet 91bc474b5f64410d9a9c21058a2feca0 (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:23:03.466952 23609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91bc474b5f64410d9a9c21058a2feca0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.469283 23681 tablet_service.cc:1511] Processing CreateTablet for tablet d524509732f8487ba5f6ad44e5811ecb (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:23:03.470889 23681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d524509732f8487ba5f6ad44e5811ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.469384 23608 tablet_service.cc:1511] Processing CreateTablet for tablet 84909b8ded8d40518c017313ef8bfeb8 (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:23:03.472255 23606 tablet_service.cc:1511] Processing CreateTablet for tablet d524509732f8487ba5f6ad44e5811ecb (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:23:03.470498 23607 tablet_service.cc:1511] Processing CreateTablet for tablet 10e65bc4fc654cd6a302b592b8b3a733 (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:23:03.473990 23608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84909b8ded8d40518c017313ef8bfeb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.476055 23607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10e65bc4fc654cd6a302b592b8b3a733. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.477231 23606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d524509732f8487ba5f6ad44e5811ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.479967 23605 tablet_service.cc:1511] Processing CreateTablet for tablet f909688025304faa9849596796e56988 (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:23:03.481729 23605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f909688025304faa9849596796e56988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.494501 23683 tablet_service.cc:1511] Processing CreateTablet for tablet f909688025304faa9849596796e56988 (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:23:03.496197 23683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f909688025304faa9849596796e56988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.503626 23604 tablet_service.cc:1511] Processing CreateTablet for tablet 4c184e66c94b4436a4a2028ec5665be5 (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:23:03.505133 23604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c184e66c94b4436a4a2028ec5665be5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.507194 23757 tablet_service.cc:1511] Processing CreateTablet for tablet 84909b8ded8d40518c017313ef8bfeb8 (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:23:03.508646 23757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84909b8ded8d40518c017313ef8bfeb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.510654 23758 tablet_service.cc:1511] Processing CreateTablet for tablet 91bc474b5f64410d9a9c21058a2feca0 (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:23:03.511752 23755 tablet_service.cc:1511] Processing CreateTablet for tablet d524509732f8487ba5f6ad44e5811ecb (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:23:03.510650 23756 tablet_service.cc:1511] Processing CreateTablet for tablet 10e65bc4fc654cd6a302b592b8b3a733 (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:23:03.514060 23754 tablet_service.cc:1511] Processing CreateTablet for tablet f909688025304faa9849596796e56988 (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:23:03.515027 23753 tablet_service.cc:1511] Processing CreateTablet for tablet 4c184e66c94b4436a4a2028ec5665be5 (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:23:03.516711 23603 tablet_service.cc:1511] Processing CreateTablet for tablet 109ce4dfd3d44642bc560d6cb563ceef (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:23:03.518121 23603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 109ce4dfd3d44642bc560d6cb563ceef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.521956 23758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91bc474b5f64410d9a9c21058a2feca0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.521944 23680 tablet_service.cc:1511] Processing CreateTablet for tablet 109ce4dfd3d44642bc560d6cb563ceef (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:23:03.523547 23680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 109ce4dfd3d44642bc560d6cb563ceef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.524909 23602 tablet_service.cc:1511] Processing CreateTablet for tablet 8531f6c11726411eab62bfa778de8701 (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:23:03.526297 23602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8531f6c11726411eab62bfa778de8701. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.527051 23753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c184e66c94b4436a4a2028ec5665be5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.529189 23752 tablet_service.cc:1511] Processing CreateTablet for tablet 109ce4dfd3d44642bc560d6cb563ceef (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:23:03.530505 23752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 109ce4dfd3d44642bc560d6cb563ceef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.532449 23751 tablet_service.cc:1511] Processing CreateTablet for tablet 8531f6c11726411eab62bfa778de8701 (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:23:03.534588 23601 tablet_service.cc:1511] Processing CreateTablet for tablet f7cf9d97b16441ca84a93f2e71f0af81 (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:23:03.535964 23601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7cf9d97b16441ca84a93f2e71f0af81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.529189 23679 tablet_service.cc:1511] Processing CreateTablet for tablet 8531f6c11726411eab62bfa778de8701 (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:23:03.539005 23679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8531f6c11726411eab62bfa778de8701. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.540629 23750 tablet_service.cc:1511] Processing CreateTablet for tablet f7cf9d97b16441ca84a93f2e71f0af81 (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:23:03.542790 23678 tablet_service.cc:1511] Processing CreateTablet for tablet f7cf9d97b16441ca84a93f2e71f0af81 (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:23:03.544145 23678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7cf9d97b16441ca84a93f2e71f0af81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.550060 23750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7cf9d97b16441ca84a93f2e71f0af81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.552659 23751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8531f6c11726411eab62bfa778de8701. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.561952 23755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d524509732f8487ba5f6ad44e5811ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.565898 23754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f909688025304faa9849596796e56988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.568054 23756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10e65bc4fc654cd6a302b592b8b3a733. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.510727 23681 tablet_service.cc:1511] Processing CreateTablet for tablet 4c184e66c94b4436a4a2028ec5665be5 (DEFAULT_TABLE table=test-workload [id=2b9d357a036d4ac7bfae2d757eca42f3]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:23:03.602386 23681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c184e66c94b4436a4a2028ec5665be5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:03.648849 23810 tablet_bootstrap.cc:492] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd: Bootstrap starting.
I20260709 13:23:03.656488 23809 tablet_bootstrap.cc:492] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a: Bootstrap starting.
I20260709 13:23:03.665261 23808 tablet_bootstrap.cc:492] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f: Bootstrap starting.
I20260709 13:23:03.667603 23810 tablet_bootstrap.cc:654] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:03.672722 23808 tablet_bootstrap.cc:654] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:03.682183 23809 tablet_bootstrap.cc:654] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:03.691488 23809 tablet_bootstrap.cc:492] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a: No bootstrap required, opened a new log
I20260709 13:23:03.692240 23809 ts_tablet_manager.cc:1403] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a: Time spent bootstrapping tablet: real 0.036s	user 0.007s	sys 0.004s
I20260709 13:23:03.691490 23810 tablet_bootstrap.cc:492] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd: No bootstrap required, opened a new log
I20260709 13:23:03.696223 23810 ts_tablet_manager.cc:1403] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd: Time spent bootstrapping tablet: real 0.048s	user 0.009s	sys 0.005s
I20260709 13:23:03.695731 23809 raft_consensus.cc:359] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.696671 23809 raft_consensus.cc:385] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:03.697010 23809 raft_consensus.cc:740] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Initialized, Role: FOLLOWER
I20260709 13:23:03.697209 23808 tablet_bootstrap.cc:492] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f: No bootstrap required, opened a new log
I20260709 13:23:03.698117 23808 ts_tablet_manager.cc:1403] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f: Time spent bootstrapping tablet: real 0.033s	user 0.009s	sys 0.004s
I20260709 13:23:03.697841 23809 consensus_queue.cc:260] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.699124 23810 raft_consensus.cc:359] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.699872 23810 raft_consensus.cc:385] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:03.700172 23810 raft_consensus.cc:740] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Initialized, Role: FOLLOWER
I20260709 13:23:03.700335 23644 heartbeater.cc:499] Master 127.11.63.126:37329 was elected leader, sending a full tablet report...
I20260709 13:23:03.700845 23809 ts_tablet_manager.cc:1434] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:23:03.700910 23810 consensus_queue.cc:260] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.701167 23808 raft_consensus.cc:359] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.702024 23808 raft_consensus.cc:385] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:03.702078 23809 tablet_bootstrap.cc:492] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a: Bootstrap starting.
I20260709 13:23:03.702384 23808 raft_consensus.cc:740] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Initialized, Role: FOLLOWER
I20260709 13:23:03.703594 23808 consensus_queue.cc:260] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.709201 23809 tablet_bootstrap.cc:654] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:03.710990 23808 ts_tablet_manager.cc:1434] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.004s
I20260709 13:23:03.712767 23719 heartbeater.cc:499] Master 127.11.63.126:37329 was elected leader, sending a full tablet report...
I20260709 13:23:03.713254 23810 ts_tablet_manager.cc:1434] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd: Time spent starting tablet: real 0.017s	user 0.013s	sys 0.004s
I20260709 13:23:03.714355 23808 tablet_bootstrap.cc:492] T 84909b8ded8d40518c017313ef8bfeb8 P b403848a6de0483c82db9cd37d1d5a2f: Bootstrap starting.
I20260709 13:23:03.721050 23793 heartbeater.cc:499] Master 127.11.63.126:37329 was elected leader, sending a full tablet report...
I20260709 13:23:03.721484 23808 tablet_bootstrap.cc:654] T 84909b8ded8d40518c017313ef8bfeb8 P b403848a6de0483c82db9cd37d1d5a2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:03.726579 23810 tablet_bootstrap.cc:492] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd: Bootstrap starting.
I20260709 13:23:03.727527 23809 tablet_bootstrap.cc:492] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a: No bootstrap required, opened a new log
I20260709 13:23:03.728132 23809 ts_tablet_manager.cc:1403] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.013s
I20260709 13:23:03.729493 23808 tablet_bootstrap.cc:492] T 84909b8ded8d40518c017313ef8bfeb8 P b403848a6de0483c82db9cd37d1d5a2f: No bootstrap required, opened a new log
I20260709 13:23:03.730083 23808 ts_tablet_manager.cc:1403] T 84909b8ded8d40518c017313ef8bfeb8 P b403848a6de0483c82db9cd37d1d5a2f: Time spent bootstrapping tablet: real 0.016s	user 0.007s	sys 0.005s
I20260709 13:23:03.731356 23809 raft_consensus.cc:359] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.732228 23809 raft_consensus.cc:385] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:03.732383 23810 tablet_bootstrap.cc:654] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:03.732601 23809 raft_consensus.cc:740] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Initialized, Role: FOLLOWER
I20260709 13:23:03.733260 23808 raft_consensus.cc:359] T 84909b8ded8d40518c017313ef8bfeb8 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.733731 23809 consensus_queue.cc:260] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.734956 23808 raft_consensus.cc:385] T 84909b8ded8d40518c017313ef8bfeb8 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:03.735342 23808 raft_consensus.cc:740] T 84909b8ded8d40518c017313ef8bfeb8 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Initialized, Role: FOLLOWER
I20260709 13:23:03.736485 23809 ts_tablet_manager.cc:1434] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:23:03.736208 23808 consensus_queue.cc:260] T 84909b8ded8d40518c017313ef8bfeb8 P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.737694 23809 tablet_bootstrap.cc:492] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a: Bootstrap starting.
I20260709 13:23:03.738895 23808 ts_tablet_manager.cc:1434] T 84909b8ded8d40518c017313ef8bfeb8 P b403848a6de0483c82db9cd37d1d5a2f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:23:03.740007 23808 tablet_bootstrap.cc:492] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f: Bootstrap starting.
I20260709 13:23:03.745550 23809 tablet_bootstrap.cc:654] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:03.747176 23808 tablet_bootstrap.cc:654] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:03.753234 23810 tablet_bootstrap.cc:492] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd: No bootstrap required, opened a new log
I20260709 13:23:03.753981 23810 ts_tablet_manager.cc:1403] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.016s
I20260709 13:23:03.756619 23809 tablet_bootstrap.cc:492] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a: No bootstrap required, opened a new log
I20260709 13:23:03.757243 23809 ts_tablet_manager.cc:1403] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.003s
I20260709 13:23:03.757153 23810 raft_consensus.cc:359] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.758193 23810 raft_consensus.cc:385] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:03.758548 23810 raft_consensus.cc:740] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Initialized, Role: FOLLOWER
I20260709 13:23:03.759517 23810 consensus_queue.cc:260] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.760764 23809 raft_consensus.cc:359] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.761622 23809 raft_consensus.cc:385] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:03.761936 23809 raft_consensus.cc:740] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Initialized, Role: FOLLOWER
I20260709 13:23:03.761940 23810 ts_tablet_manager.cc:1434] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:23:03.763458 23810 tablet_bootstrap.cc:492] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd: Bootstrap starting.
I20260709 13:23:03.765650 23808 tablet_bootstrap.cc:492] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f: No bootstrap required, opened a new log
I20260709 13:23:03.766229 23808 ts_tablet_manager.cc:1403] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.000s
I20260709 13:23:03.769524 23808 raft_consensus.cc:359] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.770201 23810 tablet_bootstrap.cc:654] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:03.762954 23809 consensus_queue.cc:260] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.772382 23809 ts_tablet_manager.cc:1434] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:23:03.770318 23808 raft_consensus.cc:385] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:03.773756 23808 raft_consensus.cc:740] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Initialized, Role: FOLLOWER
I20260709 13:23:03.774868 23808 consensus_queue.cc:260] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.777875 23808 ts_tablet_manager.cc:1434] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.004s
I20260709 13:23:03.779237 23808 tablet_bootstrap.cc:492] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f: Bootstrap starting.
I20260709 13:23:03.781587 23809 tablet_bootstrap.cc:492] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a: Bootstrap starting.
I20260709 13:23:03.788570 23810 tablet_bootstrap.cc:492] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd: No bootstrap required, opened a new log
I20260709 13:23:03.789106 23810 ts_tablet_manager.cc:1403] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.012s
I20260709 13:23:03.789253 23809 tablet_bootstrap.cc:654] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:03.791265 23808 tablet_bootstrap.cc:654] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:03.791385 23810 raft_consensus.cc:359] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.792132 23810 raft_consensus.cc:385] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:03.792373 23810 raft_consensus.cc:740] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Initialized, Role: FOLLOWER
I20260709 13:23:03.793071 23810 consensus_queue.cc:260] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.794915 23816 raft_consensus.cc:493] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:03.795410 23816 raft_consensus.cc:515] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.797972 23816 leader_election.cc:290] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:03.801851 23809 tablet_bootstrap.cc:492] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a: No bootstrap required, opened a new log
I20260709 13:23:03.802619 23809 ts_tablet_manager.cc:1403] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.006s
I20260709 13:23:03.806640 23809 raft_consensus.cc:359] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.810442 23809 raft_consensus.cc:385] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:03.806684 23810 ts_tablet_manager.cc:1434] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:23:03.810803 23809 raft_consensus.cc:740] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Initialized, Role: FOLLOWER
I20260709 13:23:03.811967 23810 tablet_bootstrap.cc:492] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd: Bootstrap starting.
I20260709 13:23:03.813171 23808 tablet_bootstrap.cc:492] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f: No bootstrap required, opened a new log
I20260709 13:23:03.813928 23808 ts_tablet_manager.cc:1403] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f: Time spent bootstrapping tablet: real 0.035s	user 0.008s	sys 0.020s
I20260709 13:23:03.817452 23808 raft_consensus.cc:359] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.818367 23808 raft_consensus.cc:385] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:03.818717 23808 raft_consensus.cc:740] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Initialized, Role: FOLLOWER
I20260709 13:23:03.819409 23810 tablet_bootstrap.cc:654] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:03.819617 23808 consensus_queue.cc:260] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.821947 23809 consensus_queue.cc:260] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.826542 23808 ts_tablet_manager.cc:1434] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.004s
I20260709 13:23:03.827174 23809 ts_tablet_manager.cc:1434] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.001s
I20260709 13:23:03.827872 23808 tablet_bootstrap.cc:492] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f: Bootstrap starting.
I20260709 13:23:03.829603 23809 tablet_bootstrap.cc:492] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a: Bootstrap starting.
I20260709 13:23:03.831008 23810 tablet_bootstrap.cc:492] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd: No bootstrap required, opened a new log
I20260709 13:23:03.831626 23810 ts_tablet_manager.cc:1403] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.003s
I20260709 13:23:03.834148 23815 raft_consensus.cc:493] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:03.834707 23815 raft_consensus.cc:515] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.834920 23810 raft_consensus.cc:359] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.835716 23810 raft_consensus.cc:385] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:03.836069 23810 raft_consensus.cc:740] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Initialized, Role: FOLLOWER
I20260709 13:23:03.844712 23814 raft_consensus.cc:493] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:03.845432 23814 raft_consensus.cc:515] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.846081 23809 tablet_bootstrap.cc:654] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:03.848860 23808 tablet_bootstrap.cc:654] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:03.855473 23815 leader_election.cc:290] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121)
I20260709 13:23:03.849582 23814 leader_election.cc:290] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:03.862102 23810 consensus_queue.cc:260] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.870994 23816 raft_consensus.cc:493] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:03.875934 23810 ts_tablet_manager.cc:1434] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd: Time spent starting tablet: real 0.044s	user 0.016s	sys 0.000s
I20260709 13:23:03.875669 23816 raft_consensus.cc:515] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.878389 23810 tablet_bootstrap.cc:492] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd: Bootstrap starting.
I20260709 13:23:03.883500 23816 leader_election.cc:290] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:03.885530 23815 raft_consensus.cc:493] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:03.886153 23815 raft_consensus.cc:515] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.888502 23815 leader_election.cc:290] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121)
I20260709 13:23:03.882627 23814 raft_consensus.cc:493] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:03.911968 23810 tablet_bootstrap.cc:654] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:03.896693 23814 raft_consensus.cc:515] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.916862 23814 leader_election.cc:290] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:03.925426 23809 tablet_bootstrap.cc:492] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a: No bootstrap required, opened a new log
I20260709 13:23:03.926204 23809 ts_tablet_manager.cc:1403] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a: Time spent bootstrapping tablet: real 0.097s	user 0.028s	sys 0.011s
I20260709 13:23:03.926266 23808 tablet_bootstrap.cc:492] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f: No bootstrap required, opened a new log
I20260709 13:23:03.926914 23808 ts_tablet_manager.cc:1403] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f: Time spent bootstrapping tablet: real 0.099s	user 0.014s	sys 0.004s
I20260709 13:23:03.931733 23808 raft_consensus.cc:359] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.932493 23808 raft_consensus.cc:385] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:03.932816 23808 raft_consensus.cc:740] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Initialized, Role: FOLLOWER
I20260709 13:23:03.933687 23808 consensus_queue.cc:260] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.935981 23808 ts_tablet_manager.cc:1434] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:23:03.937026 23808 tablet_bootstrap.cc:492] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f: Bootstrap starting.
I20260709 13:23:03.939216 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "109ce4dfd3d44642bc560d6cb563ceef" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a" is_pre_election: true
I20260709 13:23:03.940812 23694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "109ce4dfd3d44642bc560d6cb563ceef" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f" is_pre_election: true
I20260709 13:23:03.941021 23768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bc474b5f64410d9a9c21058a2feca0" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd" is_pre_election: true
I20260709 13:23:03.941720 23694 raft_consensus.cc:2468] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 0.
I20260709 13:23:03.942574 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f909688025304faa9849596796e56988" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a" is_pre_election: true
I20260709 13:23:03.943315 23618 raft_consensus.cc:2468] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 0.
I20260709 13:23:03.943428 23694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f909688025304faa9849596796e56988" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f" is_pre_election: true
I20260709 13:23:03.943907 23808 tablet_bootstrap.cc:654] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:03.944042 23694 raft_consensus.cc:2468] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 0.
I20260709 13:23:03.944509 23767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f909688025304faa9849596796e56988" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd" is_pre_election: true
I20260709 13:23:03.945178 23767 raft_consensus.cc:2468] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 0.
I20260709 13:23:03.945672 23809 raft_consensus.cc:359] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.946233 23731 leader_election.cc:304] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11398324f1664f29be659b7034c8eedd, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:03.946610 23809 raft_consensus.cc:385] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:03.947085 23809 raft_consensus.cc:740] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Initialized, Role: FOLLOWER
I20260709 13:23:03.947417 23731 leader_election.cc:304] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11398324f1664f29be659b7034c8eedd, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:03.947824 23809 consensus_queue.cc:260] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.948953 23766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84909b8ded8d40518c017313ef8bfeb8" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd" is_pre_election: true
I20260709 13:23:03.949743 23815 raft_consensus.cc:2804] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:03.950172 23815 raft_consensus.cc:493] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:03.950500 23815 raft_consensus.cc:3060] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:03.950753 23809 ts_tablet_manager.cc:1434] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:23:03.952806 23809 tablet_bootstrap.cc:492] T 109ce4dfd3d44642bc560d6cb563ceef P 025b14df93f44ae39aefc3c0573cc19a: Bootstrap starting.
W20260709 13:23:03.959600 23658 leader_election.cc:343] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:23:03.960023 23733 leader_election.cc:343] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:03.960525 23814 raft_consensus.cc:493] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:03.959638 23766 raft_consensus.cc:2468] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 0.
I20260709 13:23:03.961012 23814 raft_consensus.cc:515] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.962407 23583 leader_election.cc:304] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, 11398324f1664f29be659b7034c8eedd; no voters: 
I20260709 13:23:03.962965 23810 tablet_bootstrap.cc:492] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd: No bootstrap required, opened a new log
I20260709 13:23:03.963284 23814 leader_election.cc:290] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:03.963521 23810 ts_tablet_manager.cc:1403] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd: Time spent bootstrapping tablet: real 0.085s	user 0.016s	sys 0.000s
I20260709 13:23:03.963894 23809 tablet_bootstrap.cc:654] T 109ce4dfd3d44642bc560d6cb563ceef P 025b14df93f44ae39aefc3c0573cc19a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:03.963829 23814 raft_consensus.cc:2804] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:03.966423 23814 raft_consensus.cc:493] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:03.966782 23814 raft_consensus.cc:3060] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:03.966820 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f909688025304faa9849596796e56988" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a" is_pre_election: true
I20260709 13:23:03.967573 23619 raft_consensus.cc:2468] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 0.
I20260709 13:23:03.968937 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bc474b5f64410d9a9c21058a2feca0" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a" is_pre_election: true
I20260709 13:23:03.968714 23810 raft_consensus.cc:359] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.969465 23810 raft_consensus.cc:385] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:03.969663 23618 raft_consensus.cc:2468] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 0.
I20260709 13:23:03.969800 23810 raft_consensus.cc:740] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Initialized, Role: FOLLOWER
I20260709 13:23:03.964007 23768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f909688025304faa9849596796e56988" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd" is_pre_election: true
I20260709 13:23:03.970582 23810 consensus_queue.cc:260] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.971060 23768 raft_consensus.cc:2468] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 0.
I20260709 13:23:03.971879 23693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84909b8ded8d40518c017313ef8bfeb8" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f" is_pre_election: true
I20260709 13:23:03.972613 23693 raft_consensus.cc:2468] T 84909b8ded8d40518c017313ef8bfeb8 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 0.
I20260709 13:23:03.972420 23659 leader_election.cc:304] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:03.973785 23659 leader_election.cc:304] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f; no voters: 11398324f1664f29be659b7034c8eedd
I20260709 13:23:03.974083 23816 raft_consensus.cc:2804] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:03.974597 23816 raft_consensus.cc:493] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:03.975024 23816 raft_consensus.cc:3060] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:03.977118 23815 raft_consensus.cc:515] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.978004 23694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f909688025304faa9849596796e56988" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f" is_pre_election: true
I20260709 13:23:03.973811 23692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bc474b5f64410d9a9c21058a2feca0" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f" is_pre_election: true
I20260709 13:23:03.979293 23692 raft_consensus.cc:2468] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 0.
W20260709 13:23:03.984313 23565 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:23:03.991611 23582 leader_election.cc:304] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:03.993360 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "109ce4dfd3d44642bc560d6cb563ceef" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:03.992705 23815 leader_election.cc:290] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121)
I20260709 13:23:03.995043 23815 raft_consensus.cc:2804] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:03.995404 23815 raft_consensus.cc:493] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:03.995723 23815 raft_consensus.cc:3060] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:23:03.995733 23733 leader_election.cc:343] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:03.994696 23816 raft_consensus.cc:515] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:03.998728 23694 raft_consensus.cc:2393] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 025b14df93f44ae39aefc3c0573cc19a in current term 1: Already voted for candidate b403848a6de0483c82db9cd37d1d5a2f in this term.
W20260709 13:23:03.999179 23731 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.66:36121, user_credentials={real_user=slave}} blocked reactor thread for 51833us
I20260709 13:23:04.015826 23816 leader_election.cc:290] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:04.016574 23816 raft_consensus.cc:2804] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:04.016952 23816 raft_consensus.cc:493] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:04.017282 23816 raft_consensus.cc:3060] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:04.021144 23809 tablet_bootstrap.cc:492] T 109ce4dfd3d44642bc560d6cb563ceef P 025b14df93f44ae39aefc3c0573cc19a: No bootstrap required, opened a new log
I20260709 13:23:04.020998 23814 raft_consensus.cc:515] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.021745 23809 ts_tablet_manager.cc:1403] T 109ce4dfd3d44642bc560d6cb563ceef P 025b14df93f44ae39aefc3c0573cc19a: Time spent bootstrapping tablet: real 0.069s	user 0.013s	sys 0.005s
W20260709 13:23:04.026916 23583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:38311, user_credentials={real_user=slave}} blocked reactor thread for 64762.5us
I20260709 13:23:04.029191 23768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bc474b5f64410d9a9c21058a2feca0" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd" is_pre_election: true
I20260709 13:23:04.032485 23814 leader_election.cc:290] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 election: Requested vote from peers b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:04.032696 23809 raft_consensus.cc:359] T 109ce4dfd3d44642bc560d6cb563ceef P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.033166 23814 raft_consensus.cc:2804] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:04.033550 23814 raft_consensus.cc:493] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:04.033318 23768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f909688025304faa9849596796e56988" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:04.034003 23694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84909b8ded8d40518c017313ef8bfeb8" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
I20260709 13:23:04.034637 23694 raft_consensus.cc:3060] T 84909b8ded8d40518c017313ef8bfeb8 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:04.034791 23838 raft_consensus.cc:493] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:04.035341 23838 raft_consensus.cc:515] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
W20260709 13:23:04.037365 23659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:44419, user_credentials={real_user=slave}} blocked reactor thread for 63694.8us
I20260709 13:23:04.038367 23692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10e65bc4fc654cd6a302b592b8b3a733" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f" is_pre_election: true
W20260709 13:23:04.039800 23582 leader_election.cc:343] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:04.039649 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f909688025304faa9849596796e56988" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:04.040661 23618 raft_consensus.cc:3060] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:04.041754 23692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "109ce4dfd3d44642bc560d6cb563ceef" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
I20260709 13:23:04.042564 23692 raft_consensus.cc:3060] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:04.045153 23815 raft_consensus.cc:515] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.047927 23768 raft_consensus.cc:2393] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b403848a6de0483c82db9cd37d1d5a2f in current term 1: Already voted for candidate 11398324f1664f29be659b7034c8eedd in this term.
I20260709 13:23:04.048959 23618 raft_consensus.cc:2468] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 1.
I20260709 13:23:04.049721 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f909688025304faa9849596796e56988" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:04.050649 23619 raft_consensus.cc:2393] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 11398324f1664f29be659b7034c8eedd in current term 1: Already voted for candidate b403848a6de0483c82db9cd37d1d5a2f in this term.
I20260709 13:23:04.034221 23814 raft_consensus.cc:3060] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:04.058840 23659 leader_election.cc:304] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f; no voters: 11398324f1664f29be659b7034c8eedd
I20260709 13:23:04.061020 23815 leader_election.cc:290] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121)
I20260709 13:23:04.033466 23809 raft_consensus.cc:385] T 109ce4dfd3d44642bc560d6cb563ceef P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:04.064873 23809 raft_consensus.cc:740] T 109ce4dfd3d44642bc560d6cb563ceef P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Initialized, Role: FOLLOWER
I20260709 13:23:04.065526 23583 leader_election.cc:304] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, 11398324f1664f29be659b7034c8eedd; no voters: b403848a6de0483c82db9cd37d1d5a2f
I20260709 13:23:04.066062 23692 raft_consensus.cc:2468] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 1.
I20260709 13:23:04.065986 23809 consensus_queue.cc:260] T 109ce4dfd3d44642bc560d6cb563ceef P 025b14df93f44ae39aefc3c0573cc19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.066658 23843 raft_consensus.cc:2764] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:23:04.057261 23694 raft_consensus.cc:2468] T 84909b8ded8d40518c017313ef8bfeb8 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 1.
I20260709 13:23:04.062729 23808 tablet_bootstrap.cc:492] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f: No bootstrap required, opened a new log
I20260709 13:23:04.062431 23838 leader_election.cc:290] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:04.067750 23840 raft_consensus.cc:2804] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:04.067941 23731 leader_election.cc:304] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11398324f1664f29be659b7034c8eedd, b403848a6de0483c82db9cd37d1d5a2f; no voters: 025b14df93f44ae39aefc3c0573cc19a
I20260709 13:23:04.070593 23582 leader_election.cc:304] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:04.068011 23816 raft_consensus.cc:515] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.071475 23843 raft_consensus.cc:2804] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:04.071986 23843 raft_consensus.cc:697] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [term 1 LEADER]: Becoming Leader. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Running, Role: LEADER
W20260709 13:23:04.069775 23583 leader_election.cc:343] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:04.073088 23843 consensus_queue.cc:237] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.075963 23767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10e65bc4fc654cd6a302b592b8b3a733" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd" is_pre_election: true
I20260709 13:23:04.080646 23815 raft_consensus.cc:2804] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:04.082764 23768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84909b8ded8d40518c017313ef8bfeb8" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:04.083343 23838 raft_consensus.cc:493] T 109ce4dfd3d44642bc560d6cb563ceef P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:04.083667 23768 raft_consensus.cc:3060] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:04.083956 23838 raft_consensus.cc:515] T 109ce4dfd3d44642bc560d6cb563ceef P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.087497 23838 leader_election.cc:290] T 109ce4dfd3d44642bc560d6cb563ceef P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:04.088379 23694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "109ce4dfd3d44642bc560d6cb563ceef" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f" is_pre_election: true
W20260709 13:23:04.080956 23583 leader_election.cc:343] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:04.089315 23694 raft_consensus.cc:2393] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 025b14df93f44ae39aefc3c0573cc19a in current term 1: Already voted for candidate 11398324f1664f29be659b7034c8eedd in this term.
I20260709 13:23:04.089427 23583 leader_election.cc:304] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a; no voters: 11398324f1664f29be659b7034c8eedd, b403848a6de0483c82db9cd37d1d5a2f
I20260709 13:23:04.090785 23838 raft_consensus.cc:2749] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:04.092061 23767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "109ce4dfd3d44642bc560d6cb563ceef" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd" is_pre_election: true
I20260709 13:23:04.080314 23808 ts_tablet_manager.cc:1403] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f: Time spent bootstrapping tablet: real 0.144s	user 0.022s	sys 0.003s
I20260709 13:23:04.107055 23839 raft_consensus.cc:493] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:04.110116 23766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bc474b5f64410d9a9c21058a2feca0" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:04.110160 23839 raft_consensus.cc:515] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
W20260709 13:23:04.112479 23658 leader_election.cc:343] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:04.113018 23839 leader_election.cc:290] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121)
I20260709 13:23:04.113722 23810 ts_tablet_manager.cc:1434] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd: Time spent starting tablet: real 0.150s	user 0.040s	sys 0.058s
I20260709 13:23:04.108268 23816 leader_election.cc:290] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:04.114674 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c184e66c94b4436a4a2028ec5665be5" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a" is_pre_election: true
I20260709 13:23:04.115103 23810 tablet_bootstrap.cc:492] T 10e65bc4fc654cd6a302b592b8b3a733 P 11398324f1664f29be659b7034c8eedd: Bootstrap starting.
I20260709 13:23:04.101524 23814 raft_consensus.cc:515] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
W20260709 13:23:04.116948 23733 leader_election.cc:343] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:04.118844 23766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bc474b5f64410d9a9c21058a2feca0" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:04.119581 23809 ts_tablet_manager.cc:1434] T 109ce4dfd3d44642bc560d6cb563ceef P 025b14df93f44ae39aefc3c0573cc19a: Time spent starting tablet: real 0.097s	user 0.014s	sys 0.022s
I20260709 13:23:04.119581 23694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f909688025304faa9849596796e56988" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
W20260709 13:23:04.120764 23583 leader_election.cc:343] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:04.120779 23692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bc474b5f64410d9a9c21058a2feca0" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
I20260709 13:23:04.121765 23815 raft_consensus.cc:697] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [term 1 LEADER]: Becoming Leader. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Running, Role: LEADER
I20260709 13:23:04.122789 23692 raft_consensus.cc:2393] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 025b14df93f44ae39aefc3c0573cc19a in current term 1: Already voted for candidate b403848a6de0483c82db9cd37d1d5a2f in this term.
I20260709 13:23:04.121035 23809 tablet_bootstrap.cc:492] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a: Bootstrap starting.
I20260709 13:23:04.122527 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bc474b5f64410d9a9c21058a2feca0" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:04.109400 23808 raft_consensus.cc:359] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.124051 23808 raft_consensus.cc:385] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:04.124306 23619 raft_consensus.cc:2393] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b403848a6de0483c82db9cd37d1d5a2f in current term 1: Already voted for candidate 025b14df93f44ae39aefc3c0573cc19a in this term.
I20260709 13:23:04.124361 23808 raft_consensus.cc:740] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Initialized, Role: FOLLOWER
I20260709 13:23:04.125137 23582 leader_election.cc:304] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a; no voters: 11398324f1664f29be659b7034c8eedd, b403848a6de0483c82db9cd37d1d5a2f
I20260709 13:23:04.125379 23808 consensus_queue.cc:260] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.126274 23838 raft_consensus.cc:2749] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:23:04.122865 23768 raft_consensus.cc:2468] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 1.
I20260709 13:23:04.117851 23840 raft_consensus.cc:697] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Becoming Leader. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Running, Role: LEADER
I20260709 13:23:04.128978 23693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c184e66c94b4436a4a2028ec5665be5" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f" is_pre_election: true
I20260709 13:23:04.129803 23840 consensus_queue.cc:237] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.130752 23815 consensus_queue.cc:237] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.128978 23659 leader_election.cc:304] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b403848a6de0483c82db9cd37d1d5a2f; no voters: 025b14df93f44ae39aefc3c0573cc19a, 11398324f1664f29be659b7034c8eedd
I20260709 13:23:04.121989 23814 leader_election.cc:290] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 election: Requested vote from peers b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:04.133247 23845 raft_consensus.cc:2749] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:23:04.132793 23808 ts_tablet_manager.cc:1434] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.004s
W20260709 13:23:04.133792 23731 leader_election.cc:343] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:04.134400 23731 leader_election.cc:304] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11398324f1664f29be659b7034c8eedd; no voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f
I20260709 13:23:04.135710 23839 raft_consensus.cc:2749] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:04.137089 23845 raft_consensus.cc:493] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:04.137964 23810 tablet_bootstrap.cc:654] T 10e65bc4fc654cd6a302b592b8b3a733 P 11398324f1664f29be659b7034c8eedd: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:04.137794 23845 raft_consensus.cc:515] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.138144 23514 catalog_manager.cc:5697] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a reported cstate change: term changed from 0 to 1, leader changed from <none> to 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65). New cstate: current_term: 1 leader_uuid: "025b14df93f44ae39aefc3c0573cc19a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:04.142897 23809 tablet_bootstrap.cc:654] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:04.143905 23845 leader_election.cc:290] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:04.143682 23768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524509732f8487ba5f6ad44e5811ecb" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd" is_pre_election: true
W20260709 13:23:04.146050 23658 leader_election.cc:343] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:04.148746 23731 leader_election.cc:304] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11398324f1664f29be659b7034c8eedd; no voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f
I20260709 13:23:04.150007 23839 raft_consensus.cc:2749] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:23:04.144583 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524509732f8487ba5f6ad44e5811ecb" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a" is_pre_election: true
I20260709 13:23:04.154132 23619 raft_consensus.cc:2468] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 0.
I20260709 13:23:04.157493 23583 leader_election.cc:304] T 109ce4dfd3d44642bc560d6cb563ceef P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a; no voters: 11398324f1664f29be659b7034c8eedd, b403848a6de0483c82db9cd37d1d5a2f
I20260709 13:23:04.158070 23517 catalog_manager.cc:5697] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f reported cstate change: term changed from 0 to 1, leader changed from <none> to b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66). New cstate: current_term: 1 leader_uuid: "b403848a6de0483c82db9cd37d1d5a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:04.159186 23814 raft_consensus.cc:3060] T 109ce4dfd3d44642bc560d6cb563ceef P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:04.159615 23516 catalog_manager.cc:5697] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd reported cstate change: term changed from 0 to 1, leader changed from <none> to 11398324f1664f29be659b7034c8eedd (127.11.63.67). New cstate: current_term: 1 leader_uuid: "11398324f1664f29be659b7034c8eedd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:04.157457 23659 leader_election.cc:304] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f; no voters: 11398324f1664f29be659b7034c8eedd
I20260709 13:23:04.150913 23808 tablet_bootstrap.cc:492] T f7cf9d97b16441ca84a93f2e71f0af81 P b403848a6de0483c82db9cd37d1d5a2f: Bootstrap starting.
I20260709 13:23:04.163851 23840 raft_consensus.cc:2804] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:04.164273 23840 raft_consensus.cc:493] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:04.164628 23840 raft_consensus.cc:3060] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:04.167739 23814 raft_consensus.cc:2749] T 109ce4dfd3d44642bc560d6cb563ceef P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:04.167892 23809 tablet_bootstrap.cc:492] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a: No bootstrap required, opened a new log
I20260709 13:23:04.169183 23809 ts_tablet_manager.cc:1403] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a: Time spent bootstrapping tablet: real 0.048s	user 0.013s	sys 0.002s
I20260709 13:23:04.175562 23840 raft_consensus.cc:515] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.179054 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524509732f8487ba5f6ad44e5811ecb" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:04.179618 23840 leader_election.cc:290] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:04.179911 23768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524509732f8487ba5f6ad44e5811ecb" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd"
W20260709 13:23:04.181751 23658 leader_election.cc:343] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:04.181481 23809 raft_consensus.cc:359] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.182803 23809 raft_consensus.cc:385] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:04.183280 23809 raft_consensus.cc:740] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Initialized, Role: FOLLOWER
I20260709 13:23:04.184391 23809 consensus_queue.cc:260] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.187345 23809 ts_tablet_manager.cc:1434] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:23:04.188253 23810 tablet_bootstrap.cc:492] T 10e65bc4fc654cd6a302b592b8b3a733 P 11398324f1664f29be659b7034c8eedd: No bootstrap required, opened a new log
I20260709 13:23:04.188504 23809 tablet_bootstrap.cc:492] T f7cf9d97b16441ca84a93f2e71f0af81 P 025b14df93f44ae39aefc3c0573cc19a: Bootstrap starting.
I20260709 13:23:04.189400 23810 ts_tablet_manager.cc:1403] T 10e65bc4fc654cd6a302b592b8b3a733 P 11398324f1664f29be659b7034c8eedd: Time spent bootstrapping tablet: real 0.075s	user 0.020s	sys 0.020s
I20260709 13:23:04.190574 23814 raft_consensus.cc:493] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:04.191107 23814 raft_consensus.cc:515] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.193238 23814 leader_election.cc:290] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:04.179677 23619 raft_consensus.cc:3060] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:04.194633 23694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c184e66c94b4436a4a2028ec5665be5" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f" is_pre_election: true
I20260709 13:23:04.195333 23810 raft_consensus.cc:359] T 10e65bc4fc654cd6a302b592b8b3a733 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.196198 23810 raft_consensus.cc:385] T 10e65bc4fc654cd6a302b592b8b3a733 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:04.196549 23810 raft_consensus.cc:740] T 10e65bc4fc654cd6a302b592b8b3a733 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Initialized, Role: FOLLOWER
I20260709 13:23:04.196216 23768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c184e66c94b4436a4a2028ec5665be5" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd" is_pre_election: true
I20260709 13:23:04.197553 23768 raft_consensus.cc:2468] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 0.
I20260709 13:23:04.199203 23583 leader_election.cc:304] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, 11398324f1664f29be659b7034c8eedd; no voters: 
W20260709 13:23:04.200475 23582 leader_election.cc:343] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:04.201634 23814 raft_consensus.cc:2804] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:04.202173 23814 raft_consensus.cc:493] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:04.202618 23814 raft_consensus.cc:3060] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:04.205082 23619 raft_consensus.cc:2468] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 1.
I20260709 13:23:04.206883 23659 leader_election.cc:304] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f; no voters: 11398324f1664f29be659b7034c8eedd
I20260709 13:23:04.208035 23840 raft_consensus.cc:2804] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:04.208597 23840 raft_consensus.cc:697] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Becoming Leader. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Running, Role: LEADER
I20260709 13:23:04.209498 23840 consensus_queue.cc:237] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.211917 23814 raft_consensus.cc:515] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.179651 23808 tablet_bootstrap.cc:654] T f7cf9d97b16441ca84a93f2e71f0af81 P b403848a6de0483c82db9cd37d1d5a2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:04.222574 23768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c184e66c94b4436a4a2028ec5665be5" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:04.223507 23768 raft_consensus.cc:3060] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:04.224022 23814 leader_election.cc:290] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 election: Requested vote from peers b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:04.279307 23768 raft_consensus.cc:2468] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 1.
I20260709 13:23:04.281272 23583 leader_election.cc:304] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, 11398324f1664f29be659b7034c8eedd; no voters: 
I20260709 13:23:04.282647 23814 raft_consensus.cc:2804] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:04.283284 23814 raft_consensus.cc:697] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [term 1 LEADER]: Becoming Leader. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Running, Role: LEADER
I20260709 13:23:04.284360 23814 consensus_queue.cc:237] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.341746 23810 consensus_queue.cc:260] T 10e65bc4fc654cd6a302b592b8b3a733 P 11398324f1664f29be659b7034c8eedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
W20260709 13:23:04.339319 23659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:44419, user_credentials={real_user=slave}} blocked reactor thread for 132570us
I20260709 13:23:04.345058 23694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c184e66c94b4436a4a2028ec5665be5" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
I20260709 13:23:04.345996 23809 tablet_bootstrap.cc:654] T f7cf9d97b16441ca84a93f2e71f0af81 P 025b14df93f44ae39aefc3c0573cc19a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:04.346067 23810 ts_tablet_manager.cc:1434] T 10e65bc4fc654cd6a302b592b8b3a733 P 11398324f1664f29be659b7034c8eedd: Time spent starting tablet: real 0.155s	user 0.047s	sys 0.074s
I20260709 13:23:04.347347 23810 tablet_bootstrap.cc:492] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd: Bootstrap starting.
W20260709 13:23:04.347704 23582 leader_election.cc:343] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:04.356945 23810 tablet_bootstrap.cc:654] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:04.358783 23809 tablet_bootstrap.cc:492] T f7cf9d97b16441ca84a93f2e71f0af81 P 025b14df93f44ae39aefc3c0573cc19a: No bootstrap required, opened a new log
I20260709 13:23:04.359450 23809 ts_tablet_manager.cc:1403] T f7cf9d97b16441ca84a93f2e71f0af81 P 025b14df93f44ae39aefc3c0573cc19a: Time spent bootstrapping tablet: real 0.171s	user 0.009s	sys 0.010s
I20260709 13:23:04.362932 23809 raft_consensus.cc:359] T f7cf9d97b16441ca84a93f2e71f0af81 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.363660 23809 raft_consensus.cc:385] T f7cf9d97b16441ca84a93f2e71f0af81 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:04.363924 23809 raft_consensus.cc:740] T f7cf9d97b16441ca84a93f2e71f0af81 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Initialized, Role: FOLLOWER
I20260709 13:23:04.364615 23809 consensus_queue.cc:260] T f7cf9d97b16441ca84a93f2e71f0af81 P 025b14df93f44ae39aefc3c0573cc19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.366859 23809 ts_tablet_manager.cc:1434] T f7cf9d97b16441ca84a93f2e71f0af81 P 025b14df93f44ae39aefc3c0573cc19a: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:23:04.367792 23809 tablet_bootstrap.cc:492] T 8531f6c11726411eab62bfa778de8701 P 025b14df93f44ae39aefc3c0573cc19a: Bootstrap starting.
I20260709 13:23:04.369436 23808 tablet_bootstrap.cc:492] T f7cf9d97b16441ca84a93f2e71f0af81 P b403848a6de0483c82db9cd37d1d5a2f: No bootstrap required, opened a new log
I20260709 13:23:04.370033 23808 ts_tablet_manager.cc:1403] T f7cf9d97b16441ca84a93f2e71f0af81 P b403848a6de0483c82db9cd37d1d5a2f: Time spent bootstrapping tablet: real 0.225s	user 0.018s	sys 0.011s
I20260709 13:23:04.374276 23809 tablet_bootstrap.cc:654] T 8531f6c11726411eab62bfa778de8701 P 025b14df93f44ae39aefc3c0573cc19a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:04.373804 23808 raft_consensus.cc:359] T f7cf9d97b16441ca84a93f2e71f0af81 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.374902 23808 raft_consensus.cc:385] T f7cf9d97b16441ca84a93f2e71f0af81 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:04.375309 23808 raft_consensus.cc:740] T f7cf9d97b16441ca84a93f2e71f0af81 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Initialized, Role: FOLLOWER
I20260709 13:23:04.376098 23808 consensus_queue.cc:260] T f7cf9d97b16441ca84a93f2e71f0af81 P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.377859 23514 catalog_manager.cc:5697] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a reported cstate change: term changed from 0 to 1, leader changed from <none> to 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65). New cstate: current_term: 1 leader_uuid: "025b14df93f44ae39aefc3c0573cc19a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:04.378893 23808 ts_tablet_manager.cc:1434] T f7cf9d97b16441ca84a93f2e71f0af81 P b403848a6de0483c82db9cd37d1d5a2f: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 13:23:04.380302 23808 tablet_bootstrap.cc:492] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f: Bootstrap starting.
I20260709 13:23:04.382885 23515 catalog_manager.cc:5697] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f reported cstate change: term changed from 0 to 1, leader changed from <none> to b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66). New cstate: current_term: 1 leader_uuid: "b403848a6de0483c82db9cd37d1d5a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:04.385756 23810 tablet_bootstrap.cc:492] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd: No bootstrap required, opened a new log
I20260709 13:23:04.387079 23810 ts_tablet_manager.cc:1403] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd: Time spent bootstrapping tablet: real 0.040s	user 0.011s	sys 0.001s
I20260709 13:23:04.390692 23809 tablet_bootstrap.cc:492] T 8531f6c11726411eab62bfa778de8701 P 025b14df93f44ae39aefc3c0573cc19a: No bootstrap required, opened a new log
I20260709 13:23:04.391227 23809 ts_tablet_manager.cc:1403] T 8531f6c11726411eab62bfa778de8701 P 025b14df93f44ae39aefc3c0573cc19a: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.004s
I20260709 13:23:04.392326 23808 tablet_bootstrap.cc:654] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:04.394948 23809 raft_consensus.cc:359] T 8531f6c11726411eab62bfa778de8701 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.395706 23809 raft_consensus.cc:385] T 8531f6c11726411eab62bfa778de8701 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:04.395321 23810 raft_consensus.cc:359] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.396018 23809 raft_consensus.cc:740] T 8531f6c11726411eab62bfa778de8701 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Initialized, Role: FOLLOWER
I20260709 13:23:04.396358 23810 raft_consensus.cc:385] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:04.396853 23810 raft_consensus.cc:740] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Initialized, Role: FOLLOWER
I20260709 13:23:04.396776 23809 consensus_queue.cc:260] T 8531f6c11726411eab62bfa778de8701 P 025b14df93f44ae39aefc3c0573cc19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.398293 23810 consensus_queue.cc:260] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.399133 23809 ts_tablet_manager.cc:1434] T 8531f6c11726411eab62bfa778de8701 P 025b14df93f44ae39aefc3c0573cc19a: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:23:04.403149 23810 ts_tablet_manager.cc:1434] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:23:04.404206 23810 tablet_bootstrap.cc:492] T 91bc474b5f64410d9a9c21058a2feca0 P 11398324f1664f29be659b7034c8eedd: Bootstrap starting.
I20260709 13:23:04.406951 23808 tablet_bootstrap.cc:492] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f: No bootstrap required, opened a new log
I20260709 13:23:04.407606 23808 ts_tablet_manager.cc:1403] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f: Time spent bootstrapping tablet: real 0.028s	user 0.009s	sys 0.009s
I20260709 13:23:04.410337 23808 raft_consensus.cc:359] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.410771 23810 tablet_bootstrap.cc:654] T 91bc474b5f64410d9a9c21058a2feca0 P 11398324f1664f29be659b7034c8eedd: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:04.411064 23808 raft_consensus.cc:385] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:04.411372 23808 raft_consensus.cc:740] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Initialized, Role: FOLLOWER
I20260709 13:23:04.412209 23808 consensus_queue.cc:260] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.414248 23808 ts_tablet_manager.cc:1434] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:23:04.415138 23808 tablet_bootstrap.cc:492] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f: Bootstrap starting.
I20260709 13:23:04.420607 23808 tablet_bootstrap.cc:654] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:04.420655 23810 tablet_bootstrap.cc:492] T 91bc474b5f64410d9a9c21058a2feca0 P 11398324f1664f29be659b7034c8eedd: No bootstrap required, opened a new log
I20260709 13:23:04.421232 23810 ts_tablet_manager.cc:1403] T 91bc474b5f64410d9a9c21058a2feca0 P 11398324f1664f29be659b7034c8eedd: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.001s
I20260709 13:23:04.423609 23810 raft_consensus.cc:359] T 91bc474b5f64410d9a9c21058a2feca0 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.424113 23810 raft_consensus.cc:385] T 91bc474b5f64410d9a9c21058a2feca0 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:04.424388 23810 raft_consensus.cc:740] T 91bc474b5f64410d9a9c21058a2feca0 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Initialized, Role: FOLLOWER
I20260709 13:23:04.425194 23810 consensus_queue.cc:260] T 91bc474b5f64410d9a9c21058a2feca0 P 11398324f1664f29be659b7034c8eedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.426517 23808 tablet_bootstrap.cc:492] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f: No bootstrap required, opened a new log
I20260709 13:23:04.427017 23808 ts_tablet_manager.cc:1403] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f: Time spent bootstrapping tablet: real 0.012s	user 0.004s	sys 0.006s
I20260709 13:23:04.429121 23810 ts_tablet_manager.cc:1434] T 91bc474b5f64410d9a9c21058a2feca0 P 11398324f1664f29be659b7034c8eedd: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:23:04.429134 23808 raft_consensus.cc:359] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.430032 23808 raft_consensus.cc:385] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:04.430097 23810 tablet_bootstrap.cc:492] T d524509732f8487ba5f6ad44e5811ecb P 11398324f1664f29be659b7034c8eedd: Bootstrap starting.
I20260709 13:23:04.430365 23808 raft_consensus.cc:740] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Initialized, Role: FOLLOWER
I20260709 13:23:04.431111 23808 consensus_queue.cc:260] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.433233 23808 ts_tablet_manager.cc:1434] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:23:04.435835 23810 tablet_bootstrap.cc:654] T d524509732f8487ba5f6ad44e5811ecb P 11398324f1664f29be659b7034c8eedd: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:04.442772 23810 tablet_bootstrap.cc:492] T d524509732f8487ba5f6ad44e5811ecb P 11398324f1664f29be659b7034c8eedd: No bootstrap required, opened a new log
I20260709 13:23:04.443176 23810 ts_tablet_manager.cc:1403] T d524509732f8487ba5f6ad44e5811ecb P 11398324f1664f29be659b7034c8eedd: Time spent bootstrapping tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 13:23:04.445111 23810 raft_consensus.cc:359] T d524509732f8487ba5f6ad44e5811ecb P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.445621 23810 raft_consensus.cc:385] T d524509732f8487ba5f6ad44e5811ecb P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:04.445844 23810 raft_consensus.cc:740] T d524509732f8487ba5f6ad44e5811ecb P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Initialized, Role: FOLLOWER
I20260709 13:23:04.446411 23810 consensus_queue.cc:260] T d524509732f8487ba5f6ad44e5811ecb P 11398324f1664f29be659b7034c8eedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.448225 23810 ts_tablet_manager.cc:1434] T d524509732f8487ba5f6ad44e5811ecb P 11398324f1664f29be659b7034c8eedd: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.004s
I20260709 13:23:04.477741 23814 consensus_queue.cc:1048] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:04.478691 23840 raft_consensus.cc:493] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:04.479218 23840 raft_consensus.cc:515] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.481328 23840 leader_election.cc:290] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:04.481849 23815 raft_consensus.cc:493] T d524509732f8487ba5f6ad44e5811ecb P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:04.482383 23815 raft_consensus.cc:515] T d524509732f8487ba5f6ad44e5811ecb P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.482546 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10e65bc4fc654cd6a302b592b8b3a733" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a" is_pre_election: true
I20260709 13:23:04.482620 23768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10e65bc4fc654cd6a302b592b8b3a733" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd" is_pre_election: true
I20260709 13:23:04.483333 23619 raft_consensus.cc:2468] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 0.
I20260709 13:23:04.483547 23768 raft_consensus.cc:2468] T 10e65bc4fc654cd6a302b592b8b3a733 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 0.
I20260709 13:23:04.484508 23815 leader_election.cc:290] T d524509732f8487ba5f6ad44e5811ecb P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121)
I20260709 13:23:04.484673 23659 leader_election.cc:304] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:04.486334 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524509732f8487ba5f6ad44e5811ecb" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a" is_pre_election: true
I20260709 13:23:04.486966 23693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524509732f8487ba5f6ad44e5811ecb" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f" is_pre_election: true
I20260709 13:23:04.487085 23815 raft_consensus.cc:493] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:04.487468 23619 raft_consensus.cc:2393] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11398324f1664f29be659b7034c8eedd in current term 1: Already voted for candidate b403848a6de0483c82db9cd37d1d5a2f in this term.
I20260709 13:23:04.487660 23815 raft_consensus.cc:515] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.489310 23733 leader_election.cc:304] T d524509732f8487ba5f6ad44e5811ecb P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11398324f1664f29be659b7034c8eedd; no voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f
I20260709 13:23:04.490394 23839 raft_consensus.cc:3060] T d524509732f8487ba5f6ad44e5811ecb P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:04.490762 23693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8531f6c11726411eab62bfa778de8701" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f" is_pre_election: true
I20260709 13:23:04.491421 23693 raft_consensus.cc:2468] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 0.
I20260709 13:23:04.492498 23731 leader_election.cc:304] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11398324f1664f29be659b7034c8eedd, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:04.492499 23815 leader_election.cc:290] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121)
I20260709 13:23:04.493785 23815 raft_consensus.cc:2804] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:04.494129 23815 raft_consensus.cc:493] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:04.486645 23840 raft_consensus.cc:2804] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:04.494469 23815 raft_consensus.cc:3060] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:04.494583 23840 raft_consensus.cc:493] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:04.494972 23840 raft_consensus.cc:3060] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:04.501727 23837 raft_consensus.cc:493] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:04.501837 23840 raft_consensus.cc:515] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.501997 23815 raft_consensus.cc:515] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.501971 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8531f6c11726411eab62bfa778de8701" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a" is_pre_election: true
I20260709 13:23:04.503602 23619 raft_consensus.cc:2468] T 8531f6c11726411eab62bfa778de8701 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 0.
I20260709 13:23:04.502408 23837 raft_consensus.cc:515] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.504915 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8531f6c11726411eab62bfa778de8701" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:04.505102 23840 leader_election.cc:290] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:04.505491 23618 raft_consensus.cc:3060] T 8531f6c11726411eab62bfa778de8701 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:04.506184 23815 leader_election.cc:290] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121)
I20260709 13:23:04.506650 23693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7cf9d97b16441ca84a93f2e71f0af81" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f" is_pre_election: true
I20260709 13:23:04.506700 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7cf9d97b16441ca84a93f2e71f0af81" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a" is_pre_election: true
I20260709 13:23:04.507402 23837 leader_election.cc:290] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121)
I20260709 13:23:04.507589 23617 raft_consensus.cc:2468] T f7cf9d97b16441ca84a93f2e71f0af81 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 0.
I20260709 13:23:04.508548 23692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8531f6c11726411eab62bfa778de8701" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
I20260709 13:23:04.509155 23692 raft_consensus.cc:3060] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:04.509317 23733 leader_election.cc:304] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, 11398324f1664f29be659b7034c8eedd; no voters: 
I20260709 13:23:04.510264 23837 raft_consensus.cc:2804] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:04.510623 23837 raft_consensus.cc:493] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:04.510888 23768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10e65bc4fc654cd6a302b592b8b3a733" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:04.510939 23837 raft_consensus.cc:3060] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:04.511633 23768 raft_consensus.cc:3060] T 10e65bc4fc654cd6a302b592b8b3a733 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:04.507586 23693 raft_consensus.cc:2468] T f7cf9d97b16441ca84a93f2e71f0af81 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 0.
I20260709 13:23:04.516789 23618 raft_consensus.cc:2468] T 8531f6c11726411eab62bfa778de8701 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 1.
I20260709 13:23:04.522714 23815 consensus_queue.cc:1048] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [LEADER]: Connected to new peer: Peer: permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:04.517601 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10e65bc4fc654cd6a302b592b8b3a733" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:04.523797 23619 raft_consensus.cc:3060] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:04.520216 23692 raft_consensus.cc:2468] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 1.
I20260709 13:23:04.526667 23731 leader_election.cc:304] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11398324f1664f29be659b7034c8eedd, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:04.527767 23815 raft_consensus.cc:2804] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:04.528240 23815 raft_consensus.cc:697] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [term 1 LEADER]: Becoming Leader. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Running, Role: LEADER
I20260709 13:23:04.529089 23815 consensus_queue.cc:237] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.530107 23814 consensus_queue.cc:1048] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Connected to new peer: Peer: permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:04.522189 23839 raft_consensus.cc:2749] T d524509732f8487ba5f6ad44e5811ecb P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:04.543671 23840 consensus_queue.cc:1048] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:04.560139 23837 raft_consensus.cc:515] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.563048 23837 leader_election.cc:290] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121)
I20260709 13:23:04.564566 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7cf9d97b16441ca84a93f2e71f0af81" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:04.565493 23618 raft_consensus.cc:3060] T f7cf9d97b16441ca84a93f2e71f0af81 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:04.566061 23692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7cf9d97b16441ca84a93f2e71f0af81" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
I20260709 13:23:04.569303 23692 raft_consensus.cc:3060] T f7cf9d97b16441ca84a93f2e71f0af81 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:04.560045 23768 raft_consensus.cc:2468] T 10e65bc4fc654cd6a302b592b8b3a733 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 1.
I20260709 13:23:04.573098 23658 leader_election.cc:304] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11398324f1664f29be659b7034c8eedd, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:04.574381 23840 raft_consensus.cc:2804] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:04.576503 23840 raft_consensus.cc:697] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Becoming Leader. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Running, Role: LEADER
I20260709 13:23:04.577768 23840 consensus_queue.cc:237] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.584227 23816 consensus_queue.cc:1048] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:04.585176 23618 raft_consensus.cc:2468] T f7cf9d97b16441ca84a93f2e71f0af81 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 1.
I20260709 13:23:04.584826 23517 catalog_manager.cc:5697] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd reported cstate change: term changed from 0 to 1, leader changed from <none> to 11398324f1664f29be659b7034c8eedd (127.11.63.67). New cstate: current_term: 1 leader_uuid: "11398324f1664f29be659b7034c8eedd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:04.586874 23733 leader_election.cc:304] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, 11398324f1664f29be659b7034c8eedd; no voters: 
I20260709 13:23:04.588086 23815 raft_consensus.cc:2804] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:04.588703 23815 raft_consensus.cc:697] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd [term 1 LEADER]: Becoming Leader. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Running, Role: LEADER
I20260709 13:23:04.589838 23815 consensus_queue.cc:237] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.592216 23816 raft_consensus.cc:493] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:04.592832 23816 raft_consensus.cc:515] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.595105 23816 leader_election.cc:290] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:04.597306 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c184e66c94b4436a4a2028ec5665be5" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a" is_pre_election: true
I20260709 13:23:04.598320 23767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c184e66c94b4436a4a2028ec5665be5" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd" is_pre_election: true
I20260709 13:23:04.605285 23767 raft_consensus.cc:2393] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b403848a6de0483c82db9cd37d1d5a2f in current term 1: Already voted for candidate 025b14df93f44ae39aefc3c0573cc19a in this term.
I20260709 13:23:04.609141 23658 leader_election.cc:304] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b403848a6de0483c82db9cd37d1d5a2f; no voters: 025b14df93f44ae39aefc3c0573cc19a, 11398324f1664f29be659b7034c8eedd
I20260709 13:23:04.611793 23816 raft_consensus.cc:3060] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:04.627542 23619 raft_consensus.cc:2468] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 1.
I20260709 13:23:04.634439 23816 raft_consensus.cc:2749] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:04.639328 23516 catalog_manager.cc:5697] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd reported cstate change: term changed from 0 to 1, leader changed from <none> to 11398324f1664f29be659b7034c8eedd (127.11.63.67). New cstate: current_term: 1 leader_uuid: "11398324f1664f29be659b7034c8eedd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:04.641891 23692 raft_consensus.cc:2468] T f7cf9d97b16441ca84a93f2e71f0af81 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 1.
I20260709 13:23:04.646600 23517 catalog_manager.cc:5697] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f reported cstate change: term changed from 0 to 1, leader changed from <none> to b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66). New cstate: current_term: 1 leader_uuid: "b403848a6de0483c82db9cd37d1d5a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:04.647190 23839 consensus_queue.cc:1048] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:04.760742 23814 consensus_queue.cc:1048] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:04.772100 23838 consensus_queue.cc:1048] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Connected to new peer: Peer: permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:04.917228 23839 raft_consensus.cc:493] T 91bc474b5f64410d9a9c21058a2feca0 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:04.917768 23839 raft_consensus.cc:515] T 91bc474b5f64410d9a9c21058a2feca0 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:04.919389 23839 leader_election.cc:290] T 91bc474b5f64410d9a9c21058a2feca0 P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121), 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419)
I20260709 13:23:04.920471 23692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bc474b5f64410d9a9c21058a2feca0" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f" is_pre_election: true
I20260709 13:23:04.920615 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bc474b5f64410d9a9c21058a2feca0" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a" is_pre_election: true
I20260709 13:23:04.921192 23692 raft_consensus.cc:2393] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11398324f1664f29be659b7034c8eedd in current term 1: Already voted for candidate b403848a6de0483c82db9cd37d1d5a2f in this term.
I20260709 13:23:04.921422 23618 raft_consensus.cc:2393] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 11398324f1664f29be659b7034c8eedd in current term 1: Already voted for candidate 025b14df93f44ae39aefc3c0573cc19a in this term.
I20260709 13:23:04.922596 23733 leader_election.cc:304] T 91bc474b5f64410d9a9c21058a2feca0 P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11398324f1664f29be659b7034c8eedd; no voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f
I20260709 13:23:04.923249 23839 raft_consensus.cc:3060] T 91bc474b5f64410d9a9c21058a2feca0 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:04.928478 23839 raft_consensus.cc:2749] T 91bc474b5f64410d9a9c21058a2feca0 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:04.964766 23840 consensus_queue.cc:1048] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:04.968092 23840 consensus_queue.cc:1048] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:23:04.986482 23565 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:23:04.988026 23839 consensus_queue.cc:1048] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:05.000556 23872 consensus_queue.cc:1048] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd [LEADER]: Connected to new peer: Peer: permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:05.010231 23840 consensus_queue.cc:1048] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:05.026690 23840 consensus_queue.cc:1048] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:05.116771 23868 consensus_queue.cc:1048] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:05.128649 23873 consensus_queue.cc:1048] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [LEADER]: Connected to new peer: Peer: permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:23:05.988235 23565 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:23:05.997721 23843 raft_consensus.cc:493] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:05.998523 23843 raft_consensus.cc:515] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:06.000355 23843 leader_election.cc:290] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:06.001246 23692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bc474b5f64410d9a9c21058a2feca0" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f" is_pre_election: true
I20260709 13:23:06.001642 23766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bc474b5f64410d9a9c21058a2feca0" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd" is_pre_election: true
I20260709 13:23:06.001966 23692 raft_consensus.cc:2468] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 1.
I20260709 13:23:06.002293 23766 raft_consensus.cc:2468] T 91bc474b5f64410d9a9c21058a2feca0 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 1.
I20260709 13:23:06.003142 23582 leader_election.cc:304] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:06.003950 23843 raft_consensus.cc:2804] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:06.004313 23843 raft_consensus.cc:493] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:06.004657 23843 raft_consensus.cc:3060] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:06.009853 23843 raft_consensus.cc:515] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:06.011552 23843 leader_election.cc:290] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 2 election: Requested vote from peers b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:06.012594 23768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bc474b5f64410d9a9c21058a2feca0" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:06.012638 23692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91bc474b5f64410d9a9c21058a2feca0" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
I20260709 13:23:06.013243 23768 raft_consensus.cc:3060] T 91bc474b5f64410d9a9c21058a2feca0 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:06.013352 23692 raft_consensus.cc:3060] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:06.019237 23768 raft_consensus.cc:2468] T 91bc474b5f64410d9a9c21058a2feca0 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 2.
I20260709 13:23:06.019624 23692 raft_consensus.cc:2468] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 2.
I20260709 13:23:06.020434 23583 leader_election.cc:304] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, 11398324f1664f29be659b7034c8eedd; no voters: 
I20260709 13:23:06.021299 23843 raft_consensus.cc:2804] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:06.022159 23843 raft_consensus.cc:697] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [term 2 LEADER]: Becoming Leader. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Running, Role: LEADER
I20260709 13:23:06.023003 23843 consensus_queue.cc:237] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:06.030237 23516 catalog_manager.cc:5697] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a reported cstate change: term changed from 0 to 2, leader changed from <none> to 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65). New cstate: current_term: 2 leader_uuid: "025b14df93f44ae39aefc3c0573cc19a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:06.138216 23692 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10e65bc4fc654cd6a302b592b8b3a733"
dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
mode: GRACEFUL
new_leader_uuid: "025b14df93f44ae39aefc3c0573cc19a"
 from {username='slave'} at 127.0.0.1:38346
I20260709 13:23:06.138752 23692 raft_consensus.cc:606] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Received request to transfer leadership to TS 025b14df93f44ae39aefc3c0573cc19a
I20260709 13:23:06.144200 23692 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d524509732f8487ba5f6ad44e5811ecb"
dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
mode: GRACEFUL
new_leader_uuid: "025b14df93f44ae39aefc3c0573cc19a"
 from {username='slave'} at 127.0.0.1:38346
I20260709 13:23:06.144694 23692 raft_consensus.cc:606] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Received request to transfer leadership to TS 025b14df93f44ae39aefc3c0573cc19a
I20260709 13:23:06.162032 23618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c184e66c94b4436a4a2028ec5665be5"
dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
mode: GRACEFUL
new_leader_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
 from {username='slave'} at 127.0.0.1:49382
I20260709 13:23:06.162675 23618 raft_consensus.cc:606] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [term 1 LEADER]: Received request to transfer leadership to TS b403848a6de0483c82db9cd37d1d5a2f
I20260709 13:23:06.176034 23768 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "109ce4dfd3d44642bc560d6cb563ceef"
dest_uuid: "11398324f1664f29be659b7034c8eedd"
mode: GRACEFUL
new_leader_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
 from {username='slave'} at 127.0.0.1:38654
I20260709 13:23:06.176816 23768 raft_consensus.cc:606] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [term 1 LEADER]: Received request to transfer leadership to TS b403848a6de0483c82db9cd37d1d5a2f
I20260709 13:23:06.182742 23768 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8531f6c11726411eab62bfa778de8701"
dest_uuid: "11398324f1664f29be659b7034c8eedd"
mode: GRACEFUL
new_leader_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
 from {username='slave'} at 127.0.0.1:38654
I20260709 13:23:06.183442 23768 raft_consensus.cc:606] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [term 1 LEADER]: Received request to transfer leadership to TS b403848a6de0483c82db9cd37d1d5a2f
I20260709 13:23:06.458912 23903 raft_consensus.cc:993] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f: : Instructing follower 025b14df93f44ae39aefc3c0573cc19a to start an election
I20260709 13:23:06.459466 23903 raft_consensus.cc:1081] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Signalling peer 025b14df93f44ae39aefc3c0573cc19a to start an election
I20260709 13:23:06.461026 23618 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d524509732f8487ba5f6ad44e5811ecb"
dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
 from {username='slave'} at 127.0.0.1:49366
I20260709 13:23:06.461603 23618 raft_consensus.cc:493] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:06.461896 23618 raft_consensus.cc:3060] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:06.469707 23618 raft_consensus.cc:515] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:06.471992 23903 raft_consensus.cc:993] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f: : Instructing follower 025b14df93f44ae39aefc3c0573cc19a to start an election
I20260709 13:23:06.472656 23903 raft_consensus.cc:1081] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Signalling peer 025b14df93f44ae39aefc3c0573cc19a to start an election
I20260709 13:23:06.475636 23618 leader_election.cc:290] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 2 election: Requested vote from peers b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:06.475800 23619 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "10e65bc4fc654cd6a302b592b8b3a733"
dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
 from {username='slave'} at 127.0.0.1:49366
I20260709 13:23:06.476742 23619 raft_consensus.cc:493] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:06.477257 23619 raft_consensus.cc:3060] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:06.488592 23619 raft_consensus.cc:515] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:06.499871 23692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524509732f8487ba5f6ad44e5811ecb" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
I20260709 13:23:06.500203 23619 leader_election.cc:290] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 2 election: Requested vote from peers b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:06.500626 23692 raft_consensus.cc:3055] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:06.500989 23692 raft_consensus.cc:740] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Running, Role: LEADER
I20260709 13:23:06.501832 23692 consensus_queue.cc:260] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:06.502964 23692 raft_consensus.cc:3060] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:06.503167 23843 raft_consensus.cc:993] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a: : Instructing follower b403848a6de0483c82db9cd37d1d5a2f to start an election
I20260709 13:23:06.503710 23843 raft_consensus.cc:1081] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [term 1 LEADER]: Signalling peer b403848a6de0483c82db9cd37d1d5a2f to start an election
I20260709 13:23:06.505708 23693 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4c184e66c94b4436a4a2028ec5665be5"
dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
 from {username='slave'} at 127.0.0.1:38330
I20260709 13:23:06.505934 23694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10e65bc4fc654cd6a302b592b8b3a733" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
I20260709 13:23:06.506276 23693 raft_consensus.cc:493] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:06.506719 23693 raft_consensus.cc:3060] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:06.506548 23694 raft_consensus.cc:3055] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:06.508352 23694 raft_consensus.cc:740] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Running, Role: LEADER
I20260709 13:23:06.508579 23767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10e65bc4fc654cd6a302b592b8b3a733" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:06.509255 23694 consensus_queue.cc:260] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:06.510584 23694 raft_consensus.cc:3060] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:06.513700 23693 raft_consensus.cc:515] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:06.490908 23768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d524509732f8487ba5f6ad44e5811ecb" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:06.514824 23768 raft_consensus.cc:3060] T d524509732f8487ba5f6ad44e5811ecb P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:06.516870 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c184e66c94b4436a4a2028ec5665be5" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:06.517316 23766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c184e66c94b4436a4a2028ec5665be5" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:06.517654 23619 raft_consensus.cc:3055] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:06.518043 23766 raft_consensus.cc:3060] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:06.518775 23694 raft_consensus.cc:2468] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 2.
I20260709 13:23:06.519279 23692 raft_consensus.cc:2468] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 2.
I20260709 13:23:06.518023 23619 raft_consensus.cc:740] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Running, Role: LEADER
I20260709 13:23:06.520716 23582 leader_election.cc:304] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f; no voters: 11398324f1664f29be659b7034c8eedd
I20260709 13:23:06.520711 23619 consensus_queue.cc:260] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:06.522111 23582 leader_election.cc:304] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:06.522607 23619 raft_consensus.cc:3060] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:06.522724 23893 raft_consensus.cc:2804] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:06.520895 23693 leader_election.cc:290] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 2 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:06.523589 23843 raft_consensus.cc:2804] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:06.525177 23893 raft_consensus.cc:697] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [term 2 LEADER]: Becoming Leader. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Running, Role: LEADER
I20260709 13:23:06.526208 23893 consensus_queue.cc:237] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:06.527809 23766 raft_consensus.cc:2468] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 2.
I20260709 13:23:06.523826 23768 raft_consensus.cc:2468] T d524509732f8487ba5f6ad44e5811ecb P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 2.
I20260709 13:23:06.530684 23658 leader_election.cc:304] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11398324f1664f29be659b7034c8eedd, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:06.531571 23903 raft_consensus.cc:2804] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:06.532152 23903 raft_consensus.cc:697] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f [term 2 LEADER]: Becoming Leader. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Running, Role: LEADER
I20260709 13:23:06.531445 23619 raft_consensus.cc:2468] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 2.
I20260709 13:23:06.533036 23903 consensus_queue.cc:237] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:06.539392 23843 raft_consensus.cc:697] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a [term 2 LEADER]: Becoming Leader. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Running, Role: LEADER
I20260709 13:23:06.540783 23843 consensus_queue.cc:237] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:06.546025 23906 raft_consensus.cc:993] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd: : Instructing follower b403848a6de0483c82db9cd37d1d5a2f to start an election
I20260709 13:23:06.546648 23906 raft_consensus.cc:1081] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [term 1 LEADER]: Signalling peer b403848a6de0483c82db9cd37d1d5a2f to start an election
I20260709 13:23:06.548629 23693 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "109ce4dfd3d44642bc560d6cb563ceef"
dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
 from {username='slave'} at 127.0.0.1:38316
I20260709 13:23:06.549264 23693 raft_consensus.cc:493] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:06.549759 23693 raft_consensus.cc:3060] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:06.550104 23516 catalog_manager.cc:5697] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f reported cstate change: term changed from 1 to 2, leader changed from 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65) to b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66). New cstate: current_term: 2 leader_uuid: "b403848a6de0483c82db9cd37d1d5a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:06.555550 23693 raft_consensus.cc:515] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:06.559108 23768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "109ce4dfd3d44642bc560d6cb563ceef" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:06.559783 23768 raft_consensus.cc:3055] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:06.560140 23768 raft_consensus.cc:740] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Running, Role: LEADER
I20260709 13:23:06.560233 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "109ce4dfd3d44642bc560d6cb563ceef" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:06.560909 23619 raft_consensus.cc:3060] T 109ce4dfd3d44642bc560d6cb563ceef P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:06.561034 23768 consensus_queue.cc:260] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:06.562309 23768 raft_consensus.cc:3060] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:06.564828 23515 catalog_manager.cc:5697] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a reported cstate change: term changed from 1 to 2, leader changed from b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66) to 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65). New cstate: current_term: 2 leader_uuid: "025b14df93f44ae39aefc3c0573cc19a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:06.569149 23619 raft_consensus.cc:2468] T 109ce4dfd3d44642bc560d6cb563ceef P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 2.
I20260709 13:23:06.569914 23768 raft_consensus.cc:2468] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 2.
I20260709 13:23:06.570120 23693 leader_election.cc:290] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 2 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:06.570948 23659 leader_election.cc:304] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:06.572283 23903 raft_consensus.cc:2804] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:06.572834 23903 raft_consensus.cc:697] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f [term 2 LEADER]: Becoming Leader. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Running, Role: LEADER
I20260709 13:23:06.571812 23515 catalog_manager.cc:5697] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a reported cstate change: term changed from 1 to 2, leader changed from b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66) to 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65). New cstate: current_term: 2 leader_uuid: "025b14df93f44ae39aefc3c0573cc19a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:06.573725 23903 consensus_queue.cc:237] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:06.585440 23516 catalog_manager.cc:5697] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f reported cstate change: term changed from 1 to 2, leader changed from 11398324f1664f29be659b7034c8eedd (127.11.63.67) to b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66). New cstate: current_term: 2 leader_uuid: "b403848a6de0483c82db9cd37d1d5a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:06.588891 23693 raft_consensus.cc:1275] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Refusing update from remote peer 025b14df93f44ae39aefc3c0573cc19a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:23:06.590468 23843 consensus_queue.cc:1048] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:06.601152 23768 raft_consensus.cc:1275] T 91bc474b5f64410d9a9c21058a2feca0 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Refusing update from remote peer 025b14df93f44ae39aefc3c0573cc19a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:23:06.602584 23843 consensus_queue.cc:1048] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Connected to new peer: Peer: permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:06.713220 23906 raft_consensus.cc:993] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd: : Instructing follower b403848a6de0483c82db9cd37d1d5a2f to start an election
I20260709 13:23:06.713816 23911 raft_consensus.cc:1081] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [term 1 LEADER]: Signalling peer b403848a6de0483c82db9cd37d1d5a2f to start an election
I20260709 13:23:06.715358 23693 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8531f6c11726411eab62bfa778de8701"
dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
 from {username='slave'} at 127.0.0.1:38316
I20260709 13:23:06.715915 23693 raft_consensus.cc:493] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:06.716204 23693 raft_consensus.cc:3060] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:06.721125 23693 raft_consensus.cc:515] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:06.722756 23693 leader_election.cc:290] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 2 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:06.723788 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8531f6c11726411eab62bfa778de8701" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:06.723845 23768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8531f6c11726411eab62bfa778de8701" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:06.724503 23619 raft_consensus.cc:3060] T 8531f6c11726411eab62bfa778de8701 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:06.724632 23768 raft_consensus.cc:3055] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:06.725046 23768 raft_consensus.cc:740] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Running, Role: LEADER
I20260709 13:23:06.725777 23768 consensus_queue.cc:260] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:06.726689 23768 raft_consensus.cc:3060] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:06.730072 23619 raft_consensus.cc:2468] T 8531f6c11726411eab62bfa778de8701 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 2.
I20260709 13:23:06.731269 23659 leader_election.cc:304] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:06.731590 23768 raft_consensus.cc:2468] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 2.
I20260709 13:23:06.732059 23903 raft_consensus.cc:2804] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:06.732630 23903 raft_consensus.cc:697] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f [term 2 LEADER]: Becoming Leader. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Running, Role: LEADER
I20260709 13:23:06.733363 23903 consensus_queue.cc:237] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:06.740701 23515 catalog_manager.cc:5697] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f reported cstate change: term changed from 1 to 2, leader changed from 11398324f1664f29be659b7034c8eedd (127.11.63.67) to b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66). New cstate: current_term: 2 leader_uuid: "b403848a6de0483c82db9cd37d1d5a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:06.935844 23768 raft_consensus.cc:1275] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Refusing update from remote peer b403848a6de0483c82db9cd37d1d5a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:06.937266 23903 consensus_queue.cc:1048] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:06.945896 23619 raft_consensus.cc:1275] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Refusing update from remote peer b403848a6de0483c82db9cd37d1d5a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:06.947755 23908 consensus_queue.cc:1048] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:06.969722 23768 raft_consensus.cc:1275] T d524509732f8487ba5f6ad44e5811ecb P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Refusing update from remote peer 025b14df93f44ae39aefc3c0573cc19a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:06.970937 23843 consensus_queue.cc:1048] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Connected to new peer: Peer: permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:06.979985 23693 raft_consensus.cc:1275] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Refusing update from remote peer 025b14df93f44ae39aefc3c0573cc19a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:06.981680 23905 consensus_queue.cc:1048] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:07.008518 23619 raft_consensus.cc:1275] T 109ce4dfd3d44642bc560d6cb563ceef P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Refusing update from remote peer b403848a6de0483c82db9cd37d1d5a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:07.010023 23908 consensus_queue.cc:1048] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:07.019176 23768 raft_consensus.cc:1275] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Refusing update from remote peer b403848a6de0483c82db9cd37d1d5a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:07.020900 23903 consensus_queue.cc:1048] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:07.058751 23768 raft_consensus.cc:3060] T 10e65bc4fc654cd6a302b592b8b3a733 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:07.064623 23768 raft_consensus.cc:1275] T 10e65bc4fc654cd6a302b592b8b3a733 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Refusing update from remote peer 025b14df93f44ae39aefc3c0573cc19a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:07.066052 23905 consensus_queue.cc:1048] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Connected to new peer: Peer: permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:07.075678 23693 raft_consensus.cc:1275] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Refusing update from remote peer 025b14df93f44ae39aefc3c0573cc19a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:07.077543 23843 consensus_queue.cc:1048] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:07.248572 23619 raft_consensus.cc:1275] T 8531f6c11726411eab62bfa778de8701 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Refusing update from remote peer b403848a6de0483c82db9cd37d1d5a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:07.250156 23903 consensus_queue.cc:1048] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:07.259289 23766 raft_consensus.cc:1275] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Refusing update from remote peer b403848a6de0483c82db9cd37d1d5a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:07.260938 23931 consensus_queue.cc:1048] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:23:07.442462 23789 debug-util.cc:398] Leaking SignalData structure 0x7b080014eb60 after lost signal to thread 11520
W20260709 13:23:07.443571 23789 debug-util.cc:398] Leaking SignalData structure 0x7b08001747a0 after lost signal to thread 23551
W20260709 13:23:07.444907 23789 debug-util.cc:398] Leaking SignalData structure 0x7b0800097160 after lost signal to thread 23643
W20260709 13:23:07.446556 23789 debug-util.cc:398] Leaking SignalData structure 0x7b0800248080 after lost signal to thread 23718
W20260709 13:23:07.447655 23789 debug-util.cc:398] Leaking SignalData structure 0x7b08001ffee0 after lost signal to thread 23792
I20260709 13:23:09.318742 23516 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35640:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:23:09.321053 23516 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:23:09.375110 23602 tablet_service.cc:1511] Processing CreateTablet for tablet b78deb2eabd845dcae6bb7158b2aafff (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:23:09.376523 23602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b78deb2eabd845dcae6bb7158b2aafff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.378836 23755 tablet_service.cc:1511] Processing CreateTablet for tablet b78deb2eabd845dcae6bb7158b2aafff (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:23:09.380167 23755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b78deb2eabd845dcae6bb7158b2aafff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.380836 23679 tablet_service.cc:1511] Processing CreateTablet for tablet b78deb2eabd845dcae6bb7158b2aafff (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:23:09.382162 23679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b78deb2eabd845dcae6bb7158b2aafff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.382328 23758 tablet_service.cc:1511] Processing CreateTablet for tablet b2912e06fc9944968c191d9e3366d314 (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:23:09.383610 23758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2912e06fc9944968c191d9e3366d314. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.384912 23681 tablet_service.cc:1511] Processing CreateTablet for tablet b2912e06fc9944968c191d9e3366d314 (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:23:09.386245 23681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2912e06fc9944968c191d9e3366d314. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.386124 23604 tablet_service.cc:1511] Processing CreateTablet for tablet 5ea06411c11b43d6b60a50bb271ae151 (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:23:09.386541 23601 tablet_service.cc:1511] Processing CreateTablet for tablet b2912e06fc9944968c191d9e3366d314 (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:23:09.387418 23604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ea06411c11b43d6b60a50bb271ae151. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.387998 23601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2912e06fc9944968c191d9e3366d314. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.389737 23750 tablet_service.cc:1511] Processing CreateTablet for tablet 5ea06411c11b43d6b60a50bb271ae151 (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:23:09.390472 23678 tablet_service.cc:1511] Processing CreateTablet for tablet 5ea06411c11b43d6b60a50bb271ae151 (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:23:09.391002 23750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ea06411c11b43d6b60a50bb271ae151. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.391726 23678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ea06411c11b43d6b60a50bb271ae151. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.390472 23684 tablet_service.cc:1511] Processing CreateTablet for tablet 2edafaf1fec84770b4792ec38c697f19 (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:23:09.395059 23684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2edafaf1fec84770b4792ec38c697f19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.398278 23756 tablet_service.cc:1511] Processing CreateTablet for tablet 2edafaf1fec84770b4792ec38c697f19 (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:23:09.399556 23605 tablet_service.cc:1511] Processing CreateTablet for tablet 5a2818a134994e4c8cfc92ce9523005a (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:23:09.401023 23605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a2818a134994e4c8cfc92ce9523005a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.400841 23682 tablet_service.cc:1511] Processing CreateTablet for tablet 5a2818a134994e4c8cfc92ce9523005a (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:23:09.402112 23682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a2818a134994e4c8cfc92ce9523005a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.406193 23603 tablet_service.cc:1511] Processing CreateTablet for tablet 2edafaf1fec84770b4792ec38c697f19 (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:23:09.407552 23603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2edafaf1fec84770b4792ec38c697f19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.399628 23756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2edafaf1fec84770b4792ec38c697f19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.434103 23751 tablet_service.cc:1511] Processing CreateTablet for tablet 5a2818a134994e4c8cfc92ce9523005a (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:23:09.435652 23751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a2818a134994e4c8cfc92ce9523005a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.455515 23609 tablet_service.cc:1511] Processing CreateTablet for tablet 32aca5c2e490419a9b679c38609e37e3 (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:23:09.456933 23609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32aca5c2e490419a9b679c38609e37e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.469300 23680 tablet_service.cc:1511] Processing CreateTablet for tablet 32aca5c2e490419a9b679c38609e37e3 (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:23:09.470806 23680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32aca5c2e490419a9b679c38609e37e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.474824 23754 tablet_service.cc:1511] Processing CreateTablet for tablet 32aca5c2e490419a9b679c38609e37e3 (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:23:09.476323 23754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32aca5c2e490419a9b679c38609e37e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.484745 23601 tablet_service.cc:1511] Processing CreateTablet for tablet 0c9ba0a8c02b403ab321dc3bc2a5affb (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:23:09.486827 23601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c9ba0a8c02b403ab321dc3bc2a5affb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.507869 23752 tablet_service.cc:1511] Processing CreateTablet for tablet 0c9ba0a8c02b403ab321dc3bc2a5affb (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:23:09.509284 23752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c9ba0a8c02b403ab321dc3bc2a5affb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.509466 23946 tablet_bootstrap.cc:492] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a: Bootstrap starting.
I20260709 13:23:09.510093 23947 tablet_bootstrap.cc:492] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f: Bootstrap starting.
I20260709 13:23:09.516652 23947 tablet_bootstrap.cc:654] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:09.523809 23946 tablet_bootstrap.cc:654] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:09.526656 23679 tablet_service.cc:1511] Processing CreateTablet for tablet 0c9ba0a8c02b403ab321dc3bc2a5affb (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:23:09.528067 23679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c9ba0a8c02b403ab321dc3bc2a5affb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.529452 23603 tablet_service.cc:1511] Processing CreateTablet for tablet c97da58ded5f4813b217185ce853e15e (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:23:09.530889 23603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c97da58ded5f4813b217185ce853e15e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.534183 23684 tablet_service.cc:1511] Processing CreateTablet for tablet c97da58ded5f4813b217185ce853e15e (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:23:09.535524 23684 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c97da58ded5f4813b217185ce853e15e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.540884 23755 tablet_service.cc:1511] Processing CreateTablet for tablet c97da58ded5f4813b217185ce853e15e (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:23:09.548303 23601 tablet_service.cc:1511] Processing CreateTablet for tablet 54ccbd59a1d04accaacf5f20864df594 (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:23:09.550918 23680 tablet_service.cc:1511] Processing CreateTablet for tablet 54ccbd59a1d04accaacf5f20864df594 (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:23:09.548838 23755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c97da58ded5f4813b217185ce853e15e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.557603 23758 tablet_service.cc:1511] Processing CreateTablet for tablet 54ccbd59a1d04accaacf5f20864df594 (DEFAULT_TABLE table=soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:23:09.552403 23680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54ccbd59a1d04accaacf5f20864df594. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.551642 23601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54ccbd59a1d04accaacf5f20864df594. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.562573 23758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54ccbd59a1d04accaacf5f20864df594. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:09.580646 23948 tablet_bootstrap.cc:492] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd: Bootstrap starting.
I20260709 13:23:09.582213 23947 tablet_bootstrap.cc:492] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f: No bootstrap required, opened a new log
I20260709 13:23:09.586661 23947 ts_tablet_manager.cc:1403] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f: Time spent bootstrapping tablet: real 0.077s	user 0.009s	sys 0.011s
I20260709 13:23:09.589915 23947 raft_consensus.cc:359] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.590674 23947 raft_consensus.cc:385] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:09.590996 23947 raft_consensus.cc:740] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Initialized, Role: FOLLOWER
I20260709 13:23:09.591784 23947 consensus_queue.cc:260] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.598268 23947 ts_tablet_manager.cc:1434] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:23:09.606272 23947 tablet_bootstrap.cc:492] T 5ea06411c11b43d6b60a50bb271ae151 P b403848a6de0483c82db9cd37d1d5a2f: Bootstrap starting.
I20260709 13:23:09.611470 23947 tablet_bootstrap.cc:654] T 5ea06411c11b43d6b60a50bb271ae151 P b403848a6de0483c82db9cd37d1d5a2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:09.617964 23946 tablet_bootstrap.cc:492] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a: No bootstrap required, opened a new log
I20260709 13:23:09.621539 23946 ts_tablet_manager.cc:1403] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a: Time spent bootstrapping tablet: real 0.112s	user 0.004s	sys 0.019s
I20260709 13:23:09.620885 23948 tablet_bootstrap.cc:654] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:09.631357 23946 raft_consensus.cc:359] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.632056 23946 raft_consensus.cc:385] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:09.632339 23946 raft_consensus.cc:740] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Initialized, Role: FOLLOWER
I20260709 13:23:09.633020 23946 consensus_queue.cc:260] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.635277 23946 ts_tablet_manager.cc:1434] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a: Time spent starting tablet: real 0.013s	user 0.002s	sys 0.003s
I20260709 13:23:09.636910 23946 tablet_bootstrap.cc:492] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a: Bootstrap starting.
I20260709 13:23:09.640944 23947 tablet_bootstrap.cc:492] T 5ea06411c11b43d6b60a50bb271ae151 P b403848a6de0483c82db9cd37d1d5a2f: No bootstrap required, opened a new log
I20260709 13:23:09.641484 23947 ts_tablet_manager.cc:1403] T 5ea06411c11b43d6b60a50bb271ae151 P b403848a6de0483c82db9cd37d1d5a2f: Time spent bootstrapping tablet: real 0.035s	user 0.025s	sys 0.006s
I20260709 13:23:09.648936 23948 tablet_bootstrap.cc:492] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd: No bootstrap required, opened a new log
I20260709 13:23:09.649574 23948 ts_tablet_manager.cc:1403] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd: Time spent bootstrapping tablet: real 0.069s	user 0.009s	sys 0.005s
I20260709 13:23:09.651503 23946 tablet_bootstrap.cc:654] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:09.652796 23948 raft_consensus.cc:359] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.653651 23948 raft_consensus.cc:385] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:09.653952 23948 raft_consensus.cc:740] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Initialized, Role: FOLLOWER
I20260709 13:23:09.653757 23947 raft_consensus.cc:359] T 5ea06411c11b43d6b60a50bb271ae151 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.654925 23947 raft_consensus.cc:385] T 5ea06411c11b43d6b60a50bb271ae151 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:09.655294 23948 consensus_queue.cc:260] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.656282 23933 raft_consensus.cc:493] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:09.655309 23947 raft_consensus.cc:740] T 5ea06411c11b43d6b60a50bb271ae151 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Initialized, Role: FOLLOWER
I20260709 13:23:09.656903 23933 raft_consensus.cc:515] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.658948 23948 ts_tablet_manager.cc:1434] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:23:09.659104 23933 leader_election.cc:290] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:09.659732 23947 consensus_queue.cc:260] T 5ea06411c11b43d6b60a50bb271ae151 P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.662185 23947 ts_tablet_manager.cc:1434] T 5ea06411c11b43d6b60a50bb271ae151 P b403848a6de0483c82db9cd37d1d5a2f: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
I20260709 13:23:09.662549 23693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ea06411c11b43d6b60a50bb271ae151" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f" is_pre_election: true
I20260709 13:23:09.663194 23947 tablet_bootstrap.cc:492] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f: Bootstrap starting.
I20260709 13:23:09.663143 23693 raft_consensus.cc:2468] T 5ea06411c11b43d6b60a50bb271ae151 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 0.
I20260709 13:23:09.660385 23948 tablet_bootstrap.cc:492] T 5ea06411c11b43d6b60a50bb271ae151 P 11398324f1664f29be659b7034c8eedd: Bootstrap starting.
I20260709 13:23:09.664310 23766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ea06411c11b43d6b60a50bb271ae151" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd" is_pre_election: true
I20260709 13:23:09.670780 23947 tablet_bootstrap.cc:654] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:09.672276 23948 tablet_bootstrap.cc:654] T 5ea06411c11b43d6b60a50bb271ae151 P 11398324f1664f29be659b7034c8eedd: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:09.675544 23582 leader_election.cc:304] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
W20260709 13:23:09.682219 23583 leader_election.cc:343] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:09.684751 23948 tablet_bootstrap.cc:492] T 5ea06411c11b43d6b60a50bb271ae151 P 11398324f1664f29be659b7034c8eedd: No bootstrap required, opened a new log
I20260709 13:23:09.685464 23948 ts_tablet_manager.cc:1403] T 5ea06411c11b43d6b60a50bb271ae151 P 11398324f1664f29be659b7034c8eedd: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.008s
I20260709 13:23:09.686156 23933 raft_consensus.cc:2804] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:09.686599 23933 raft_consensus.cc:493] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:09.686923 23933 raft_consensus.cc:3060] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:09.688941 23948 raft_consensus.cc:359] T 5ea06411c11b43d6b60a50bb271ae151 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.689973 23948 raft_consensus.cc:385] T 5ea06411c11b43d6b60a50bb271ae151 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:09.690349 23948 raft_consensus.cc:740] T 5ea06411c11b43d6b60a50bb271ae151 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Initialized, Role: FOLLOWER
I20260709 13:23:09.691352 23948 consensus_queue.cc:260] T 5ea06411c11b43d6b60a50bb271ae151 P 11398324f1664f29be659b7034c8eedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.694145 23948 ts_tablet_manager.cc:1434] T 5ea06411c11b43d6b60a50bb271ae151 P 11398324f1664f29be659b7034c8eedd: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:23:09.700661 23948 tablet_bootstrap.cc:492] T 2edafaf1fec84770b4792ec38c697f19 P 11398324f1664f29be659b7034c8eedd: Bootstrap starting.
I20260709 13:23:09.707767 23948 tablet_bootstrap.cc:654] T 2edafaf1fec84770b4792ec38c697f19 P 11398324f1664f29be659b7034c8eedd: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:09.723985 23946 tablet_bootstrap.cc:492] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a: No bootstrap required, opened a new log
I20260709 13:23:09.724738 23946 ts_tablet_manager.cc:1403] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a: Time spent bootstrapping tablet: real 0.088s	user 0.004s	sys 0.008s
I20260709 13:23:09.728472 23948 tablet_bootstrap.cc:492] T 2edafaf1fec84770b4792ec38c697f19 P 11398324f1664f29be659b7034c8eedd: No bootstrap required, opened a new log
I20260709 13:23:09.729207 23948 ts_tablet_manager.cc:1403] T 2edafaf1fec84770b4792ec38c697f19 P 11398324f1664f29be659b7034c8eedd: Time spent bootstrapping tablet: real 0.029s	user 0.008s	sys 0.011s
I20260709 13:23:09.733551 23933 raft_consensus.cc:515] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.738066 23948 raft_consensus.cc:359] T 2edafaf1fec84770b4792ec38c697f19 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.736824 23946 raft_consensus.cc:359] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.741034 23948 raft_consensus.cc:385] T 2edafaf1fec84770b4792ec38c697f19 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:09.741050 23946 raft_consensus.cc:385] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:09.741503 23948 raft_consensus.cc:740] T 2edafaf1fec84770b4792ec38c697f19 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Initialized, Role: FOLLOWER
I20260709 13:23:09.742501 23948 consensus_queue.cc:260] T 2edafaf1fec84770b4792ec38c697f19 P 11398324f1664f29be659b7034c8eedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.743067 23693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ea06411c11b43d6b60a50bb271ae151" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
I20260709 13:23:09.743396 23766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ea06411c11b43d6b60a50bb271ae151" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:09.743688 23693 raft_consensus.cc:3060] T 5ea06411c11b43d6b60a50bb271ae151 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:09.744071 23766 raft_consensus.cc:3060] T 5ea06411c11b43d6b60a50bb271ae151 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:09.744971 23948 ts_tablet_manager.cc:1434] T 2edafaf1fec84770b4792ec38c697f19 P 11398324f1664f29be659b7034c8eedd: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:23:09.746116 23948 tablet_bootstrap.cc:492] T b2912e06fc9944968c191d9e3366d314 P 11398324f1664f29be659b7034c8eedd: Bootstrap starting.
I20260709 13:23:09.752104 23911 raft_consensus.cc:493] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:09.752238 23766 raft_consensus.cc:2468] T 5ea06411c11b43d6b60a50bb271ae151 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 1.
I20260709 13:23:09.752861 23948 tablet_bootstrap.cc:654] T b2912e06fc9944968c191d9e3366d314 P 11398324f1664f29be659b7034c8eedd: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:09.752712 23911 raft_consensus.cc:515] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.753772 23583 leader_election.cc:304] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, 11398324f1664f29be659b7034c8eedd; no voters: 
I20260709 13:23:09.755213 23956 raft_consensus.cc:2804] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:09.757014 23692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a2818a134994e4c8cfc92ce9523005a" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f" is_pre_election: true
I20260709 13:23:09.758574 23911 leader_election.cc:290] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121)
W20260709 13:23:09.760895 23731 leader_election.cc:343] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:09.766074 23956 raft_consensus.cc:697] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a [term 1 LEADER]: Becoming Leader. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Running, Role: LEADER
I20260709 13:23:09.767046 23956 consensus_queue.cc:237] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.741765 23946 raft_consensus.cc:740] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Initialized, Role: FOLLOWER
I20260709 13:23:09.769284 23946 consensus_queue.cc:260] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.772768 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a2818a134994e4c8cfc92ce9523005a" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a" is_pre_election: true
W20260709 13:23:09.774845 23733 leader_election.cc:343] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:09.775342 23733 leader_election.cc:304] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 11398324f1664f29be659b7034c8eedd; no voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f
I20260709 13:23:09.776458 23933 leader_election.cc:290] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 election: Requested vote from peers b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:09.780038 23947 tablet_bootstrap.cc:492] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f: No bootstrap required, opened a new log
I20260709 13:23:09.780665 23947 ts_tablet_manager.cc:1403] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f: Time spent bootstrapping tablet: real 0.118s	user 0.022s	sys 0.038s
I20260709 13:23:09.784688 23911 raft_consensus.cc:2749] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:09.783974 23516 catalog_manager.cc:5697] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a reported cstate change: term changed from 0 to 1, leader changed from <none> to 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65). New cstate: current_term: 1 leader_uuid: "025b14df93f44ae39aefc3c0573cc19a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:09.783967 23947 raft_consensus.cc:359] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.790747 23947 raft_consensus.cc:385] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:09.791149 23947 raft_consensus.cc:740] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Initialized, Role: FOLLOWER
I20260709 13:23:09.794101 23693 raft_consensus.cc:2468] T 5ea06411c11b43d6b60a50bb271ae151 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 1.
I20260709 13:23:09.788682 23946 ts_tablet_manager.cc:1434] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a: Time spent starting tablet: real 0.064s	user 0.004s	sys 0.006s
I20260709 13:23:09.797451 23946 tablet_bootstrap.cc:492] T 32aca5c2e490419a9b679c38609e37e3 P 025b14df93f44ae39aefc3c0573cc19a: Bootstrap starting.
I20260709 13:23:09.791992 23947 consensus_queue.cc:260] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.803755 23947 ts_tablet_manager.cc:1434] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f: Time spent starting tablet: real 0.023s	user 0.001s	sys 0.005s
I20260709 13:23:09.815722 23947 tablet_bootstrap.cc:492] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f: Bootstrap starting.
I20260709 13:23:09.821698 23946 tablet_bootstrap.cc:654] T 32aca5c2e490419a9b679c38609e37e3 P 025b14df93f44ae39aefc3c0573cc19a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:09.821801 23948 tablet_bootstrap.cc:492] T b2912e06fc9944968c191d9e3366d314 P 11398324f1664f29be659b7034c8eedd: No bootstrap required, opened a new log
I20260709 13:23:09.824904 23948 ts_tablet_manager.cc:1403] T b2912e06fc9944968c191d9e3366d314 P 11398324f1664f29be659b7034c8eedd: Time spent bootstrapping tablet: real 0.079s	user 0.019s	sys 0.023s
I20260709 13:23:09.828137 23948 raft_consensus.cc:359] T b2912e06fc9944968c191d9e3366d314 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.828858 23948 raft_consensus.cc:385] T b2912e06fc9944968c191d9e3366d314 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:09.829136 23948 raft_consensus.cc:740] T b2912e06fc9944968c191d9e3366d314 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Initialized, Role: FOLLOWER
I20260709 13:23:09.829900 23948 consensus_queue.cc:260] T b2912e06fc9944968c191d9e3366d314 P 11398324f1664f29be659b7034c8eedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.831498 23947 tablet_bootstrap.cc:654] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:09.832206 23948 ts_tablet_manager.cc:1434] T b2912e06fc9944968c191d9e3366d314 P 11398324f1664f29be659b7034c8eedd: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.002s
I20260709 13:23:09.833274 23948 tablet_bootstrap.cc:492] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd: Bootstrap starting.
I20260709 13:23:09.839771 23948 tablet_bootstrap.cc:654] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:09.845202 23935 raft_consensus.cc:493] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:09.845504 23947 tablet_bootstrap.cc:492] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f: No bootstrap required, opened a new log
I20260709 13:23:09.846128 23947 ts_tablet_manager.cc:1403] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.008s
I20260709 13:23:09.845958 23935 raft_consensus.cc:515] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.849222 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2edafaf1fec84770b4792ec38c697f19" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a" is_pre_election: true
I20260709 13:23:09.849383 23948 tablet_bootstrap.cc:492] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd: No bootstrap required, opened a new log
I20260709 13:23:09.849413 23947 raft_consensus.cc:359] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.850086 23948 ts_tablet_manager.cc:1403] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.003s
I20260709 13:23:09.850417 23947 raft_consensus.cc:385] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:09.850754 23947 raft_consensus.cc:740] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Initialized, Role: FOLLOWER
W20260709 13:23:09.850979 23659 leader_election.cc:343] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:09.851553 23947 consensus_queue.cc:260] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.853410 23948 raft_consensus.cc:359] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.854142 23948 raft_consensus.cc:385] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:09.854161 23947 ts_tablet_manager.cc:1434] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:23:09.854549 23948 raft_consensus.cc:740] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Initialized, Role: FOLLOWER
I20260709 13:23:09.855461 23947 tablet_bootstrap.cc:492] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f: Bootstrap starting.
I20260709 13:23:09.855512 23948 consensus_queue.cc:260] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.855471 23766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2edafaf1fec84770b4792ec38c697f19" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd" is_pre_election: true
I20260709 13:23:09.857030 23766 raft_consensus.cc:2468] T 2edafaf1fec84770b4792ec38c697f19 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 0.
I20260709 13:23:09.858121 23935 leader_election.cc:290] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:09.858263 23658 leader_election.cc:304] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11398324f1664f29be659b7034c8eedd, b403848a6de0483c82db9cd37d1d5a2f; no voters: 025b14df93f44ae39aefc3c0573cc19a
I20260709 13:23:09.859319 23948 ts_tablet_manager.cc:1434] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:23:09.860534 23934 raft_consensus.cc:2804] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:09.860894 23934 raft_consensus.cc:493] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:09.861217 23934 raft_consensus.cc:3060] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:09.861299 23948 tablet_bootstrap.cc:492] T 32aca5c2e490419a9b679c38609e37e3 P 11398324f1664f29be659b7034c8eedd: Bootstrap starting.
I20260709 13:23:09.865063 23947 tablet_bootstrap.cc:654] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:09.868474 23948 tablet_bootstrap.cc:654] T 32aca5c2e490419a9b679c38609e37e3 P 11398324f1664f29be659b7034c8eedd: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:09.870463 23934 raft_consensus.cc:515] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.873245 23946 tablet_bootstrap.cc:492] T 32aca5c2e490419a9b679c38609e37e3 P 025b14df93f44ae39aefc3c0573cc19a: No bootstrap required, opened a new log
I20260709 13:23:09.873792 23946 ts_tablet_manager.cc:1403] T 32aca5c2e490419a9b679c38609e37e3 P 025b14df93f44ae39aefc3c0573cc19a: Time spent bootstrapping tablet: real 0.077s	user 0.012s	sys 0.001s
I20260709 13:23:09.874333 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2edafaf1fec84770b4792ec38c697f19" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:09.876812 23946 raft_consensus.cc:359] T 32aca5c2e490419a9b679c38609e37e3 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.877722 23946 raft_consensus.cc:385] T 32aca5c2e490419a9b679c38609e37e3 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:09.878002 23946 raft_consensus.cc:740] T 32aca5c2e490419a9b679c38609e37e3 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Initialized, Role: FOLLOWER
I20260709 13:23:09.878710 23946 consensus_queue.cc:260] T 32aca5c2e490419a9b679c38609e37e3 P 025b14df93f44ae39aefc3c0573cc19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
W20260709 13:23:09.879392 23659 leader_election.cc:343] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:09.881102 23946 ts_tablet_manager.cc:1434] T 32aca5c2e490419a9b679c38609e37e3 P 025b14df93f44ae39aefc3c0573cc19a: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 13:23:09.882475 23946 tablet_bootstrap.cc:492] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a: Bootstrap starting.
I20260709 13:23:09.882941 23947 tablet_bootstrap.cc:492] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f: No bootstrap required, opened a new log
I20260709 13:23:09.883458 23947 ts_tablet_manager.cc:1403] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.006s
I20260709 13:23:09.887007 23947 raft_consensus.cc:359] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.887858 23947 raft_consensus.cc:385] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:09.888271 23947 raft_consensus.cc:740] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Initialized, Role: FOLLOWER
I20260709 13:23:09.889101 23947 consensus_queue.cc:260] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.890167 23946 tablet_bootstrap.cc:654] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:09.891793 23947 ts_tablet_manager.cc:1434] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:23:09.893054 23947 tablet_bootstrap.cc:492] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f: Bootstrap starting.
I20260709 13:23:09.895953 23948 tablet_bootstrap.cc:492] T 32aca5c2e490419a9b679c38609e37e3 P 11398324f1664f29be659b7034c8eedd: No bootstrap required, opened a new log
I20260709 13:23:09.896526 23948 ts_tablet_manager.cc:1403] T 32aca5c2e490419a9b679c38609e37e3 P 11398324f1664f29be659b7034c8eedd: Time spent bootstrapping tablet: real 0.035s	user 0.007s	sys 0.007s
I20260709 13:23:09.899555 23948 raft_consensus.cc:359] T 32aca5c2e490419a9b679c38609e37e3 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.900295 23948 raft_consensus.cc:385] T 32aca5c2e490419a9b679c38609e37e3 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:09.900596 23948 raft_consensus.cc:740] T 32aca5c2e490419a9b679c38609e37e3 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Initialized, Role: FOLLOWER
I20260709 13:23:09.900858 23947 tablet_bootstrap.cc:654] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:09.911928 23948 consensus_queue.cc:260] T 32aca5c2e490419a9b679c38609e37e3 P 11398324f1664f29be659b7034c8eedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.913087 23947 tablet_bootstrap.cc:492] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f: No bootstrap required, opened a new log
I20260709 13:23:09.913707 23947 ts_tablet_manager.cc:1403] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.007s
I20260709 13:23:09.914641 23948 ts_tablet_manager.cc:1434] T 32aca5c2e490419a9b679c38609e37e3 P 11398324f1664f29be659b7034c8eedd: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.003s
I20260709 13:23:09.915712 23948 tablet_bootstrap.cc:492] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd: Bootstrap starting.
I20260709 13:23:09.916972 23947 raft_consensus.cc:359] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.917871 23947 raft_consensus.cc:385] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:09.918291 23947 raft_consensus.cc:740] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Initialized, Role: FOLLOWER
I20260709 13:23:09.918049 23946 tablet_bootstrap.cc:492] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a: No bootstrap required, opened a new log
I20260709 13:23:09.920908 23766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2edafaf1fec84770b4792ec38c697f19" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:09.921260 23946 ts_tablet_manager.cc:1403] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a: Time spent bootstrapping tablet: real 0.039s	user 0.012s	sys 0.002s
I20260709 13:23:09.921593 23766 raft_consensus.cc:3060] T 2edafaf1fec84770b4792ec38c697f19 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:09.923669 23947 consensus_queue.cc:260] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.924335 23946 raft_consensus.cc:359] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.925112 23946 raft_consensus.cc:385] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:09.925390 23946 raft_consensus.cc:740] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Initialized, Role: FOLLOWER
I20260709 13:23:09.926445 23947 ts_tablet_manager.cc:1434] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:23:09.926234 23946 consensus_queue.cc:260] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.926473 23935 raft_consensus.cc:493] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:09.927691 23947 tablet_bootstrap.cc:492] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f: Bootstrap starting.
I20260709 13:23:09.928225 23935 raft_consensus.cc:515] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.939657 23935 leader_election.cc:290] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:09.941236 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32aca5c2e490419a9b679c38609e37e3" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a" is_pre_election: true
I20260709 13:23:09.942415 23619 raft_consensus.cc:2468] T 32aca5c2e490419a9b679c38609e37e3 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 0.
I20260709 13:23:09.928792 23946 ts_tablet_manager.cc:1434] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:23:09.934469 23948 tablet_bootstrap.cc:654] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:09.945525 23946 tablet_bootstrap.cc:492] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a: Bootstrap starting.
I20260709 13:23:09.950452 23934 leader_election.cc:290] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:09.947640 23659 leader_election.cc:304] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:09.952271 23935 raft_consensus.cc:2804] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:09.952831 23935 raft_consensus.cc:493] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:09.953465 23935 raft_consensus.cc:3060] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:09.936609 23766 raft_consensus.cc:2468] T 2edafaf1fec84770b4792ec38c697f19 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 1.
I20260709 13:23:09.935438 23947 tablet_bootstrap.cc:654] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:09.965823 23766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32aca5c2e490419a9b679c38609e37e3" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd" is_pre_election: true
I20260709 13:23:09.966578 23766 raft_consensus.cc:2468] T 32aca5c2e490419a9b679c38609e37e3 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 0.
I20260709 13:23:09.967979 23658 leader_election.cc:304] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11398324f1664f29be659b7034c8eedd, b403848a6de0483c82db9cd37d1d5a2f; no voters: 025b14df93f44ae39aefc3c0573cc19a
I20260709 13:23:09.970296 23969 raft_consensus.cc:2804] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:09.970861 23969 raft_consensus.cc:697] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Becoming Leader. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Running, Role: LEADER
I20260709 13:23:09.972420 23969 consensus_queue.cc:237] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.973918 23948 tablet_bootstrap.cc:492] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd: No bootstrap required, opened a new log
I20260709 13:23:09.974503 23948 ts_tablet_manager.cc:1403] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd: Time spent bootstrapping tablet: real 0.059s	user 0.008s	sys 0.010s
I20260709 13:23:09.976369 23947 tablet_bootstrap.cc:492] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f: No bootstrap required, opened a new log
I20260709 13:23:09.976775 23946 tablet_bootstrap.cc:654] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:09.976949 23947 ts_tablet_manager.cc:1403] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f: Time spent bootstrapping tablet: real 0.049s	user 0.015s	sys 0.002s
I20260709 13:23:09.977337 23934 raft_consensus.cc:493] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:09.977727 23948 raft_consensus.cc:359] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.977885 23934 raft_consensus.cc:515] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.978516 23948 raft_consensus.cc:385] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:09.978856 23948 raft_consensus.cc:740] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Initialized, Role: FOLLOWER
I20260709 13:23:09.981205 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2912e06fc9944968c191d9e3366d314" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a" is_pre_election: true
I20260709 13:23:09.981873 23619 raft_consensus.cc:2468] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 0.
I20260709 13:23:09.982226 23935 raft_consensus.cc:515] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.983280 23659 leader_election.cc:304] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:09.985450 23947 raft_consensus.cc:359] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.986254 23766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32aca5c2e490419a9b679c38609e37e3" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:09.986403 23947 raft_consensus.cc:385] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:09.986934 23947 raft_consensus.cc:740] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Initialized, Role: FOLLOWER
I20260709 13:23:09.986963 23766 raft_consensus.cc:3060] T 32aca5c2e490419a9b679c38609e37e3 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:09.987932 23947 consensus_queue.cc:260] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.979630 23948 consensus_queue.cc:260] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:09.991892 23948 ts_tablet_manager.cc:1434] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:23:09.993227 23948 tablet_bootstrap.cc:492] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd: Bootstrap starting.
I20260709 13:23:09.994429 23766 raft_consensus.cc:2468] T 32aca5c2e490419a9b679c38609e37e3 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 1.
I20260709 13:23:09.995618 23658 leader_election.cc:304] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11398324f1664f29be659b7034c8eedd, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:09.997689 23947 ts_tablet_manager.cc:1434] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f: Time spent starting tablet: real 0.020s	user 0.000s	sys 0.014s
I20260709 13:23:09.998756 23947 tablet_bootstrap.cc:492] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f: Bootstrap starting.
I20260709 13:23:10.000144 23948 tablet_bootstrap.cc:654] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:10.001973 23974 raft_consensus.cc:2804] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:10.002460 23974 raft_consensus.cc:493] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:10.002851 23974 raft_consensus.cc:3060] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:10.004863 23935 leader_election.cc:290] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:10.005858 23935 raft_consensus.cc:2804] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:10.006340 23935 raft_consensus.cc:697] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Becoming Leader. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Running, Role: LEADER
I20260709 13:23:10.009743 23947 tablet_bootstrap.cc:654] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:10.010380 23934 leader_election.cc:290] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:10.007228 23935 consensus_queue.cc:237] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.020352 23974 raft_consensus.cc:515] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.025157 23974 leader_election.cc:290] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:10.025378 23948 tablet_bootstrap.cc:492] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd: No bootstrap required, opened a new log
I20260709 13:23:10.025945 23948 ts_tablet_manager.cc:1403] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.004s
I20260709 13:23:10.027642 23947 tablet_bootstrap.cc:492] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f: No bootstrap required, opened a new log
I20260709 13:23:10.028177 23947 ts_tablet_manager.cc:1403] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f: Time spent bootstrapping tablet: real 0.030s	user 0.007s	sys 0.005s
I20260709 13:23:10.028877 23948 raft_consensus.cc:359] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.029615 23948 raft_consensus.cc:385] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:10.029874 23948 raft_consensus.cc:740] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Initialized, Role: FOLLOWER
I20260709 13:23:10.030895 23947 raft_consensus.cc:359] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.031637 23947 raft_consensus.cc:385] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:10.031893 23947 raft_consensus.cc:740] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Initialized, Role: FOLLOWER
W20260709 13:23:10.031925 23565 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:23:10.032581 23948 consensus_queue.cc:260] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.032581 23947 consensus_queue.cc:260] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.034768 23948 ts_tablet_manager.cc:1434] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:23:10.035009 23947 ts_tablet_manager.cc:1434] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
W20260709 13:23:10.032766 23659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.65:44419, user_credentials={real_user=slave}} blocked reactor thread for 49567.5us
I20260709 13:23:10.035984 23947 tablet_bootstrap.cc:492] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f: Bootstrap starting.
I20260709 13:23:10.037389 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32aca5c2e490419a9b679c38609e37e3" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:10.038177 23619 raft_consensus.cc:3060] T 32aca5c2e490419a9b679c38609e37e3 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:10.043301 23947 tablet_bootstrap.cc:654] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:10.046368 23948 tablet_bootstrap.cc:492] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd: Bootstrap starting.
I20260709 13:23:10.056046 23948 tablet_bootstrap.cc:654] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:10.056269 23947 tablet_bootstrap.cc:492] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f: No bootstrap required, opened a new log
I20260709 13:23:10.056773 23947 ts_tablet_manager.cc:1403] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.004s
I20260709 13:23:10.059684 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2912e06fc9944968c191d9e3366d314" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:10.060658 23618 raft_consensus.cc:3060] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:10.061174 23947 raft_consensus.cc:359] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.061997 23947 raft_consensus.cc:385] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:10.062299 23947 raft_consensus.cc:740] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Initialized, Role: FOLLOWER
I20260709 13:23:10.063119 23947 consensus_queue.cc:260] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.065768 23947 ts_tablet_manager.cc:1434] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:23:10.079308 23515 catalog_manager.cc:5697] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f reported cstate change: term changed from 0 to 1, leader changed from <none> to b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66). New cstate: current_term: 1 leader_uuid: "b403848a6de0483c82db9cd37d1d5a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:10.111611 23618 raft_consensus.cc:2468] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 1.
W20260709 13:23:10.113636 23658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.63.67:38311, user_credentials={real_user=slave}} blocked reactor thread for 118102us
I20260709 13:23:10.115927 23766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2912e06fc9944968c191d9e3366d314" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd" is_pre_election: true
I20260709 13:23:10.116586 23766 raft_consensus.cc:2468] T b2912e06fc9944968c191d9e3366d314 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 0.
I20260709 13:23:10.117672 23659 leader_election.cc:304] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:10.119901 23969 raft_consensus.cc:2804] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:10.120271 23766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2912e06fc9944968c191d9e3366d314" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:10.120644 23969 raft_consensus.cc:697] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Becoming Leader. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Running, Role: LEADER
I20260709 13:23:10.120919 23766 raft_consensus.cc:3060] T b2912e06fc9944968c191d9e3366d314 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:10.121795 23969 consensus_queue.cc:237] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
W20260709 13:23:10.128053 23978 log.cc:927] Time spent T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:23:10.128638 23766 raft_consensus.cc:2468] T b2912e06fc9944968c191d9e3366d314 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 1.
I20260709 13:23:10.130682 23619 raft_consensus.cc:2468] T 32aca5c2e490419a9b679c38609e37e3 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 1.
I20260709 13:23:10.142798 23517 catalog_manager.cc:5697] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f reported cstate change: term changed from 0 to 1, leader changed from <none> to b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66). New cstate: current_term: 1 leader_uuid: "b403848a6de0483c82db9cd37d1d5a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:10.146663 23946 tablet_bootstrap.cc:492] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a: No bootstrap required, opened a new log
I20260709 13:23:10.147258 23946 ts_tablet_manager.cc:1403] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a: Time spent bootstrapping tablet: real 0.202s	user 0.022s	sys 0.026s
I20260709 13:23:10.152797 23946 raft_consensus.cc:359] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.153493 23946 raft_consensus.cc:385] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:10.153853 23946 raft_consensus.cc:740] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Initialized, Role: FOLLOWER
I20260709 13:23:10.153664 23948 tablet_bootstrap.cc:492] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd: No bootstrap required, opened a new log
I20260709 13:23:10.155638 23948 ts_tablet_manager.cc:1403] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd: Time spent bootstrapping tablet: real 0.109s	user 0.040s	sys 0.051s
I20260709 13:23:10.155260 23946 consensus_queue.cc:260] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.157982 23946 ts_tablet_manager.cc:1434] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:23:10.158543 23960 raft_consensus.cc:493] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:10.158987 23960 raft_consensus.cc:515] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.158941 23948 raft_consensus.cc:359] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.161331 23948 raft_consensus.cc:385] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:10.161725 23948 raft_consensus.cc:740] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Initialized, Role: FOLLOWER
I20260709 13:23:10.162612 23948 consensus_queue.cc:260] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.163332 23693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a2818a134994e4c8cfc92ce9523005a" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f" is_pre_election: true
I20260709 13:23:10.164083 23693 raft_consensus.cc:2468] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 0.
I20260709 13:23:10.161801 23946 tablet_bootstrap.cc:492] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a: Bootstrap starting.
I20260709 13:23:10.165586 23582 leader_election.cc:304] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:10.166536 23933 raft_consensus.cc:2804] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:10.166932 23933 raft_consensus.cc:493] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:10.167312 23933 raft_consensus.cc:3060] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:10.168747 23948 ts_tablet_manager.cc:1434] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:23:10.170010 23969 raft_consensus.cc:493] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:10.176754 23969 raft_consensus.cc:515] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.178547 23969 leader_election.cc:290] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:10.172058 23766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a2818a134994e4c8cfc92ce9523005a" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd" is_pre_election: true
I20260709 13:23:10.175160 23946 tablet_bootstrap.cc:654] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:10.179234 23766 raft_consensus.cc:2468] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 0.
I20260709 13:23:10.180461 23768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b78deb2eabd845dcae6bb7158b2aafff" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd" is_pre_election: true
I20260709 13:23:10.181142 23768 raft_consensus.cc:2468] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 0.
I20260709 13:23:10.182830 23658 leader_election.cc:304] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11398324f1664f29be659b7034c8eedd, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:10.186107 23960 leader_election.cc:290] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:10.187156 23969 raft_consensus.cc:2804] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:10.187619 23969 raft_consensus.cc:493] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:10.187975 23969 raft_consensus.cc:3060] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:10.190448 23946 tablet_bootstrap.cc:492] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a: No bootstrap required, opened a new log
I20260709 13:23:10.191022 23946 ts_tablet_manager.cc:1403] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a: Time spent bootstrapping tablet: real 0.029s	user 0.019s	sys 0.000s
I20260709 13:23:10.191334 23933 raft_consensus.cc:515] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.196563 23693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a2818a134994e4c8cfc92ce9523005a" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
I20260709 13:23:10.197167 23693 raft_consensus.cc:3060] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:10.194252 23946 raft_consensus.cc:359] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.199616 23946 raft_consensus.cc:385] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:10.199950 23946 raft_consensus.cc:740] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Initialized, Role: FOLLOWER
I20260709 13:23:10.200771 23946 consensus_queue.cc:260] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.201642 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b78deb2eabd845dcae6bb7158b2aafff" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a" is_pre_election: true
I20260709 13:23:10.202368 23619 raft_consensus.cc:2468] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 0.
I20260709 13:23:10.203202 23946 ts_tablet_manager.cc:1434] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:23:10.204334 23946 tablet_bootstrap.cc:492] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a: Bootstrap starting.
I20260709 13:23:10.205503 23693 raft_consensus.cc:2468] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 1.
I20260709 13:23:10.206862 23582 leader_election.cc:304] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:10.207363 23911 raft_consensus.cc:493] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:10.207804 23960 raft_consensus.cc:2804] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:10.207883 23911 raft_consensus.cc:515] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.208817 23960 raft_consensus.cc:697] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [term 1 LEADER]: Becoming Leader. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Running, Role: LEADER
I20260709 13:23:10.209895 23960 consensus_queue.cc:237] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.210397 23911 leader_election.cc:290] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121)
I20260709 13:23:10.212590 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9ba0a8c02b403ab321dc3bc2a5affb" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a" is_pre_election: true
W20260709 13:23:10.214231 23733 leader_election.cc:343] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:10.215821 23946 tablet_bootstrap.cc:654] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:10.218552 23768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a2818a134994e4c8cfc92ce9523005a" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:10.219408 23768 raft_consensus.cc:3060] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:10.221657 23692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9ba0a8c02b403ab321dc3bc2a5affb" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f" is_pre_election: true
I20260709 13:23:10.222445 23692 raft_consensus.cc:2468] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 0.
I20260709 13:23:10.225176 23731 leader_election.cc:304] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11398324f1664f29be659b7034c8eedd, b403848a6de0483c82db9cd37d1d5a2f; no voters: 025b14df93f44ae39aefc3c0573cc19a
I20260709 13:23:10.226475 23911 raft_consensus.cc:2804] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:10.226864 23911 raft_consensus.cc:493] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:10.227206 23911 raft_consensus.cc:3060] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:10.236380 23911 raft_consensus.cc:515] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.243522 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9ba0a8c02b403ab321dc3bc2a5affb" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:10.243790 23517 catalog_manager.cc:5697] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f reported cstate change: term changed from 0 to 1, leader changed from <none> to b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66). New cstate: current_term: 1 leader_uuid: "b403848a6de0483c82db9cd37d1d5a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: UNKNOWN } } }
W20260709 13:23:10.245716 23733 leader_election.cc:343] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:10.246834 23933 leader_election.cc:290] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 election: Requested vote from peers b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:10.250147 23768 raft_consensus.cc:2468] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 1.
I20260709 13:23:10.260040 23693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9ba0a8c02b403ab321dc3bc2a5affb" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
I20260709 13:23:10.260699 23693 raft_consensus.cc:3060] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:10.261406 23946 tablet_bootstrap.cc:492] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a: No bootstrap required, opened a new log
I20260709 13:23:10.261916 23946 ts_tablet_manager.cc:1403] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a: Time spent bootstrapping tablet: real 0.058s	user 0.047s	sys 0.004s
I20260709 13:23:10.264818 23946 raft_consensus.cc:359] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.265513 23946 raft_consensus.cc:385] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:10.265816 23946 raft_consensus.cc:740] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Initialized, Role: FOLLOWER
I20260709 13:23:10.266669 23946 consensus_queue.cc:260] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.261205 23516 catalog_manager.cc:5697] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a reported cstate change: term changed from 0 to 1, leader changed from <none> to 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65). New cstate: current_term: 1 leader_uuid: "025b14df93f44ae39aefc3c0573cc19a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:10.269308 23946 ts_tablet_manager.cc:1434] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 13:23:10.269335 23693 raft_consensus.cc:2468] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 1.
I20260709 13:23:10.270375 23946 tablet_bootstrap.cc:492] T c97da58ded5f4813b217185ce853e15e P 025b14df93f44ae39aefc3c0573cc19a: Bootstrap starting.
I20260709 13:23:10.275652 23731 leader_election.cc:304] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11398324f1664f29be659b7034c8eedd, b403848a6de0483c82db9cd37d1d5a2f; no voters: 025b14df93f44ae39aefc3c0573cc19a
I20260709 13:23:10.276000 23960 consensus_queue.cc:1048] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Connected to new peer: Peer: permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:10.283259 23960 raft_consensus.cc:493] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:10.285269 23933 consensus_queue.cc:1048] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:10.286638 23987 raft_consensus.cc:2804] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:10.285274 23956 raft_consensus.cc:493] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:10.287184 23911 leader_election.cc:290] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121)
I20260709 13:23:10.287473 23956 raft_consensus.cc:515] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.196154 23969 raft_consensus.cc:515] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.288105 23987 raft_consensus.cc:697] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 1 LEADER]: Becoming Leader. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Running, Role: LEADER
I20260709 13:23:10.289094 23987 consensus_queue.cc:237] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.290766 23768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b78deb2eabd845dcae6bb7158b2aafff" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:10.291432 23768 raft_consensus.cc:3060] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:10.292392 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b78deb2eabd845dcae6bb7158b2aafff" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:10.293121 23619 raft_consensus.cc:3060] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:10.296479 23946 tablet_bootstrap.cc:654] T c97da58ded5f4813b217185ce853e15e P 025b14df93f44ae39aefc3c0573cc19a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:10.298633 23692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9ba0a8c02b403ab321dc3bc2a5affb" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f" is_pre_election: true
I20260709 13:23:10.298873 23969 leader_election.cc:290] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:10.299556 23692 raft_consensus.cc:2393] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 025b14df93f44ae39aefc3c0573cc19a in current term 1: Already voted for candidate 11398324f1664f29be659b7034c8eedd in this term.
I20260709 13:23:10.299474 23768 raft_consensus.cc:2468] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 1.
I20260709 13:23:10.305838 23658 leader_election.cc:304] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11398324f1664f29be659b7034c8eedd, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:10.306670 23969 raft_consensus.cc:2804] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:10.307374 23969 raft_consensus.cc:697] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Becoming Leader. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Running, Role: LEADER
I20260709 13:23:10.307547 23956 leader_election.cc:290] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:10.283784 23960 raft_consensus.cc:515] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.308305 23969 consensus_queue.cc:237] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.314622 23692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2edafaf1fec84770b4792ec38c697f19" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f" is_pre_election: true
I20260709 13:23:10.317200 23767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2edafaf1fec84770b4792ec38c697f19" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd" is_pre_election: true
I20260709 13:23:10.318210 23767 raft_consensus.cc:2393] T 2edafaf1fec84770b4792ec38c697f19 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 025b14df93f44ae39aefc3c0573cc19a in current term 1: Already voted for candidate b403848a6de0483c82db9cd37d1d5a2f in this term.
I20260709 13:23:10.319789 23960 leader_election.cc:290] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:10.320453 23582 leader_election.cc:304] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a; no voters: 11398324f1664f29be659b7034c8eedd, b403848a6de0483c82db9cd37d1d5a2f
I20260709 13:23:10.325038 23956 raft_consensus.cc:3060] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:10.329144 23946 tablet_bootstrap.cc:492] T c97da58ded5f4813b217185ce853e15e P 025b14df93f44ae39aefc3c0573cc19a: No bootstrap required, opened a new log
I20260709 13:23:10.329393 23768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9ba0a8c02b403ab321dc3bc2a5affb" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd" is_pre_election: true
I20260709 13:23:10.329717 23946 ts_tablet_manager.cc:1403] T c97da58ded5f4813b217185ce853e15e P 025b14df93f44ae39aefc3c0573cc19a: Time spent bootstrapping tablet: real 0.060s	user 0.017s	sys 0.003s
I20260709 13:23:10.330914 23583 leader_election.cc:304] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a; no voters: 11398324f1664f29be659b7034c8eedd, b403848a6de0483c82db9cd37d1d5a2f
I20260709 13:23:10.332082 23960 raft_consensus.cc:3060] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:10.332451 23946 raft_consensus.cc:359] T c97da58ded5f4813b217185ce853e15e P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.333163 23946 raft_consensus.cc:385] T c97da58ded5f4813b217185ce853e15e P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:10.333406 23946 raft_consensus.cc:740] T c97da58ded5f4813b217185ce853e15e P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Initialized, Role: FOLLOWER
I20260709 13:23:10.334136 23946 consensus_queue.cc:260] T c97da58ded5f4813b217185ce853e15e P 025b14df93f44ae39aefc3c0573cc19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.336194 23946 ts_tablet_manager.cc:1434] T c97da58ded5f4813b217185ce853e15e P 025b14df93f44ae39aefc3c0573cc19a: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:23:10.340473 23619 raft_consensus.cc:2468] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 1.
I20260709 13:23:10.341243 23956 raft_consensus.cc:2749] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:10.348914 23946 tablet_bootstrap.cc:492] T 54ccbd59a1d04accaacf5f20864df594 P 025b14df93f44ae39aefc3c0573cc19a: Bootstrap starting.
I20260709 13:23:10.354784 23960 raft_consensus.cc:2749] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:10.356029 23946 tablet_bootstrap.cc:654] T 54ccbd59a1d04accaacf5f20864df594 P 025b14df93f44ae39aefc3c0573cc19a: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:10.355556 23516 catalog_manager.cc:5697] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f reported cstate change: term changed from 0 to 1, leader changed from <none> to b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66). New cstate: current_term: 1 leader_uuid: "b403848a6de0483c82db9cd37d1d5a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:10.370769 23946 tablet_bootstrap.cc:492] T 54ccbd59a1d04accaacf5f20864df594 P 025b14df93f44ae39aefc3c0573cc19a: No bootstrap required, opened a new log
I20260709 13:23:10.371304 23946 ts_tablet_manager.cc:1403] T 54ccbd59a1d04accaacf5f20864df594 P 025b14df93f44ae39aefc3c0573cc19a: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.003s
I20260709 13:23:10.374095 23946 raft_consensus.cc:359] T 54ccbd59a1d04accaacf5f20864df594 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.374797 23946 raft_consensus.cc:385] T 54ccbd59a1d04accaacf5f20864df594 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:10.375061 23946 raft_consensus.cc:740] T 54ccbd59a1d04accaacf5f20864df594 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Initialized, Role: FOLLOWER
I20260709 13:23:10.375855 23946 consensus_queue.cc:260] T 54ccbd59a1d04accaacf5f20864df594 P 025b14df93f44ae39aefc3c0573cc19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.377959 23946 ts_tablet_manager.cc:1434] T 54ccbd59a1d04accaacf5f20864df594 P 025b14df93f44ae39aefc3c0573cc19a: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.003s
I20260709 13:23:10.384315 23516 catalog_manager.cc:5697] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd reported cstate change: term changed from 0 to 1, leader changed from <none> to 11398324f1664f29be659b7034c8eedd (127.11.63.67). New cstate: current_term: 1 leader_uuid: "11398324f1664f29be659b7034c8eedd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:10.447873 23987 raft_consensus.cc:493] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:10.448379 23987 raft_consensus.cc:515] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.450344 23987 leader_election.cc:290] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121)
I20260709 13:23:10.451597 23693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97da58ded5f4813b217185ce853e15e" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f" is_pre_election: true
I20260709 13:23:10.451747 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97da58ded5f4813b217185ce853e15e" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a" is_pre_election: true
I20260709 13:23:10.452219 23693 raft_consensus.cc:2468] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 0.
I20260709 13:23:10.452543 23618 raft_consensus.cc:2468] T c97da58ded5f4813b217185ce853e15e P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 0.
I20260709 13:23:10.453325 23731 leader_election.cc:304] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11398324f1664f29be659b7034c8eedd, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:10.454208 23987 raft_consensus.cc:2804] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:10.454591 23987 raft_consensus.cc:493] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:10.454948 23987 raft_consensus.cc:3060] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:10.463697 23987 raft_consensus.cc:515] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.466169 23987 leader_election.cc:290] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121)
I20260709 13:23:10.466998 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97da58ded5f4813b217185ce853e15e" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:10.467515 23693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97da58ded5f4813b217185ce853e15e" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
I20260709 13:23:10.467692 23618 raft_consensus.cc:3060] T c97da58ded5f4813b217185ce853e15e P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:10.468147 23693 raft_consensus.cc:3060] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:10.475533 23618 raft_consensus.cc:2468] T c97da58ded5f4813b217185ce853e15e P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 1.
I20260709 13:23:10.475816 23693 raft_consensus.cc:2468] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 1.
I20260709 13:23:10.476980 23733 leader_election.cc:304] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, 11398324f1664f29be659b7034c8eedd; no voters: 
I20260709 13:23:10.480134 23987 raft_consensus.cc:2804] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:10.480675 23987 raft_consensus.cc:697] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [term 1 LEADER]: Becoming Leader. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Running, Role: LEADER
I20260709 13:23:10.481678 23987 consensus_queue.cc:237] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.491672 23515 catalog_manager.cc:5697] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd reported cstate change: term changed from 0 to 1, leader changed from <none> to 11398324f1664f29be659b7034c8eedd (127.11.63.67). New cstate: current_term: 1 leader_uuid: "11398324f1664f29be659b7034c8eedd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:10.502653 23969 consensus_queue.cc:1048] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:10.515801 23974 consensus_queue.cc:1048] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:10.561717 23974 raft_consensus.cc:493] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:10.562261 23974 raft_consensus.cc:515] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.564419 23974 leader_election.cc:290] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:10.565172 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54ccbd59a1d04accaacf5f20864df594" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a" is_pre_election: true
I20260709 13:23:10.565608 23766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54ccbd59a1d04accaacf5f20864df594" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd" is_pre_election: true
I20260709 13:23:10.566040 23618 raft_consensus.cc:2468] T 54ccbd59a1d04accaacf5f20864df594 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 0.
I20260709 13:23:10.566233 23766 raft_consensus.cc:2468] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 0.
I20260709 13:23:10.567493 23659 leader_election.cc:304] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:10.568410 23974 raft_consensus.cc:2804] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:10.568977 23974 raft_consensus.cc:493] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:10.569291 23974 raft_consensus.cc:3060] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:10.575670 23974 raft_consensus.cc:515] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.577421 23974 leader_election.cc:290] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:10.578444 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54ccbd59a1d04accaacf5f20864df594" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:10.578651 23766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54ccbd59a1d04accaacf5f20864df594" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:10.579032 23618 raft_consensus.cc:3060] T 54ccbd59a1d04accaacf5f20864df594 P 025b14df93f44ae39aefc3c0573cc19a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:10.579391 23766 raft_consensus.cc:3060] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:10.584558 23618 raft_consensus.cc:2468] T 54ccbd59a1d04accaacf5f20864df594 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 1.
I20260709 13:23:10.586141 23659 leader_election.cc:304] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:10.587189 23974 raft_consensus.cc:2804] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:10.588231 23974 raft_consensus.cc:697] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Becoming Leader. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Running, Role: LEADER
I20260709 13:23:10.588529 23973 consensus_queue.cc:1048] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:10.588888 23766 raft_consensus.cc:2468] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 1.
I20260709 13:23:10.589099 23974 consensus_queue.cc:237] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.599761 23516 catalog_manager.cc:5697] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f reported cstate change: term changed from 0 to 1, leader changed from <none> to b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66). New cstate: current_term: 1 leader_uuid: "b403848a6de0483c82db9cd37d1d5a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:10.606010 23973 consensus_queue.cc:1048] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:10.634832 23995 consensus_queue.cc:1048] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:10.647158 23956 consensus_queue.cc:1048] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Connected to new peer: Peer: permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:10.705993 23973 consensus_queue.cc:1048] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:10.706782 23975 consensus_queue.cc:1048] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:10.726338 23975 consensus_queue.cc:1048] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:10.765130 23973 consensus_queue.cc:1048] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:10.774515 23911 consensus_queue.cc:1048] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:10.787460 23987 consensus_queue.cc:1048] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [LEADER]: Connected to new peer: Peer: permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:10.818116 23692 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b78deb2eabd845dcae6bb7158b2aafff"
dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
mode: GRACEFUL
new_leader_uuid: "025b14df93f44ae39aefc3c0573cc19a"
 from {username='slave'} at 127.0.0.1:38346
I20260709 13:23:10.818893 23692 raft_consensus.cc:606] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Received request to transfer leadership to TS 025b14df93f44ae39aefc3c0573cc19a
I20260709 13:23:10.826427 23692 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b2912e06fc9944968c191d9e3366d314"
dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
mode: GRACEFUL
new_leader_uuid: "025b14df93f44ae39aefc3c0573cc19a"
 from {username='slave'} at 127.0.0.1:38346
I20260709 13:23:10.827129 23692 raft_consensus.cc:606] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Received request to transfer leadership to TS 025b14df93f44ae39aefc3c0573cc19a
I20260709 13:23:10.836418 23692 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2edafaf1fec84770b4792ec38c697f19"
dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
mode: GRACEFUL
new_leader_uuid: "025b14df93f44ae39aefc3c0573cc19a"
 from {username='slave'} at 127.0.0.1:38346
I20260709 13:23:10.837100 23692 raft_consensus.cc:606] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Received request to transfer leadership to TS 025b14df93f44ae39aefc3c0573cc19a
I20260709 13:23:10.843228 23618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a2818a134994e4c8cfc92ce9523005a"
dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
mode: GRACEFUL
new_leader_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
 from {username='slave'} at 127.0.0.1:49382
I20260709 13:23:10.843982 23618 raft_consensus.cc:606] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [term 1 LEADER]: Received request to transfer leadership to TS b403848a6de0483c82db9cd37d1d5a2f
I20260709 13:23:10.907760 23973 raft_consensus.cc:993] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f: : Instructing follower 025b14df93f44ae39aefc3c0573cc19a to start an election
I20260709 13:23:10.909240 23975 raft_consensus.cc:1081] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Signalling peer 025b14df93f44ae39aefc3c0573cc19a to start an election
I20260709 13:23:10.910552 23766 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c9ba0a8c02b403ab321dc3bc2a5affb"
dest_uuid: "11398324f1664f29be659b7034c8eedd"
mode: GRACEFUL
new_leader_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
 from {username='slave'} at 127.0.0.1:38654
I20260709 13:23:10.911803 23766 raft_consensus.cc:606] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 1 LEADER]: Received request to transfer leadership to TS b403848a6de0483c82db9cd37d1d5a2f
I20260709 13:23:10.912140 23618 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2edafaf1fec84770b4792ec38c697f19"
dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
 from {username='slave'} at 127.0.0.1:49366
I20260709 13:23:10.913012 23618 raft_consensus.cc:493] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:10.913666 23618 raft_consensus.cc:3060] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:10.922447 23766 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c97da58ded5f4813b217185ce853e15e"
dest_uuid: "11398324f1664f29be659b7034c8eedd"
mode: GRACEFUL
new_leader_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
 from {username='slave'} at 127.0.0.1:38654
I20260709 13:23:10.923416 23766 raft_consensus.cc:606] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [term 1 LEADER]: Received request to transfer leadership to TS b403848a6de0483c82db9cd37d1d5a2f
I20260709 13:23:10.927003 23618 raft_consensus.cc:515] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.931979 23692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2edafaf1fec84770b4792ec38c697f19" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
I20260709 13:23:10.932210 23618 leader_election.cc:290] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 2 election: Requested vote from peers b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:10.935819 23693 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54ccbd59a1d04accaacf5f20864df594"
dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
mode: GRACEFUL
new_leader_uuid: "11398324f1664f29be659b7034c8eedd"
 from {username='slave'} at 127.0.0.1:38346
I20260709 13:23:10.936614 23693 raft_consensus.cc:606] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Received request to transfer leadership to TS 11398324f1664f29be659b7034c8eedd
I20260709 13:23:10.936975 23766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2edafaf1fec84770b4792ec38c697f19" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:10.932997 23692 raft_consensus.cc:3055] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:10.938800 23692 raft_consensus.cc:740] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Running, Role: LEADER
I20260709 13:23:10.940438 23692 consensus_queue.cc:260] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.942974 23692 raft_consensus.cc:3060] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:10.953902 23692 raft_consensus.cc:2468] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 2.
I20260709 13:23:10.955762 23582 leader_election.cc:304] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f; no voters: 11398324f1664f29be659b7034c8eedd
I20260709 13:23:10.957048 23956 raft_consensus.cc:2804] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:10.957697 23956 raft_consensus.cc:697] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a [term 2 LEADER]: Becoming Leader. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Running, Role: LEADER
I20260709 13:23:10.959251 23956 consensus_queue.cc:237] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:10.971067 23516 catalog_manager.cc:5697] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a reported cstate change: term changed from 1 to 2, leader changed from b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66) to 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65). New cstate: current_term: 2 leader_uuid: "025b14df93f44ae39aefc3c0573cc19a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:11.010759 23973 consensus_queue.cc:1048] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:11.021278 23975 raft_consensus.cc:993] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f: : Instructing follower 11398324f1664f29be659b7034c8eedd to start an election
I20260709 13:23:11.021944 23969 raft_consensus.cc:1081] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Signalling peer 11398324f1664f29be659b7034c8eedd to start an election
I20260709 13:23:11.026238 23766 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "54ccbd59a1d04accaacf5f20864df594"
dest_uuid: "11398324f1664f29be659b7034c8eedd"
 from {username='slave'} at 127.0.0.1:38642
I20260709 13:23:11.026908 23766 raft_consensus.cc:493] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:11.027383 23766 raft_consensus.cc:3060] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:11.028412 23969 consensus_queue.cc:1048] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:11.044828 23766 raft_consensus.cc:515] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:11.049057 23766 leader_election.cc:290] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 2 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121)
I20260709 13:23:11.049552 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54ccbd59a1d04accaacf5f20864df594" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:11.052933 23692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54ccbd59a1d04accaacf5f20864df594" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
I20260709 13:23:11.053663 23692 raft_consensus.cc:3055] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:11.054041 23692 raft_consensus.cc:740] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Running, Role: LEADER
I20260709 13:23:11.055085 23692 consensus_queue.cc:260] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
W20260709 13:23:11.056164 23969 consensus_queue.cc:1175] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "025b14df93f44ae39aefc3c0573cc19a" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 13:23:11.056326 23692 raft_consensus.cc:3060] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:11.059167 24012 consensus_queue.cc:1048] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [LEADER]: Connected to new peer: Peer: permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:11.067858 23692 raft_consensus.cc:2468] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 2.
I20260709 13:23:11.069819 23731 leader_election.cc:304] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 11398324f1664f29be659b7034c8eedd, b403848a6de0483c82db9cd37d1d5a2f; no voters: 025b14df93f44ae39aefc3c0573cc19a
I20260709 13:23:11.071425 24012 raft_consensus.cc:2804] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:11.072011 24012 raft_consensus.cc:697] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd [term 2 LEADER]: Becoming Leader. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Running, Role: LEADER
I20260709 13:23:11.073118 24012 consensus_queue.cc:237] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:11.090164 23987 consensus_queue.cc:1048] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:11.091928 23517 catalog_manager.cc:5697] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd reported cstate change: term changed from 1 to 2, leader changed from b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66) to 11398324f1664f29be659b7034c8eedd (127.11.63.67). New cstate: current_term: 2 leader_uuid: "11398324f1664f29be659b7034c8eedd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:11.102421 23911 raft_consensus.cc:993] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd: : Instructing follower b403848a6de0483c82db9cd37d1d5a2f to start an election
I20260709 13:23:11.102916 23911 raft_consensus.cc:1081] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [term 1 LEADER]: Signalling peer b403848a6de0483c82db9cd37d1d5a2f to start an election
I20260709 13:23:11.104946 23692 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c97da58ded5f4813b217185ce853e15e"
dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
 from {username='slave'} at 127.0.0.1:38316
I20260709 13:23:11.107386 23692 raft_consensus.cc:493] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:11.107889 23692 raft_consensus.cc:3060] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:11.115700 23692 raft_consensus.cc:515] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:11.117523 23692 leader_election.cc:290] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 2 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:11.118255 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97da58ded5f4813b217185ce853e15e" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:11.118664 23766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97da58ded5f4813b217185ce853e15e" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:11.118865 23618 raft_consensus.cc:3060] T c97da58ded5f4813b217185ce853e15e P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:11.119218 23766 raft_consensus.cc:3055] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:11.119531 23766 raft_consensus.cc:740] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Running, Role: LEADER
I20260709 13:23:11.120393 23766 consensus_queue.cc:260] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:11.121554 23766 raft_consensus.cc:3060] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:11.126102 23618 raft_consensus.cc:2468] T c97da58ded5f4813b217185ce853e15e P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 2.
I20260709 13:23:11.127326 23659 leader_election.cc:304] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:11.128031 23969 raft_consensus.cc:2804] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:11.128499 23969 raft_consensus.cc:697] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f [term 2 LEADER]: Becoming Leader. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Running, Role: LEADER
I20260709 13:23:11.128981 23766 raft_consensus.cc:2468] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 2.
I20260709 13:23:11.129523 23969 consensus_queue.cc:237] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:11.134513 23995 raft_consensus.cc:993] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a: : Instructing follower b403848a6de0483c82db9cd37d1d5a2f to start an election
I20260709 13:23:11.135171 23956 raft_consensus.cc:1081] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [term 1 LEADER]: Signalling peer b403848a6de0483c82db9cd37d1d5a2f to start an election
I20260709 13:23:11.136693 23692 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a2818a134994e4c8cfc92ce9523005a"
dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
 from {username='slave'} at 127.0.0.1:38330
I20260709 13:23:11.137250 23692 raft_consensus.cc:493] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:11.137672 23692 raft_consensus.cc:3060] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:11.137367 23517 catalog_manager.cc:5697] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f reported cstate change: term changed from 1 to 2, leader changed from 11398324f1664f29be659b7034c8eedd (127.11.63.67) to b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66). New cstate: current_term: 2 leader_uuid: "b403848a6de0483c82db9cd37d1d5a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:11.144754 23692 raft_consensus.cc:515] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:11.147274 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a2818a134994e4c8cfc92ce9523005a" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:11.147389 23692 leader_election.cc:290] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 2 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:11.147487 23766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a2818a134994e4c8cfc92ce9523005a" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:11.147868 23618 raft_consensus.cc:3055] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:11.148216 23618 raft_consensus.cc:740] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Running, Role: LEADER
I20260709 13:23:11.148283 23766 raft_consensus.cc:3060] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:11.149029 23618 consensus_queue.cc:260] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:11.150147 23618 raft_consensus.cc:3060] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:11.153970 23766 raft_consensus.cc:2468] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 2.
I20260709 13:23:11.154470 23618 raft_consensus.cc:2468] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 2.
I20260709 13:23:11.155375 23658 leader_election.cc:304] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11398324f1664f29be659b7034c8eedd, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:11.156183 23969 raft_consensus.cc:2804] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:11.156724 23969 raft_consensus.cc:697] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f [term 2 LEADER]: Becoming Leader. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Running, Role: LEADER
I20260709 13:23:11.157814 23969 consensus_queue.cc:237] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:11.164944 23517 catalog_manager.cc:5697] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f reported cstate change: term changed from 1 to 2, leader changed from 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65) to b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66). New cstate: current_term: 2 leader_uuid: "b403848a6de0483c82db9cd37d1d5a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:11.207419 23969 raft_consensus.cc:993] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f: : Instructing follower 025b14df93f44ae39aefc3c0573cc19a to start an election
I20260709 13:23:11.207989 23969 raft_consensus.cc:1081] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Signalling peer 025b14df93f44ae39aefc3c0573cc19a to start an election
I20260709 13:23:11.209662 23618 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b2912e06fc9944968c191d9e3366d314"
dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
 from {username='slave'} at 127.0.0.1:49366
I20260709 13:23:11.210059 23618 raft_consensus.cc:493] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:11.210291 23618 raft_consensus.cc:3060] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:11.214668 23618 raft_consensus.cc:515] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:11.216152 23618 leader_election.cc:290] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 2 election: Requested vote from peers b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:11.217239 23766 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2912e06fc9944968c191d9e3366d314" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:11.217231 23692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2912e06fc9944968c191d9e3366d314" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
I20260709 13:23:11.218395 23766 raft_consensus.cc:3060] T b2912e06fc9944968c191d9e3366d314 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:11.218580 23692 raft_consensus.cc:3055] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:11.219002 23692 raft_consensus.cc:740] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Running, Role: LEADER
I20260709 13:23:11.219858 23692 consensus_queue.cc:260] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:11.221174 23692 raft_consensus.cc:3060] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:11.223883 23766 raft_consensus.cc:2468] T b2912e06fc9944968c191d9e3366d314 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 2.
I20260709 13:23:11.225144 23583 leader_election.cc:304] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, 11398324f1664f29be659b7034c8eedd; no voters: 
I20260709 13:23:11.226073 23956 raft_consensus.cc:2804] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:11.226346 23692 raft_consensus.cc:2468] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 2.
I20260709 13:23:11.226634 23956 raft_consensus.cc:697] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a [term 2 LEADER]: Becoming Leader. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Running, Role: LEADER
I20260709 13:23:11.227761 23956 consensus_queue.cc:237] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:11.236263 23517 catalog_manager.cc:5697] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a reported cstate change: term changed from 1 to 2, leader changed from b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66) to 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65). New cstate: current_term: 2 leader_uuid: "025b14df93f44ae39aefc3c0573cc19a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:11.285902 23973 raft_consensus.cc:993] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f: : Instructing follower 025b14df93f44ae39aefc3c0573cc19a to start an election
I20260709 13:23:11.286638 23969 raft_consensus.cc:1081] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Signalling peer 025b14df93f44ae39aefc3c0573cc19a to start an election
I20260709 13:23:11.288287 23618 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b78deb2eabd845dcae6bb7158b2aafff"
dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
 from {username='slave'} at 127.0.0.1:49366
I20260709 13:23:11.288806 23618 raft_consensus.cc:493] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:11.289158 23618 raft_consensus.cc:3060] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:11.294150 23618 raft_consensus.cc:515] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:11.295557 23618 leader_election.cc:290] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 2 election: Requested vote from peers b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:11.296588 23692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b78deb2eabd845dcae6bb7158b2aafff" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
I20260709 13:23:11.296741 23768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b78deb2eabd845dcae6bb7158b2aafff" candidate_uuid: "025b14df93f44ae39aefc3c0573cc19a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:11.297346 23692 raft_consensus.cc:3055] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:11.297400 23768 raft_consensus.cc:3060] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:11.297730 23692 raft_consensus.cc:740] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Running, Role: LEADER
I20260709 13:23:11.298689 23692 consensus_queue.cc:260] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:11.299702 23692 raft_consensus.cc:3060] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:11.304409 23768 raft_consensus.cc:2468] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 2.
I20260709 13:23:11.304454 23692 raft_consensus.cc:2468] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 025b14df93f44ae39aefc3c0573cc19a in term 2.
I20260709 13:23:11.305819 23583 leader_election.cc:304] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, 11398324f1664f29be659b7034c8eedd; no voters: 
I20260709 13:23:11.306790 23956 raft_consensus.cc:2804] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:11.307267 23956 raft_consensus.cc:697] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [term 2 LEADER]: Becoming Leader. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Running, Role: LEADER
I20260709 13:23:11.308557 23956 consensus_queue.cc:237] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:11.315397 23517 catalog_manager.cc:5697] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a reported cstate change: term changed from 1 to 2, leader changed from b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66) to 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65). New cstate: current_term: 2 leader_uuid: "025b14df93f44ae39aefc3c0573cc19a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:11.371487 23987 raft_consensus.cc:993] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd: : Instructing follower b403848a6de0483c82db9cd37d1d5a2f to start an election
I20260709 13:23:11.371959 23911 raft_consensus.cc:1081] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 1 LEADER]: Signalling peer b403848a6de0483c82db9cd37d1d5a2f to start an election
I20260709 13:23:11.373390 23692 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0c9ba0a8c02b403ab321dc3bc2a5affb"
dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
 from {username='slave'} at 127.0.0.1:38316
I20260709 13:23:11.373842 23692 raft_consensus.cc:493] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:11.374078 23692 raft_consensus.cc:3060] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:11.379112 23692 raft_consensus.cc:515] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:11.381145 23692 leader_election.cc:290] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 2 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), 11398324f1664f29be659b7034c8eedd (127.11.63.67:38311)
I20260709 13:23:11.382325 23768 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9ba0a8c02b403ab321dc3bc2a5affb" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "11398324f1664f29be659b7034c8eedd"
I20260709 13:23:11.382453 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9ba0a8c02b403ab321dc3bc2a5affb" candidate_uuid: "b403848a6de0483c82db9cd37d1d5a2f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:11.383112 23768 raft_consensus.cc:3055] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:11.383152 23619 raft_consensus.cc:3060] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:11.383553 23768 raft_consensus.cc:740] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Running, Role: LEADER
I20260709 13:23:11.384497 23768 consensus_queue.cc:260] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:11.386070 23768 raft_consensus.cc:3060] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:11.391945 23619 raft_consensus.cc:2468] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 2.
I20260709 13:23:11.393092 23768 raft_consensus.cc:2468] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b403848a6de0483c82db9cd37d1d5a2f in term 2.
I20260709 13:23:11.393272 23659 leader_election.cc:304] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:11.394199 23969 raft_consensus.cc:2804] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:11.394536 23969 raft_consensus.cc:697] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [term 2 LEADER]: Becoming Leader. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Running, Role: LEADER
I20260709 13:23:11.395364 23969 consensus_queue.cc:237] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:11.401758 23517 catalog_manager.cc:5697] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f reported cstate change: term changed from 1 to 2, leader changed from 11398324f1664f29be659b7034c8eedd (127.11.63.67) to b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66). New cstate: current_term: 2 leader_uuid: "b403848a6de0483c82db9cd37d1d5a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:11.457445 23619 raft_consensus.cc:3060] T 54ccbd59a1d04accaacf5f20864df594 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:11.462483 23619 raft_consensus.cc:1275] T 54ccbd59a1d04accaacf5f20864df594 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Refusing update from remote peer 11398324f1664f29be659b7034c8eedd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:11.463723 23911 consensus_queue.cc:1048] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd [LEADER]: Connected to new peer: Peer: permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:11.471730 23692 raft_consensus.cc:1275] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Refusing update from remote peer 11398324f1664f29be659b7034c8eedd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:11.473150 24017 consensus_queue.cc:1048] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:11.511931 23619 raft_consensus.cc:1275] T c97da58ded5f4813b217185ce853e15e P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Refusing update from remote peer b403848a6de0483c82db9cd37d1d5a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:11.513406 23969 consensus_queue.cc:1048] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:11.520298 23768 raft_consensus.cc:3060] T 2edafaf1fec84770b4792ec38c697f19 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:11.521797 23766 raft_consensus.cc:1275] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Refusing update from remote peer b403848a6de0483c82db9cd37d1d5a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:11.523454 23973 consensus_queue.cc:1048] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:11.527878 23768 raft_consensus.cc:1275] T 2edafaf1fec84770b4792ec38c697f19 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Refusing update from remote peer 025b14df93f44ae39aefc3c0573cc19a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:11.529500 23956 consensus_queue.cc:1048] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Connected to new peer: Peer: permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:11.542995 23692 raft_consensus.cc:1275] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Refusing update from remote peer 025b14df93f44ae39aefc3c0573cc19a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:11.544505 23956 consensus_queue.cc:1048] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:11.621222 23767 raft_consensus.cc:1275] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Refusing update from remote peer b403848a6de0483c82db9cd37d1d5a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:11.622608 23973 consensus_queue.cc:1048] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:11.632206 23619 raft_consensus.cc:1275] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Refusing update from remote peer b403848a6de0483c82db9cd37d1d5a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:11.635202 24022 consensus_queue.cc:1048] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:11.725360 23767 raft_consensus.cc:1275] T b2912e06fc9944968c191d9e3366d314 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Refusing update from remote peer 025b14df93f44ae39aefc3c0573cc19a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:11.727147 24020 consensus_queue.cc:1048] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Connected to new peer: Peer: permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:11.736586 23692 raft_consensus.cc:1275] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Refusing update from remote peer 025b14df93f44ae39aefc3c0573cc19a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:11.738300 24020 consensus_queue.cc:1048] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:11.746536 23767 raft_consensus.cc:1275] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Refusing update from remote peer 025b14df93f44ae39aefc3c0573cc19a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:11.748361 23995 consensus_queue.cc:1048] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Connected to new peer: Peer: permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:11.759783 23692 raft_consensus.cc:1275] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Refusing update from remote peer 025b14df93f44ae39aefc3c0573cc19a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:11.761572 23956 consensus_queue.cc:1048] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:11.900833 23767 raft_consensus.cc:1275] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Refusing update from remote peer b403848a6de0483c82db9cd37d1d5a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:11.902220 23969 consensus_queue.cc:1048] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:11.913367 23619 raft_consensus.cc:1275] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Refusing update from remote peer b403848a6de0483c82db9cd37d1d5a2f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:11.915164 23969 consensus_queue.cc:1048] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:13.943193 23516 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35640:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:23:13.964821 23516 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35640:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:23:14.008647 11517 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 13:23:14.124008 24041 log.cc:927] Time spent T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:23:14.222885 24072 tablet.cc:1724] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:14.259312 24075 tablet.cc:1724] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:14.267961 24070 tablet.cc:1724] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:14.330061 24076 tablet.cc:1724] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:14.255429 24074 tablet.cc:1724] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:23:14.337105 23658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:38311, user_credentials={real_user=slave}} blocked reactor thread for 72153.1us
W20260709 13:23:14.337278 23720 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:5a2818a134994e4c8cfc92ce9523005a)
W20260709 13:23:14.345808 23720 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:5a2818a134994e4c8cfc92ce9523005a)
W20260709 13:23:14.356824 23720 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:5a2818a134994e4c8cfc92ce9523005a)
I20260709 13:23:14.365428 24080 tablet.cc:1724] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:14.352360 24083 tablet.cc:1724] T 54ccbd59a1d04accaacf5f20864df594 P 025b14df93f44ae39aefc3c0573cc19a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:14.397853 24092 tablet.cc:1724] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:14.319448 24078 tablet.cc:1724] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:14.431638 24090 tablet.cc:1724] T 5ea06411c11b43d6b60a50bb271ae151 P b403848a6de0483c82db9cd37d1d5a2f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:14.453285 24072 tablet.cc:1724] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:14.454492 23694 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c9ba0a8c02b403ab321dc3bc2a5affb"
dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
mode: GRACEFUL
new_leader_uuid: "11398324f1664f29be659b7034c8eedd"
 from {username='slave'} at 127.0.0.1:40846
I20260709 13:23:14.455487 24074 tablet.cc:1724] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:14.477041 24078 tablet.cc:1724] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:14.481276 24070 tablet.cc:1724] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:14.491786 23516 catalog_manager.cc:5789] TS b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121) does not have the latest schema for tablet 0c9ba0a8c02b403ab321dc3bc2a5affb (table soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]). Expected version 1 got 0
I20260709 13:23:14.498838 23516 catalog_manager.cc:5789] TS b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121) does not have the latest schema for tablet 32aca5c2e490419a9b679c38609e37e3 (table soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]). Expected version 1 got 0
I20260709 13:23:14.499423 23516 catalog_manager.cc:5789] TS b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121) does not have the latest schema for tablet c97da58ded5f4813b217185ce853e15e (table soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]). Expected version 1 got 0
I20260709 13:23:14.494972 23694 raft_consensus.cc:606] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [term 2 LEADER]: Received request to transfer leadership to TS 11398324f1664f29be659b7034c8eedd
I20260709 13:23:14.492707 24080 tablet.cc:1724] T 5ea06411c11b43d6b60a50bb271ae151 P 11398324f1664f29be659b7034c8eedd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:14.503753 24094 tablet.cc:1724] T b2912e06fc9944968c191d9e3366d314 P 11398324f1664f29be659b7034c8eedd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:14.546409 24096 tablet.cc:1724] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:14.549095 24093 raft_consensus.cc:993] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f: : Instructing follower 11398324f1664f29be659b7034c8eedd to start an election
I20260709 13:23:14.511067 24092 tablet.cc:1724] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:14.555388 24093 raft_consensus.cc:1081] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [term 2 LEADER]: Signalling peer 11398324f1664f29be659b7034c8eedd to start an election
I20260709 13:23:14.554630 24078 tablet.cc:1724] T 32aca5c2e490419a9b679c38609e37e3 P 11398324f1664f29be659b7034c8eedd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:14.549221 24072 tablet.cc:1724] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:14.557833 23768 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0c9ba0a8c02b403ab321dc3bc2a5affb"
dest_uuid: "11398324f1664f29be659b7034c8eedd"
 from {username='slave'} at 127.0.0.1:38642
I20260709 13:23:14.558441 23768 raft_consensus.cc:493] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:14.558822 23768 raft_consensus.cc:3060] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:14.566601 23768 raft_consensus.cc:515] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:14.569061 23768 leader_election.cc:290] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 3 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121)
I20260709 13:23:14.569829 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9ba0a8c02b403ab321dc3bc2a5affb" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:14.570760 23617 raft_consensus.cc:3060] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:14.584969 24075 tablet.cc:1724] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:23:14.590591 23659 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:44419, user_credentials={real_user=slave}} blocked reactor thread for 84666.5us
I20260709 13:23:14.596833 23617 raft_consensus.cc:2468] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 3.
I20260709 13:23:14.602802 23733 leader_election.cc:304] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, 11398324f1664f29be659b7034c8eedd; no voters: 
I20260709 13:23:14.596999 24080 tablet.cc:1724] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:14.603924 24065 raft_consensus.cc:2804] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:23:14.604568 24065 raft_consensus.cc:697] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 3 LEADER]: Becoming Leader. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Running, Role: LEADER
I20260709 13:23:14.605832 24090 tablet.cc:1724] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:14.606809 23693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c9ba0a8c02b403ab321dc3bc2a5affb" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
I20260709 13:23:14.607491 23693 raft_consensus.cc:3055] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:23:14.607811 23693 raft_consensus.cc:740] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Running, Role: LEADER
I20260709 13:23:14.608034 23619 raft_consensus.cc:1240] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a [term 3 FOLLOWER]: Rejecting Update request from peer b403848a6de0483c82db9cd37d1d5a2f for earlier term 2. Current term is 3. Ops: []
I20260709 13:23:14.608604 23693 consensus_queue.cc:260] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:14.599260 24096 tablet.cc:1724] T 2edafaf1fec84770b4792ec38c697f19 P 11398324f1664f29be659b7034c8eedd: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:14.614028 24083 tablet.cc:1724] T 32aca5c2e490419a9b679c38609e37e3 P 025b14df93f44ae39aefc3c0573cc19a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:14.616091 24074 tablet.cc:1724] T c97da58ded5f4813b217185ce853e15e P 025b14df93f44ae39aefc3c0573cc19a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:14.605475 24065 consensus_queue.cc:237] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:14.622330 23693 raft_consensus.cc:3060] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:14.663658 23517 catalog_manager.cc:5697] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd reported cstate change: term changed from 2 to 3, leader changed from b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66) to 11398324f1664f29be659b7034c8eedd (127.11.63.67). New cstate: current_term: 3 leader_uuid: "11398324f1664f29be659b7034c8eedd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:14.667409 23693 raft_consensus.cc:2468] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 3.
I20260709 13:23:14.674168 23619 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b78deb2eabd845dcae6bb7158b2aafff"
dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
mode: GRACEFUL
new_leader_uuid: "11398324f1664f29be659b7034c8eedd"
 from {username='slave'} at 127.0.0.1:36820
I20260709 13:23:14.674783 23619 raft_consensus.cc:606] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [term 2 LEADER]: Received request to transfer leadership to TS 11398324f1664f29be659b7034c8eedd
I20260709 13:23:14.676260 11517 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:23:14.676890 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:23:14.686980 23619 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "84909b8ded8d40518c017313ef8bfeb8"
dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
mode: GRACEFUL
new_leader_uuid: "11398324f1664f29be659b7034c8eedd"
 from {username='slave'} at 127.0.0.1:36820
I20260709 13:23:14.687556 23619 raft_consensus.cc:606] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [term 1 LEADER]: Received request to transfer leadership to TS 11398324f1664f29be659b7034c8eedd
I20260709 13:23:14.689893 23516 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e: soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051] alter complete (version 1)
I20260709 13:23:14.690217 23693 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f909688025304faa9849596796e56988"
dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
mode: GRACEFUL
new_leader_uuid: "11398324f1664f29be659b7034c8eedd"
 from {username='slave'} at 127.0.0.1:40846
I20260709 13:23:14.690982 23693 raft_consensus.cc:606] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Received request to transfer leadership to TS 11398324f1664f29be659b7034c8eedd
I20260709 13:23:14.692468 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:23:14.692912 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:23:14.697475 24098 raft_consensus.cc:993] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f: : Instructing follower 11398324f1664f29be659b7034c8eedd to start an election
I20260709 13:23:14.698190 24030 raft_consensus.cc:1081] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Signalling peer 11398324f1664f29be659b7034c8eedd to start an election
I20260709 13:23:14.699656 23764 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f909688025304faa9849596796e56988"
dest_uuid: "11398324f1664f29be659b7034c8eedd"
 from {username='slave'} at 127.0.0.1:38642
I20260709 13:23:14.700121 23764 raft_consensus.cc:493] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:14.700428 23764 raft_consensus.cc:3060] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:14.706205 23764 raft_consensus.cc:515] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:14.708086 23764 leader_election.cc:290] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 2 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121)
I20260709 13:23:14.708762 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f909688025304faa9849596796e56988" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:14.709120 23693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f909688025304faa9849596796e56988" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
I20260709 13:23:14.709424 23619 raft_consensus.cc:3060] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:14.709720 23693 raft_consensus.cc:3055] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:14.710035 23693 raft_consensus.cc:740] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b403848a6de0483c82db9cd37d1d5a2f, State: Running, Role: LEADER
I20260709 13:23:14.710709 23693 consensus_queue.cc:260] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:14.711627 23693 raft_consensus.cc:3060] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:14.714589 23619 raft_consensus.cc:2468] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 2.
I20260709 13:23:14.715708 23733 leader_election.cc:304] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, 11398324f1664f29be659b7034c8eedd; no voters: 
I20260709 13:23:14.716302 23693 raft_consensus.cc:2468] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 2.
I20260709 13:23:14.716594 24065 raft_consensus.cc:2804] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:14.717269 24065 raft_consensus.cc:697] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [term 2 LEADER]: Becoming Leader. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Running, Role: LEADER
I20260709 13:23:14.718254 24065 consensus_queue.cc:237] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:14.726235 23516 catalog_manager.cc:5697] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd reported cstate change: term changed from 1 to 2, leader changed from b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66) to 11398324f1664f29be659b7034c8eedd (127.11.63.67). New cstate: current_term: 2 leader_uuid: "11398324f1664f29be659b7034c8eedd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:14.752892 24079 raft_consensus.cc:993] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a: : Instructing follower 11398324f1664f29be659b7034c8eedd to start an election
I20260709 13:23:14.753369 24079 raft_consensus.cc:1081] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [term 1 LEADER]: Signalling peer 11398324f1664f29be659b7034c8eedd to start an election
I20260709 13:23:14.754855 23764 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "84909b8ded8d40518c017313ef8bfeb8"
dest_uuid: "11398324f1664f29be659b7034c8eedd"
 from {username='slave'} at 127.0.0.1:38648
I20260709 13:23:14.755321 23764 raft_consensus.cc:493] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:14.755584 23764 raft_consensus.cc:3060] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:14.761132 23764 raft_consensus.cc:515] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:14.763103 23764 leader_election.cc:290] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 2 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121)
I20260709 13:23:14.763751 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84909b8ded8d40518c017313ef8bfeb8" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:14.764119 23693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84909b8ded8d40518c017313ef8bfeb8" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
I20260709 13:23:14.764539 23619 raft_consensus.cc:3055] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:14.764725 23693 raft_consensus.cc:3060] T 84909b8ded8d40518c017313ef8bfeb8 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:14.764907 23619 raft_consensus.cc:740] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Running, Role: LEADER
I20260709 13:23:14.765798 23619 consensus_queue.cc:260] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:14.766772 23619 raft_consensus.cc:3060] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:14.770009 23693 raft_consensus.cc:2468] T 84909b8ded8d40518c017313ef8bfeb8 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 2.
I20260709 13:23:14.771073 23731 leader_election.cc:304] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11398324f1664f29be659b7034c8eedd, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:14.771899 24065 raft_consensus.cc:2804] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:14.771942 23619 raft_consensus.cc:2468] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 2.
I20260709 13:23:14.772514 24065 raft_consensus.cc:697] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd [term 2 LEADER]: Becoming Leader. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Running, Role: LEADER
I20260709 13:23:14.773584 24065 consensus_queue.cc:237] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:14.781383 23516 catalog_manager.cc:5697] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd reported cstate change: term changed from 1 to 2, leader changed from 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65) to 11398324f1664f29be659b7034c8eedd (127.11.63.67). New cstate: current_term: 2 leader_uuid: "11398324f1664f29be659b7034c8eedd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:15.076507 23694 raft_consensus.cc:1275] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [term 3 FOLLOWER]: Refusing update from remote peer 11398324f1664f29be659b7034c8eedd: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:23:15.077936 23911 consensus_queue.cc:1048] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:23:15.090535 23619 raft_consensus.cc:1275] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a [term 3 FOLLOWER]: Refusing update from remote peer 11398324f1664f29be659b7034c8eedd: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:23:15.092175 23911 consensus_queue.cc:1048] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [LEADER]: Connected to new peer: Peer: permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:23:15.096050 24074 tablet.cc:1724] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:15.105175 23516 catalog_manager.cc:5789] TS 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419) does not have the latest schema for tablet 0c9ba0a8c02b403ab321dc3bc2a5affb (table soft_deleted_table [id=4898ea4c94ef411d9ed0f26668a0c051]). Expected version 1 got 0
I20260709 13:23:15.125662 23619 raft_consensus.cc:1275] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Refusing update from remote peer 11398324f1664f29be659b7034c8eedd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:15.127120 23911 consensus_queue.cc:1048] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [LEADER]: Connected to new peer: Peer: permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:15.136556 23694 raft_consensus.cc:1275] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Refusing update from remote peer 11398324f1664f29be659b7034c8eedd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:15.138301 24106 consensus_queue.cc:1048] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:15.180172 24079 raft_consensus.cc:993] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a: : Instructing follower 11398324f1664f29be659b7034c8eedd to start an election
I20260709 13:23:15.180671 24020 raft_consensus.cc:1081] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [term 2 LEADER]: Signalling peer 11398324f1664f29be659b7034c8eedd to start an election
I20260709 13:23:15.182195 23764 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b78deb2eabd845dcae6bb7158b2aafff"
dest_uuid: "11398324f1664f29be659b7034c8eedd"
 from {username='slave'} at 127.0.0.1:38648
I20260709 13:23:15.182632 23764 raft_consensus.cc:493] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:15.182906 23764 raft_consensus.cc:3060] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:15.188849 23764 raft_consensus.cc:515] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:15.191159 23764 leader_election.cc:290] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 3 election: Requested vote from peers 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419), b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121)
I20260709 13:23:15.191941 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b78deb2eabd845dcae6bb7158b2aafff" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "025b14df93f44ae39aefc3c0573cc19a"
I20260709 13:23:15.192265 23694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b78deb2eabd845dcae6bb7158b2aafff" candidate_uuid: "11398324f1664f29be659b7034c8eedd" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "b403848a6de0483c82db9cd37d1d5a2f"
I20260709 13:23:15.192620 23619 raft_consensus.cc:3055] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:23:15.192840 23694 raft_consensus.cc:3060] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:15.192951 23619 raft_consensus.cc:740] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 025b14df93f44ae39aefc3c0573cc19a, State: Running, Role: LEADER
I20260709 13:23:15.193800 23619 consensus_queue.cc:260] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:15.195122 23619 raft_consensus.cc:3060] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:15.201979 23619 raft_consensus.cc:2468] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 3.
I20260709 13:23:15.201979 23694 raft_consensus.cc:2468] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11398324f1664f29be659b7034c8eedd in term 3.
I20260709 13:23:15.203248 23733 leader_election.cc:304] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 025b14df93f44ae39aefc3c0573cc19a, 11398324f1664f29be659b7034c8eedd, b403848a6de0483c82db9cd37d1d5a2f; no voters: 
I20260709 13:23:15.204034 24106 raft_consensus.cc:2804] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:23:15.204453 24106 raft_consensus.cc:697] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd [term 3 LEADER]: Becoming Leader. State: Replica: 11398324f1664f29be659b7034c8eedd, State: Running, Role: LEADER
I20260709 13:23:15.205130 24106 consensus_queue.cc:237] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } }
I20260709 13:23:15.211808 23517 catalog_manager.cc:5697] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd reported cstate change: term changed from 2 to 3, leader changed from 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65) to 11398324f1664f29be659b7034c8eedd (127.11.63.67). New cstate: current_term: 3 leader_uuid: "11398324f1664f29be659b7034c8eedd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11398324f1664f29be659b7034c8eedd" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 38311 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:15.288226 23694 raft_consensus.cc:1275] T 84909b8ded8d40518c017313ef8bfeb8 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Refusing update from remote peer 11398324f1664f29be659b7034c8eedd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:15.289541 24106 consensus_queue.cc:1048] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:15.299608 23619 raft_consensus.cc:1275] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Refusing update from remote peer 11398324f1664f29be659b7034c8eedd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:15.301227 24065 consensus_queue.cc:1048] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd [LEADER]: Connected to new peer: Peer: permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:15.655238 23619 raft_consensus.cc:1275] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [term 3 FOLLOWER]: Refusing update from remote peer 11398324f1664f29be659b7034c8eedd: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:23:15.656386 24065 consensus_queue.cc:1048] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd [LEADER]: Connected to new peer: Peer: permanent_uuid: "025b14df93f44ae39aefc3c0573cc19a" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 44419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:23:15.663403 23694 raft_consensus.cc:1275] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [term 3 FOLLOWER]: Refusing update from remote peer 11398324f1664f29be659b7034c8eedd: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:23:15.664680 23911 consensus_queue.cc:1048] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b403848a6de0483c82db9cd37d1d5a2f" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 36121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:23:15.755722 11517 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:23:15.761085 11517 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:23:15.761454 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:23:15.767313 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:23:15.767910 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:23:16.799517 11517 tablet_server.cc:179] TabletServer@127.11.63.65:0 shutting down...
I20260709 13:23:16.824998 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:23:16.826000 11517 tablet_replica.cc:333] T 54ccbd59a1d04accaacf5f20864df594 P 025b14df93f44ae39aefc3c0573cc19a: stopping tablet replica
I20260709 13:23:16.827147 11517 raft_consensus.cc:2243] T 54ccbd59a1d04accaacf5f20864df594 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.827816 11517 raft_consensus.cc:2272] T 54ccbd59a1d04accaacf5f20864df594 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.830938 11517 tablet_replica.cc:333] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a: stopping tablet replica
I20260709 13:23:16.831746 11517 raft_consensus.cc:2243] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.832224 11517 raft_consensus.cc:2272] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 025b14df93f44ae39aefc3c0573cc19a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.834313 11517 tablet_replica.cc:333] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a: stopping tablet replica
I20260709 13:23:16.834985 11517 raft_consensus.cc:2243] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.835426 11517 raft_consensus.cc:2272] T b78deb2eabd845dcae6bb7158b2aafff P 025b14df93f44ae39aefc3c0573cc19a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.837477 11517 tablet_replica.cc:333] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a: stopping tablet replica
I20260709 13:23:16.838088 11517 raft_consensus.cc:2243] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.838522 11517 raft_consensus.cc:2272] T 5a2818a134994e4c8cfc92ce9523005a P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.840518 11517 tablet_replica.cc:333] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a: stopping tablet replica
I20260709 13:23:16.841230 11517 raft_consensus.cc:2243] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.842056 11517 raft_consensus.cc:2272] T 2edafaf1fec84770b4792ec38c697f19 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:16.842778 23733 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 4 similar messages]
I20260709 13:23:16.844340 11517 tablet_replica.cc:333] T f7cf9d97b16441ca84a93f2e71f0af81 P 025b14df93f44ae39aefc3c0573cc19a: stopping tablet replica
I20260709 13:23:16.845028 11517 raft_consensus.cc:2243] T f7cf9d97b16441ca84a93f2e71f0af81 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.845541 11517 raft_consensus.cc:2272] T f7cf9d97b16441ca84a93f2e71f0af81 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:16.845888 23733 consensus_peers.cc:597] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd -> Peer 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419): Couldn't send request to peer 025b14df93f44ae39aefc3c0573cc19a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:16.847826 11517 tablet_replica.cc:333] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a: stopping tablet replica
I20260709 13:23:16.848495 11517 raft_consensus.cc:2243] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.849287 11517 raft_consensus.cc:2272] T b2912e06fc9944968c191d9e3366d314 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.852053 11517 tablet_replica.cc:333] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a: stopping tablet replica
I20260709 13:23:16.852788 11517 raft_consensus.cc:2243] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.853478 11517 raft_consensus.cc:2272] T 5ea06411c11b43d6b60a50bb271ae151 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.856019 11517 tablet_replica.cc:333] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a: stopping tablet replica
I20260709 13:23:16.856631 11517 raft_consensus.cc:2243] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.857242 11517 raft_consensus.cc:2272] T 4c184e66c94b4436a4a2028ec5665be5 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.859381 11517 tablet_replica.cc:333] T 32aca5c2e490419a9b679c38609e37e3 P 025b14df93f44ae39aefc3c0573cc19a: stopping tablet replica
I20260709 13:23:16.860174 11517 raft_consensus.cc:2243] T 32aca5c2e490419a9b679c38609e37e3 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.860837 11517 raft_consensus.cc:2272] T 32aca5c2e490419a9b679c38609e37e3 P 025b14df93f44ae39aefc3c0573cc19a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.862874 11517 tablet_replica.cc:333] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a: stopping tablet replica
I20260709 13:23:16.863391 11517 raft_consensus.cc:2243] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.863741 11517 raft_consensus.cc:2272] T f909688025304faa9849596796e56988 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.865831 11517 tablet_replica.cc:333] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a: stopping tablet replica
I20260709 13:23:16.866293 11517 raft_consensus.cc:2243] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.866695 11517 raft_consensus.cc:2272] T 84909b8ded8d40518c017313ef8bfeb8 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.868857 11517 tablet_replica.cc:333] T 109ce4dfd3d44642bc560d6cb563ceef P 025b14df93f44ae39aefc3c0573cc19a: stopping tablet replica
I20260709 13:23:16.869439 11517 raft_consensus.cc:2243] T 109ce4dfd3d44642bc560d6cb563ceef P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.869907 11517 raft_consensus.cc:2272] T 109ce4dfd3d44642bc560d6cb563ceef P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.872102 11517 tablet_replica.cc:333] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a: stopping tablet replica
I20260709 13:23:16.872623 11517 raft_consensus.cc:2243] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.873333 11517 raft_consensus.cc:2272] T d524509732f8487ba5f6ad44e5811ecb P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.875388 11517 tablet_replica.cc:333] T 8531f6c11726411eab62bfa778de8701 P 025b14df93f44ae39aefc3c0573cc19a: stopping tablet replica
I20260709 13:23:16.875946 11517 raft_consensus.cc:2243] T 8531f6c11726411eab62bfa778de8701 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.876381 11517 raft_consensus.cc:2272] T 8531f6c11726411eab62bfa778de8701 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.878438 11517 tablet_replica.cc:333] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a: stopping tablet replica
I20260709 13:23:16.878934 11517 raft_consensus.cc:2243] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.879837 11517 raft_consensus.cc:2272] T 10e65bc4fc654cd6a302b592b8b3a733 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.882058 11517 tablet_replica.cc:333] T c97da58ded5f4813b217185ce853e15e P 025b14df93f44ae39aefc3c0573cc19a: stopping tablet replica
I20260709 13:23:16.882553 11517 raft_consensus.cc:2243] T c97da58ded5f4813b217185ce853e15e P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.882985 11517 raft_consensus.cc:2272] T c97da58ded5f4813b217185ce853e15e P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.884877 11517 tablet_replica.cc:333] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a: stopping tablet replica
I20260709 13:23:16.885339 11517 raft_consensus.cc:2243] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:16.886150 11517 raft_consensus.cc:2272] T 91bc474b5f64410d9a9c21058a2feca0 P 025b14df93f44ae39aefc3c0573cc19a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:16.899003 23733 consensus_peers.cc:597] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd -> Peer 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419): Couldn't send request to peer 025b14df93f44ae39aefc3c0573cc19a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:16.929239 11517 tablet_server.cc:196] TabletServer@127.11.63.65:0 shutdown complete.
I20260709 13:23:16.964157 11517 tablet_server.cc:179] TabletServer@127.11.63.66:0 shutting down...
I20260709 13:23:16.993438 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:23:16.994274 11517 tablet_replica.cc:333] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f: stopping tablet replica
I20260709 13:23:16.995054 11517 raft_consensus.cc:2243] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.995652 11517 raft_consensus.cc:2272] T 54ccbd59a1d04accaacf5f20864df594 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:16.998399 11517 tablet_replica.cc:333] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f: stopping tablet replica
I20260709 13:23:16.999082 11517 raft_consensus.cc:2243] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:16.999590 11517 raft_consensus.cc:2272] T 0c9ba0a8c02b403ab321dc3bc2a5affb P b403848a6de0483c82db9cd37d1d5a2f [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:17.001464 23731 consensus_peers.cc:597] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd -> Peer b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121): Couldn't send request to peer b403848a6de0483c82db9cd37d1d5a2f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:17.002242 11517 tablet_replica.cc:333] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f: stopping tablet replica
I20260709 13:23:17.003013 11517 raft_consensus.cc:2243] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:23:17.003732 23731 consensus_peers.cc:597] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd -> Peer b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121): Couldn't send request to peer b403848a6de0483c82db9cd37d1d5a2f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:17.003947 11517 raft_consensus.cc:2272] T 5a2818a134994e4c8cfc92ce9523005a P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.006119 11517 tablet_replica.cc:333] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f: stopping tablet replica
I20260709 13:23:17.006779 11517 raft_consensus.cc:2243] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:17.007261 11517 raft_consensus.cc:2272] T b78deb2eabd845dcae6bb7158b2aafff P b403848a6de0483c82db9cd37d1d5a2f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.009056 11517 tablet_replica.cc:333] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f: stopping tablet replica
I20260709 13:23:17.009681 11517 raft_consensus.cc:2243] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:17.010108 11517 raft_consensus.cc:2272] T 2edafaf1fec84770b4792ec38c697f19 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.011930 11517 tablet_replica.cc:333] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f: stopping tablet replica
I20260709 13:23:17.012535 11517 raft_consensus.cc:2243] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:17.013327 11517 raft_consensus.cc:2272] T 4c184e66c94b4436a4a2028ec5665be5 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.015300 11517 tablet_replica.cc:333] T 5ea06411c11b43d6b60a50bb271ae151 P b403848a6de0483c82db9cd37d1d5a2f: stopping tablet replica
I20260709 13:23:17.015853 11517 raft_consensus.cc:2243] T 5ea06411c11b43d6b60a50bb271ae151 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:17.016278 11517 raft_consensus.cc:2272] T 5ea06411c11b43d6b60a50bb271ae151 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.018321 11517 tablet_replica.cc:333] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f: stopping tablet replica
I20260709 13:23:17.018822 11517 raft_consensus.cc:2243] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:17.019189 11517 raft_consensus.cc:2272] T b2912e06fc9944968c191d9e3366d314 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.020864 11517 tablet_replica.cc:333] T f7cf9d97b16441ca84a93f2e71f0af81 P b403848a6de0483c82db9cd37d1d5a2f: stopping tablet replica
I20260709 13:23:17.021317 11517 raft_consensus.cc:2243] T f7cf9d97b16441ca84a93f2e71f0af81 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:17.021780 11517 raft_consensus.cc:2272] T f7cf9d97b16441ca84a93f2e71f0af81 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.023669 11517 tablet_replica.cc:333] T 84909b8ded8d40518c017313ef8bfeb8 P b403848a6de0483c82db9cd37d1d5a2f: stopping tablet replica
I20260709 13:23:17.024158 11517 raft_consensus.cc:2243] T 84909b8ded8d40518c017313ef8bfeb8 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:17.024519 11517 raft_consensus.cc:2272] T 84909b8ded8d40518c017313ef8bfeb8 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.026687 11517 tablet_replica.cc:333] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f: stopping tablet replica
I20260709 13:23:17.027150 11517 raft_consensus.cc:2243] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:17.027899 11517 raft_consensus.cc:2272] T 109ce4dfd3d44642bc560d6cb563ceef P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.030397 11517 tablet_replica.cc:333] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f: stopping tablet replica
I20260709 13:23:17.030841 11517 raft_consensus.cc:2243] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:17.031222 11517 raft_consensus.cc:2272] T d524509732f8487ba5f6ad44e5811ecb P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.033025 11517 tablet_replica.cc:333] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f: stopping tablet replica
I20260709 13:23:17.033506 11517 raft_consensus.cc:2243] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:17.034297 11517 raft_consensus.cc:2272] T 32aca5c2e490419a9b679c38609e37e3 P b403848a6de0483c82db9cd37d1d5a2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.036201 11517 tablet_replica.cc:333] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f: stopping tablet replica
I20260709 13:23:17.036697 11517 raft_consensus.cc:2243] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:17.037045 11517 raft_consensus.cc:2272] T f909688025304faa9849596796e56988 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.038866 11517 tablet_replica.cc:333] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f: stopping tablet replica
I20260709 13:23:17.039325 11517 raft_consensus.cc:2243] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:17.040045 11517 raft_consensus.cc:2272] T 8531f6c11726411eab62bfa778de8701 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.041949 11517 tablet_replica.cc:333] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f: stopping tablet replica
I20260709 13:23:17.042440 11517 raft_consensus.cc:2243] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:17.042790 11517 raft_consensus.cc:2272] T 10e65bc4fc654cd6a302b592b8b3a733 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.044555 11517 tablet_replica.cc:333] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f: stopping tablet replica
I20260709 13:23:17.045002 11517 raft_consensus.cc:2243] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:17.045733 11517 raft_consensus.cc:2272] T c97da58ded5f4813b217185ce853e15e P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.047374 11517 tablet_replica.cc:333] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f: stopping tablet replica
I20260709 13:23:17.047776 11517 raft_consensus.cc:2243] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:17.048153 11517 raft_consensus.cc:2272] T 91bc474b5f64410d9a9c21058a2feca0 P b403848a6de0483c82db9cd37d1d5a2f [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:17.068033 23731 consensus_peers.cc:597] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd -> Peer b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121): Couldn't send request to peer b403848a6de0483c82db9cd37d1d5a2f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:17.089135 11517 tablet_server.cc:196] TabletServer@127.11.63.66:0 shutdown complete.
W20260709 13:23:17.104565 23733 consensus_peers.cc:597] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd -> Peer 025b14df93f44ae39aefc3c0573cc19a (127.11.63.65:44419): Couldn't send request to peer 025b14df93f44ae39aefc3c0573cc19a. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:44419: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:17.123482 11517 tablet_server.cc:179] TabletServer@127.11.63.67:0 shutting down...
W20260709 13:23:17.156126 23731 consensus_peers.cc:597] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd -> Peer b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121): Couldn't send request to peer b403848a6de0483c82db9cd37d1d5a2f. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:36121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:17.156905 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:23:17.157725 11517 tablet_replica.cc:333] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd: stopping tablet replica
I20260709 13:23:17.158795 11517 raft_consensus.cc:2243] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:17.159754 11517 raft_consensus.cc:2272] T 54ccbd59a1d04accaacf5f20864df594 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.161791 11517 tablet_replica.cc:333] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd: stopping tablet replica
I20260709 13:23:17.162475 11517 raft_consensus.cc:2243] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:23:17.163177 11517 raft_consensus.cc:2272] T 0c9ba0a8c02b403ab321dc3bc2a5affb P 11398324f1664f29be659b7034c8eedd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.165072 11517 tablet_replica.cc:333] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd: stopping tablet replica
I20260709 13:23:17.165735 11517 raft_consensus.cc:2243] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:23:17.166435 11517 raft_consensus.cc:2272] T b78deb2eabd845dcae6bb7158b2aafff P 11398324f1664f29be659b7034c8eedd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.168344 11517 tablet_replica.cc:333] T 2edafaf1fec84770b4792ec38c697f19 P 11398324f1664f29be659b7034c8eedd: stopping tablet replica
I20260709 13:23:17.168898 11517 raft_consensus.cc:2243] T 2edafaf1fec84770b4792ec38c697f19 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:17.169365 11517 raft_consensus.cc:2272] T 2edafaf1fec84770b4792ec38c697f19 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.171185 11517 tablet_replica.cc:333] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd: stopping tablet replica
I20260709 13:23:17.171729 11517 raft_consensus.cc:2243] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:17.172245 11517 raft_consensus.cc:2272] T 5a2818a134994e4c8cfc92ce9523005a P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.174374 11517 tablet_replica.cc:333] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd: stopping tablet replica
I20260709 13:23:17.174913 11517 raft_consensus.cc:2243] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:17.175360 11517 raft_consensus.cc:2272] T c97da58ded5f4813b217185ce853e15e P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.177253 11517 tablet_replica.cc:333] T 91bc474b5f64410d9a9c21058a2feca0 P 11398324f1664f29be659b7034c8eedd: stopping tablet replica
I20260709 13:23:17.177838 11517 raft_consensus.cc:2243] T 91bc474b5f64410d9a9c21058a2feca0 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:17.178262 11517 raft_consensus.cc:2272] T 91bc474b5f64410d9a9c21058a2feca0 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.180231 11517 tablet_replica.cc:333] T 5ea06411c11b43d6b60a50bb271ae151 P 11398324f1664f29be659b7034c8eedd: stopping tablet replica
I20260709 13:23:17.180867 11517 raft_consensus.cc:2243] T 5ea06411c11b43d6b60a50bb271ae151 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:17.181283 11517 raft_consensus.cc:2272] T 5ea06411c11b43d6b60a50bb271ae151 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.183084 11517 tablet_replica.cc:333] T b2912e06fc9944968c191d9e3366d314 P 11398324f1664f29be659b7034c8eedd: stopping tablet replica
I20260709 13:23:17.183640 11517 raft_consensus.cc:2243] T b2912e06fc9944968c191d9e3366d314 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:17.184091 11517 raft_consensus.cc:2272] T b2912e06fc9944968c191d9e3366d314 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.185985 11517 tablet_replica.cc:333] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd: stopping tablet replica
I20260709 13:23:17.186538 11517 raft_consensus.cc:2243] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:17.187327 11517 raft_consensus.cc:2272] T f7cf9d97b16441ca84a93f2e71f0af81 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.189293 11517 tablet_replica.cc:333] T 10e65bc4fc654cd6a302b592b8b3a733 P 11398324f1664f29be659b7034c8eedd: stopping tablet replica
I20260709 13:23:17.189862 11517 raft_consensus.cc:2243] T 10e65bc4fc654cd6a302b592b8b3a733 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:17.190343 11517 raft_consensus.cc:2272] T 10e65bc4fc654cd6a302b592b8b3a733 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.192255 11517 tablet_replica.cc:333] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd: stopping tablet replica
I20260709 13:23:17.192802 11517 raft_consensus.cc:2243] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:17.193811 11517 raft_consensus.cc:2272] T 84909b8ded8d40518c017313ef8bfeb8 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.196610 11517 tablet_replica.cc:333] T d524509732f8487ba5f6ad44e5811ecb P 11398324f1664f29be659b7034c8eedd: stopping tablet replica
I20260709 13:23:17.197338 11517 raft_consensus.cc:2243] T d524509732f8487ba5f6ad44e5811ecb P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:17.197953 11517 raft_consensus.cc:2272] T d524509732f8487ba5f6ad44e5811ecb P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:17.198135 23731 consensus_peers.cc:597] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd -> Peer b403848a6de0483c82db9cd37d1d5a2f (127.11.63.66:36121): Couldn't send request to peer b403848a6de0483c82db9cd37d1d5a2f. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.66:36121: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:17.200110 11517 tablet_replica.cc:333] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd: stopping tablet replica
I20260709 13:23:17.200749 11517 raft_consensus.cc:2243] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:17.201401 11517 raft_consensus.cc:2272] T 109ce4dfd3d44642bc560d6cb563ceef P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.203612 11517 tablet_replica.cc:333] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd: stopping tablet replica
I20260709 13:23:17.204208 11517 raft_consensus.cc:2243] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:17.204660 11517 raft_consensus.cc:2272] T 8531f6c11726411eab62bfa778de8701 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.206694 11517 tablet_replica.cc:333] T 32aca5c2e490419a9b679c38609e37e3 P 11398324f1664f29be659b7034c8eedd: stopping tablet replica
I20260709 13:23:17.207219 11517 raft_consensus.cc:2243] T 32aca5c2e490419a9b679c38609e37e3 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:17.207643 11517 raft_consensus.cc:2272] T 32aca5c2e490419a9b679c38609e37e3 P 11398324f1664f29be659b7034c8eedd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.209462 11517 tablet_replica.cc:333] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd: stopping tablet replica
I20260709 13:23:17.210038 11517 raft_consensus.cc:2243] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:17.210800 11517 raft_consensus.cc:2272] T f909688025304faa9849596796e56988 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.212812 11517 tablet_replica.cc:333] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd: stopping tablet replica
I20260709 13:23:17.213307 11517 raft_consensus.cc:2243] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:17.213776 11517 raft_consensus.cc:2272] T 4c184e66c94b4436a4a2028ec5665be5 P 11398324f1664f29be659b7034c8eedd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.271119 11517 tablet_server.cc:196] TabletServer@127.11.63.67:0 shutdown complete.
I20260709 13:23:17.307144 11517 master.cc:562] Master@127.11.63.126:37329 shutting down...
I20260709 13:23:17.333812 11517 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:17.334414 11517 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:17.334923 11517 tablet_replica.cc:333] T 00000000000000000000000000000000 P 2633b9a6b4c7452a9f983097f7ba060e: stopping tablet replica
I20260709 13:23:17.355201 11517 master.cc:584] Master@127.11.63.126:37329 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (14569 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:23:17.413130 11517 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.11.63.126:35149
I20260709 13:23:17.414733 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:23:17.421159 24120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:17.421296 24121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:17.422648 24123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:23:17.426301 11517 server_base.cc:1061] running on GCE node
I20260709 13:23:17.427402 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:23:17.427599 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:23:17.427791 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603397427776 us; error 0 us; skew 500 ppm
I20260709 13:23:17.428514 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:23:17.431473 11517 webserver.cc:533] Webserver started at http://127.11.63.126:40041/ using document root <none> and password file <none>
I20260709 13:23:17.432009 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:23:17.432219 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:23:17.432554 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:23:17.433872 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603019296230-11517-0/minicluster-data/master-0-root/instance:
uuid: "62cb1a883353423da73ff1a94b77b261"
format_stamp: "Formatted at 2026-07-09 13:23:17 on dist-test-slave-qv7r"
I20260709 13:23:17.438894 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:23:17.442703 24128 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:17.443653 11517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:23:17.443980 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603019296230-11517-0/minicluster-data/master-0-root
uuid: "62cb1a883353423da73ff1a94b77b261"
format_stamp: "Formatted at 2026-07-09 13:23:17 on dist-test-slave-qv7r"
I20260709 13:23:17.444283 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603019296230-11517-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603019296230-11517-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603019296230-11517-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:23:17.467588 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:23:17.468859 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:23:17.509912 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.126:35149
I20260709 13:23:17.510022 24179 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.126:35149 every 8 connection(s)
I20260709 13:23:17.513789 24180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:17.525367 24180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261: Bootstrap starting.
I20260709 13:23:17.530162 24180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:17.534485 24180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261: No bootstrap required, opened a new log
I20260709 13:23:17.536684 24180 raft_consensus.cc:359] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62cb1a883353423da73ff1a94b77b261" member_type: VOTER }
I20260709 13:23:17.537096 24180 raft_consensus.cc:385] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:17.537309 24180 raft_consensus.cc:740] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62cb1a883353423da73ff1a94b77b261, State: Initialized, Role: FOLLOWER
I20260709 13:23:17.537937 24180 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62cb1a883353423da73ff1a94b77b261" member_type: VOTER }
I20260709 13:23:17.538389 24180 raft_consensus.cc:399] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:23:17.538612 24180 raft_consensus.cc:493] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:23:17.538895 24180 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:17.543744 24180 raft_consensus.cc:515] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62cb1a883353423da73ff1a94b77b261" member_type: VOTER }
I20260709 13:23:17.544267 24180 leader_election.cc:304] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62cb1a883353423da73ff1a94b77b261; no voters: 
I20260709 13:23:17.545382 24180 leader_election.cc:290] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:23:17.545796 24183 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:17.547349 24183 raft_consensus.cc:697] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261 [term 1 LEADER]: Becoming Leader. State: Replica: 62cb1a883353423da73ff1a94b77b261, State: Running, Role: LEADER
I20260709 13:23:17.548013 24183 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62cb1a883353423da73ff1a94b77b261" member_type: VOTER }
I20260709 13:23:17.548465 24180 sys_catalog.cc:565] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:23:17.552534 24185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62cb1a883353423da73ff1a94b77b261. Latest consensus state: current_term: 1 leader_uuid: "62cb1a883353423da73ff1a94b77b261" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62cb1a883353423da73ff1a94b77b261" member_type: VOTER } }
I20260709 13:23:17.552486 24184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62cb1a883353423da73ff1a94b77b261" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62cb1a883353423da73ff1a94b77b261" member_type: VOTER } }
I20260709 13:23:17.553406 24185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261 [sys.catalog]: This master's current role is: LEADER
I20260709 13:23:17.553448 24184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261 [sys.catalog]: This master's current role is: LEADER
I20260709 13:23:17.556084 24190 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:23:17.562718 24190 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:23:17.563951 11517 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:23:17.571739 24190 catalog_manager.cc:1383] Generated new cluster ID: bf019d3530184f6ea9c4e87573e4d0c2
I20260709 13:23:17.572033 24190 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:23:17.591303 24190 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:23:17.592679 24190 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:23:17.606535 24190 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261: Generated new TSK 0
I20260709 13:23:17.607225 24190 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:23:17.631054 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:23:17.637468 24201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:17.638805 24202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:17.640344 24204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:23:17.642908 11517 server_base.cc:1061] running on GCE node
I20260709 13:23:17.646786 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:23:17.646981 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:23:17.647102 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603397647091 us; error 0 us; skew 500 ppm
I20260709 13:23:17.647660 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:23:17.650189 11517 webserver.cc:533] Webserver started at http://127.11.63.65:43229/ using document root <none> and password file <none>
I20260709 13:23:17.650697 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:23:17.650923 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:23:17.651193 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:23:17.652441 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603019296230-11517-0/minicluster-data/ts-0-root/instance:
uuid: "66e4fe095997420ba9949bad99bf4d50"
format_stamp: "Formatted at 2026-07-09 13:23:17 on dist-test-slave-qv7r"
I20260709 13:23:17.657191 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.003s
I20260709 13:23:17.660780 24209 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:17.661689 11517 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:23:17.662014 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603019296230-11517-0/minicluster-data/ts-0-root
uuid: "66e4fe095997420ba9949bad99bf4d50"
format_stamp: "Formatted at 2026-07-09 13:23:17 on dist-test-slave-qv7r"
I20260709 13:23:17.662300 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603019296230-11517-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603019296230-11517-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603019296230-11517-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:23:17.682652 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:23:17.683934 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:23:17.685459 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:23:17.687856 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:23:17.688081 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:17.688333 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:23:17.688515 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:17.730095 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.65:36043
I20260709 13:23:17.730216 24271 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.65:36043 every 8 connection(s)
I20260709 13:23:17.735288 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:23:17.743120 24276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:17.745106 24277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:23:17.746948 11517 server_base.cc:1061] running on GCE node
W20260709 13:23:17.748153 24279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:23:17.749168 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:23:17.749383 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:23:17.749648 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603397749626 us; error 0 us; skew 500 ppm
I20260709 13:23:17.750237 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:23:17.752897 11517 webserver.cc:533] Webserver started at http://127.11.63.66:44257/ using document root <none> and password file <none>
I20260709 13:23:17.753417 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:23:17.753675 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:23:17.753993 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:23:17.755368 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603019296230-11517-0/minicluster-data/ts-1-root/instance:
uuid: "d3ac721977e3467892cbf730c5262219"
format_stamp: "Formatted at 2026-07-09 13:23:17 on dist-test-slave-qv7r"
I20260709 13:23:17.755537 24272 heartbeater.cc:344] Connected to a master server at 127.11.63.126:35149
I20260709 13:23:17.755986 24272 heartbeater.cc:461] Registering TS with master...
I20260709 13:23:17.756742 24272 heartbeater.cc:507] Master 127.11.63.126:35149 requested a full tablet report, sending...
I20260709 13:23:17.758898 24145 ts_manager.cc:194] Registered new tserver with Master: 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043)
I20260709 13:23:17.760588 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:23:17.760739 24145 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39982
I20260709 13:23:17.764413 24284 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:17.765340 11517 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:23:17.765664 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603019296230-11517-0/minicluster-data/ts-1-root
uuid: "d3ac721977e3467892cbf730c5262219"
format_stamp: "Formatted at 2026-07-09 13:23:17 on dist-test-slave-qv7r"
I20260709 13:23:17.765928 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603019296230-11517-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603019296230-11517-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603019296230-11517-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:23:17.782083 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:23:17.783289 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:23:17.784705 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:23:17.787096 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:23:17.787268 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:17.787568 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:23:17.787724 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:17.834244 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.66:40237
I20260709 13:23:17.834370 24346 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.66:40237 every 8 connection(s)
I20260709 13:23:17.839180 11517 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:23:17.847769 24350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:23:17.849037 24351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:23:17.852468 24347 heartbeater.cc:344] Connected to a master server at 127.11.63.126:35149
I20260709 13:23:17.853116 24347 heartbeater.cc:461] Registering TS with master...
W20260709 13:23:17.853921 24353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:23:17.854621 24347 heartbeater.cc:507] Master 127.11.63.126:35149 requested a full tablet report, sending...
I20260709 13:23:17.858364 24145 ts_manager.cc:194] Registered new tserver with Master: d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237)
I20260709 13:23:17.859555 11517 server_base.cc:1061] running on GCE node
I20260709 13:23:17.860369 24145 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39992
I20260709 13:23:17.860634 11517 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:23:17.860980 11517 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:23:17.861163 11517 hybrid_clock.cc:648] HybridClock initialized: now 1783603397861147 us; error 0 us; skew 500 ppm
I20260709 13:23:17.861945 11517 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:23:17.864893 11517 webserver.cc:533] Webserver started at http://127.11.63.67:40217/ using document root <none> and password file <none>
I20260709 13:23:17.865551 11517 fs_manager.cc:362] Metadata directory not provided
I20260709 13:23:17.865785 11517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:23:17.866010 11517 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:23:17.867177 11517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603019296230-11517-0/minicluster-data/ts-2-root/instance:
uuid: "3aa42500bbcd4121a43413ecd5be246d"
format_stamp: "Formatted at 2026-07-09 13:23:17 on dist-test-slave-qv7r"
I20260709 13:23:17.871860 11517 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:23:17.875619 24358 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:17.876580 11517 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 13:23:17.876900 11517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603019296230-11517-0/minicluster-data/ts-2-root
uuid: "3aa42500bbcd4121a43413ecd5be246d"
format_stamp: "Formatted at 2026-07-09 13:23:17 on dist-test-slave-qv7r"
I20260709 13:23:17.877195 11517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603019296230-11517-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603019296230-11517-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskV528fw/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783603019296230-11517-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:23:17.890012 11517 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:23:17.891417 11517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:23:17.893165 11517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:23:17.895496 11517 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:23:17.895740 11517 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:17.895977 11517 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:23:17.896149 11517 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:23:17.938978 11517 rpc_server.cc:307] RPC server started. Bound to: 127.11.63.67:41619
I20260709 13:23:17.939085 24420 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.63.67:41619 every 8 connection(s)
I20260709 13:23:17.953249 24421 heartbeater.cc:344] Connected to a master server at 127.11.63.126:35149
I20260709 13:23:17.953747 24421 heartbeater.cc:461] Registering TS with master...
I20260709 13:23:17.954471 24421 heartbeater.cc:507] Master 127.11.63.126:35149 requested a full tablet report, sending...
I20260709 13:23:17.956413 24145 ts_manager.cc:194] Registered new tserver with Master: 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:17.957257 11517 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014942443s
I20260709 13:23:17.957940 24145 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40006
I20260709 13:23:17.984457 24145 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40014:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:23:17.987239 24145 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:23:18.068905 24386 tablet_service.cc:1511] Processing CreateTablet for tablet 7f2735d0953445d8b941fd6ad09e76fc (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:23:18.069691 24385 tablet_service.cc:1511] Processing CreateTablet for tablet 0f82955421054b95ad8df878b1827aa4 (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:23:18.071431 24386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f2735d0953445d8b941fd6ad09e76fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.072321 24385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f82955421054b95ad8df878b1827aa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.073441 24383 tablet_service.cc:1511] Processing CreateTablet for tablet 0689d527efd74dc1a43383e7f3034595 (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:23:18.073590 24382 tablet_service.cc:1511] Processing CreateTablet for tablet 05cc80ee32e0409c959205fbbcf09e58 (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:23:18.074990 24383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0689d527efd74dc1a43383e7f3034595. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.074618 24384 tablet_service.cc:1511] Processing CreateTablet for tablet a91cd6890c73435e9b43afc4262d4f01 (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:23:18.075649 24237 tablet_service.cc:1511] Processing CreateTablet for tablet 7f2735d0953445d8b941fd6ad09e76fc (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:23:18.076998 24237 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f2735d0953445d8b941fd6ad09e76fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.077790 24236 tablet_service.cc:1511] Processing CreateTablet for tablet 0f82955421054b95ad8df878b1827aa4 (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:23:18.079169 24236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f82955421054b95ad8df878b1827aa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.082756 24235 tablet_service.cc:1511] Processing CreateTablet for tablet a91cd6890c73435e9b43afc4262d4f01 (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:23:18.084168 24235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a91cd6890c73435e9b43afc4262d4f01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.071985 24381 tablet_service.cc:1511] Processing CreateTablet for tablet 0713792c0cb2436b932fadc8d1f428a2 (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:23:18.088441 24232 tablet_service.cc:1511] Processing CreateTablet for tablet 0713792c0cb2436b932fadc8d1f428a2 (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:23:18.090337 24232 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0713792c0cb2436b932fadc8d1f428a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.090817 24233 tablet_service.cc:1511] Processing CreateTablet for tablet 05cc80ee32e0409c959205fbbcf09e58 (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:23:18.092504 24233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05cc80ee32e0409c959205fbbcf09e58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.095098 24234 tablet_service.cc:1511] Processing CreateTablet for tablet 0689d527efd74dc1a43383e7f3034595 (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:23:18.097044 24234 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0689d527efd74dc1a43383e7f3034595. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.098237 24384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a91cd6890c73435e9b43afc4262d4f01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.099215 24381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0713792c0cb2436b932fadc8d1f428a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.100003 24382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05cc80ee32e0409c959205fbbcf09e58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.114297 24312 tablet_service.cc:1511] Processing CreateTablet for tablet 7f2735d0953445d8b941fd6ad09e76fc (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:23:18.115732 24312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f2735d0953445d8b941fd6ad09e76fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.118827 24311 tablet_service.cc:1511] Processing CreateTablet for tablet 0f82955421054b95ad8df878b1827aa4 (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:23:18.120680 24311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f82955421054b95ad8df878b1827aa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.123216 24310 tablet_service.cc:1511] Processing CreateTablet for tablet a91cd6890c73435e9b43afc4262d4f01 (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:23:18.125161 24310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a91cd6890c73435e9b43afc4262d4f01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.129798 24307 tablet_service.cc:1511] Processing CreateTablet for tablet 0713792c0cb2436b932fadc8d1f428a2 (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:23:18.133045 24308 tablet_service.cc:1511] Processing CreateTablet for tablet 05cc80ee32e0409c959205fbbcf09e58 (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:23:18.136301 24309 tablet_service.cc:1511] Processing CreateTablet for tablet 0689d527efd74dc1a43383e7f3034595 (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:23:18.138953 24309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0689d527efd74dc1a43383e7f3034595. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.139703 24307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0713792c0cb2436b932fadc8d1f428a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.146086 24308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05cc80ee32e0409c959205fbbcf09e58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.153896 24231 tablet_service.cc:1511] Processing CreateTablet for tablet 33f74f7b25d0436fa15cc6053960e3a3 (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:23:18.155423 24231 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33f74f7b25d0436fa15cc6053960e3a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.176424 24436 tablet_bootstrap.cc:492] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50: Bootstrap starting.
I20260709 13:23:18.183367 24436 tablet_bootstrap.cc:654] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.194062 24306 tablet_service.cc:1511] Processing CreateTablet for tablet 33f74f7b25d0436fa15cc6053960e3a3 (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:23:18.196393 24306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33f74f7b25d0436fa15cc6053960e3a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.197990 24437 tablet_bootstrap.cc:492] T 0713792c0cb2436b932fadc8d1f428a2 P 3aa42500bbcd4121a43413ecd5be246d: Bootstrap starting.
I20260709 13:23:18.197677 24381 tablet_service.cc:1511] Processing CreateTablet for tablet 33f74f7b25d0436fa15cc6053960e3a3 (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:23:18.199462 24381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33f74f7b25d0436fa15cc6053960e3a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.224766 24439 tablet_bootstrap.cc:492] T 7f2735d0953445d8b941fd6ad09e76fc P d3ac721977e3467892cbf730c5262219: Bootstrap starting.
I20260709 13:23:18.225723 24436 tablet_bootstrap.cc:492] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50: No bootstrap required, opened a new log
I20260709 13:23:18.226375 24436 ts_tablet_manager.cc:1403] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50: Time spent bootstrapping tablet: real 0.050s	user 0.026s	sys 0.010s
I20260709 13:23:18.229835 24436 raft_consensus.cc:359] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } }
I20260709 13:23:18.231604 24436 raft_consensus.cc:385] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.232038 24436 raft_consensus.cc:740] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.233036 24436 consensus_queue.cc:260] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } }
I20260709 13:23:18.246100 24437 tablet_bootstrap.cc:654] T 0713792c0cb2436b932fadc8d1f428a2 P 3aa42500bbcd4121a43413ecd5be246d: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.252043 24439 tablet_bootstrap.cc:654] T 7f2735d0953445d8b941fd6ad09e76fc P d3ac721977e3467892cbf730c5262219: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.258199 24436 ts_tablet_manager.cc:1434] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50: Time spent starting tablet: real 0.031s	user 0.010s	sys 0.008s
I20260709 13:23:18.258282 24272 heartbeater.cc:499] Master 127.11.63.126:35149 was elected leader, sending a full tablet report...
I20260709 13:23:18.266732 24233 tablet_service.cc:1511] Processing CreateTablet for tablet b027fa0920c8440fab0837dd1142fd57 (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:23:18.270430 24436 tablet_bootstrap.cc:492] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50: Bootstrap starting.
I20260709 13:23:18.276221 24233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b027fa0920c8440fab0837dd1142fd57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.290004 24309 tablet_service.cc:1511] Processing CreateTablet for tablet 32b70a25b0264a8a96170bbecee4837d (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:23:18.291350 24436 tablet_bootstrap.cc:654] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.291440 24382 tablet_service.cc:1511] Processing CreateTablet for tablet 32b70a25b0264a8a96170bbecee4837d (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:23:18.292088 24309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32b70a25b0264a8a96170bbecee4837d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.292800 24382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32b70a25b0264a8a96170bbecee4837d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.294538 24310 tablet_service.cc:1511] Processing CreateTablet for tablet b027fa0920c8440fab0837dd1142fd57 (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:23:18.295833 24310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b027fa0920c8440fab0837dd1142fd57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.290534 24236 tablet_service.cc:1511] Processing CreateTablet for tablet 32b70a25b0264a8a96170bbecee4837d (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:23:18.303565 24236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32b70a25b0264a8a96170bbecee4837d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.282668 24381 tablet_service.cc:1511] Processing CreateTablet for tablet b027fa0920c8440fab0837dd1142fd57 (DEFAULT_TABLE table=test-workload [id=03a59babab6d4d92b1570fc14a907eb1]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:23:18.308405 24381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b027fa0920c8440fab0837dd1142fd57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:18.313951 24439 tablet_bootstrap.cc:492] T 7f2735d0953445d8b941fd6ad09e76fc P d3ac721977e3467892cbf730c5262219: No bootstrap required, opened a new log
I20260709 13:23:18.314576 24439 ts_tablet_manager.cc:1403] T 7f2735d0953445d8b941fd6ad09e76fc P d3ac721977e3467892cbf730c5262219: Time spent bootstrapping tablet: real 0.090s	user 0.011s	sys 0.029s
I20260709 13:23:18.317461 24439 raft_consensus.cc:359] T 7f2735d0953445d8b941fd6ad09e76fc P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } }
I20260709 13:23:18.318233 24439 raft_consensus.cc:385] T 7f2735d0953445d8b941fd6ad09e76fc P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.318513 24439 raft_consensus.cc:740] T 7f2735d0953445d8b941fd6ad09e76fc P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3ac721977e3467892cbf730c5262219, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.319234 24439 consensus_queue.cc:260] T 7f2735d0953445d8b941fd6ad09e76fc P d3ac721977e3467892cbf730c5262219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } }
I20260709 13:23:18.322700 24437 tablet_bootstrap.cc:492] T 0713792c0cb2436b932fadc8d1f428a2 P 3aa42500bbcd4121a43413ecd5be246d: No bootstrap required, opened a new log
I20260709 13:23:18.323309 24437 ts_tablet_manager.cc:1403] T 0713792c0cb2436b932fadc8d1f428a2 P 3aa42500bbcd4121a43413ecd5be246d: Time spent bootstrapping tablet: real 0.128s	user 0.018s	sys 0.000s
I20260709 13:23:18.326592 24437 raft_consensus.cc:359] T 0713792c0cb2436b932fadc8d1f428a2 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.327278 24437 raft_consensus.cc:385] T 0713792c0cb2436b932fadc8d1f428a2 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.327561 24437 raft_consensus.cc:740] T 0713792c0cb2436b932fadc8d1f428a2 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.328325 24437 consensus_queue.cc:260] T 0713792c0cb2436b932fadc8d1f428a2 P 3aa42500bbcd4121a43413ecd5be246d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.339131 24439 ts_tablet_manager.cc:1434] T 7f2735d0953445d8b941fd6ad09e76fc P d3ac721977e3467892cbf730c5262219: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.003s
I20260709 13:23:18.340642 24347 heartbeater.cc:499] Master 127.11.63.126:35149 was elected leader, sending a full tablet report...
I20260709 13:23:18.347534 24437 ts_tablet_manager.cc:1434] T 0713792c0cb2436b932fadc8d1f428a2 P 3aa42500bbcd4121a43413ecd5be246d: Time spent starting tablet: real 0.024s	user 0.010s	sys 0.000s
I20260709 13:23:18.349145 24421 heartbeater.cc:499] Master 127.11.63.126:35149 was elected leader, sending a full tablet report...
I20260709 13:23:18.353591 24437 tablet_bootstrap.cc:492] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d: Bootstrap starting.
I20260709 13:23:18.355902 24436 tablet_bootstrap.cc:492] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50: No bootstrap required, opened a new log
I20260709 13:23:18.356568 24436 ts_tablet_manager.cc:1403] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50: Time spent bootstrapping tablet: real 0.086s	user 0.034s	sys 0.033s
I20260709 13:23:18.359088 24439 tablet_bootstrap.cc:492] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219: Bootstrap starting.
I20260709 13:23:18.359752 24436 raft_consensus.cc:359] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.360625 24436 raft_consensus.cc:385] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.360967 24436 raft_consensus.cc:740] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.361886 24436 consensus_queue.cc:260] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.364341 24436 ts_tablet_manager.cc:1434] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:23:18.365576 24436 tablet_bootstrap.cc:492] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50: Bootstrap starting.
I20260709 13:23:18.371479 24437 tablet_bootstrap.cc:654] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.372871 24436 tablet_bootstrap.cc:654] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.384949 24436 tablet_bootstrap.cc:492] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50: No bootstrap required, opened a new log
I20260709 13:23:18.385684 24436 ts_tablet_manager.cc:1403] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.006s
I20260709 13:23:18.381354 24439 tablet_bootstrap.cc:654] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.388970 24436 raft_consensus.cc:359] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.389909 24436 raft_consensus.cc:385] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.390277 24436 raft_consensus.cc:740] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.391185 24436 consensus_queue.cc:260] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.393926 24436 ts_tablet_manager.cc:1434] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:23:18.394358 24437 tablet_bootstrap.cc:492] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d: No bootstrap required, opened a new log
I20260709 13:23:18.394942 24437 ts_tablet_manager.cc:1403] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d: Time spent bootstrapping tablet: real 0.042s	user 0.015s	sys 0.012s
I20260709 13:23:18.395293 24436 tablet_bootstrap.cc:492] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50: Bootstrap starting.
I20260709 13:23:18.398367 24439 tablet_bootstrap.cc:492] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219: No bootstrap required, opened a new log
I20260709 13:23:18.398260 24437 raft_consensus.cc:359] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } }
I20260709 13:23:18.399045 24439 ts_tablet_manager.cc:1403] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219: Time spent bootstrapping tablet: real 0.048s	user 0.010s	sys 0.008s
I20260709 13:23:18.399189 24437 raft_consensus.cc:385] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.399600 24437 raft_consensus.cc:740] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.400413 24437 consensus_queue.cc:260] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } }
I20260709 13:23:18.402248 24439 raft_consensus.cc:359] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.402915 24437 ts_tablet_manager.cc:1434] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 13:23:18.403045 24439 raft_consensus.cc:385] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.403712 24436 tablet_bootstrap.cc:654] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.403635 24439 raft_consensus.cc:740] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3ac721977e3467892cbf730c5262219, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.404018 24437 tablet_bootstrap.cc:492] T 0f82955421054b95ad8df878b1827aa4 P 3aa42500bbcd4121a43413ecd5be246d: Bootstrap starting.
I20260709 13:23:18.404678 24439 consensus_queue.cc:260] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.410198 24439 ts_tablet_manager.cc:1434] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:23:18.411304 24437 tablet_bootstrap.cc:654] T 0f82955421054b95ad8df878b1827aa4 P 3aa42500bbcd4121a43413ecd5be246d: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.411809 24439 tablet_bootstrap.cc:492] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219: Bootstrap starting.
I20260709 13:23:18.415522 24436 tablet_bootstrap.cc:492] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50: No bootstrap required, opened a new log
I20260709 13:23:18.416189 24436 ts_tablet_manager.cc:1403] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.005s
I20260709 13:23:18.419269 24439 tablet_bootstrap.cc:654] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.419232 24436 raft_consensus.cc:359] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.420271 24436 raft_consensus.cc:385] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.420672 24436 raft_consensus.cc:740] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.421536 24436 consensus_queue.cc:260] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.424264 24436 ts_tablet_manager.cc:1434] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:23:18.425833 24436 tablet_bootstrap.cc:492] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50: Bootstrap starting.
I20260709 13:23:18.433040 24439 tablet_bootstrap.cc:492] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219: No bootstrap required, opened a new log
I20260709 13:23:18.433291 24436 tablet_bootstrap.cc:654] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.433732 24439 ts_tablet_manager.cc:1403] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.001s
I20260709 13:23:18.437155 24439 raft_consensus.cc:359] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.438352 24439 raft_consensus.cc:385] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:23:18.443118 24422 tablet.cc:2406] T 0713792c0cb2436b932fadc8d1f428a2 P 3aa42500bbcd4121a43413ecd5be246d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:23:18.438807 24439 raft_consensus.cc:740] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3ac721977e3467892cbf730c5262219, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.446007 24439 consensus_queue.cc:260] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.450940 24439 ts_tablet_manager.cc:1434] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:23:18.452067 24439 tablet_bootstrap.cc:492] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219: Bootstrap starting.
I20260709 13:23:18.453102 24437 tablet_bootstrap.cc:492] T 0f82955421054b95ad8df878b1827aa4 P 3aa42500bbcd4121a43413ecd5be246d: No bootstrap required, opened a new log
I20260709 13:23:18.453711 24437 ts_tablet_manager.cc:1403] T 0f82955421054b95ad8df878b1827aa4 P 3aa42500bbcd4121a43413ecd5be246d: Time spent bootstrapping tablet: real 0.050s	user 0.013s	sys 0.034s
I20260709 13:23:18.456928 24437 raft_consensus.cc:359] T 0f82955421054b95ad8df878b1827aa4 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.457719 24437 raft_consensus.cc:385] T 0f82955421054b95ad8df878b1827aa4 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.458057 24437 raft_consensus.cc:740] T 0f82955421054b95ad8df878b1827aa4 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.459131 24439 tablet_bootstrap.cc:654] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.459801 24437 consensus_queue.cc:260] T 0f82955421054b95ad8df878b1827aa4 P 3aa42500bbcd4121a43413ecd5be246d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.461472 24436 tablet_bootstrap.cc:492] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50: No bootstrap required, opened a new log
I20260709 13:23:18.462121 24436 ts_tablet_manager.cc:1403] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50: Time spent bootstrapping tablet: real 0.037s	user 0.023s	sys 0.012s
I20260709 13:23:18.462442 24437 ts_tablet_manager.cc:1434] T 0f82955421054b95ad8df878b1827aa4 P 3aa42500bbcd4121a43413ecd5be246d: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:23:18.463613 24437 tablet_bootstrap.cc:492] T 0689d527efd74dc1a43383e7f3034595 P 3aa42500bbcd4121a43413ecd5be246d: Bootstrap starting.
I20260709 13:23:18.464543 24440 raft_consensus.cc:493] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:18.465080 24440 raft_consensus.cc:515] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.471037 24437 tablet_bootstrap.cc:654] T 0689d527efd74dc1a43383e7f3034595 P 3aa42500bbcd4121a43413ecd5be246d: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.477427 24440 leader_election.cc:290] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:18.483089 24436 raft_consensus.cc:359] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.483949 24436 raft_consensus.cc:385] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.484294 24436 raft_consensus.cc:740] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.484647 24437 tablet_bootstrap.cc:492] T 0689d527efd74dc1a43383e7f3034595 P 3aa42500bbcd4121a43413ecd5be246d: No bootstrap required, opened a new log
I20260709 13:23:18.485307 24437 ts_tablet_manager.cc:1403] T 0689d527efd74dc1a43383e7f3034595 P 3aa42500bbcd4121a43413ecd5be246d: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.011s
I20260709 13:23:18.485121 24436 consensus_queue.cc:260] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.488296 24439 tablet_bootstrap.cc:492] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219: No bootstrap required, opened a new log
I20260709 13:23:18.488503 24437 raft_consensus.cc:359] T 0689d527efd74dc1a43383e7f3034595 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.489006 24439 ts_tablet_manager.cc:1403] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.006s
I20260709 13:23:18.489406 24437 raft_consensus.cc:385] T 0689d527efd74dc1a43383e7f3034595 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.489864 24437 raft_consensus.cc:740] T 0689d527efd74dc1a43383e7f3034595 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.490509 24436 ts_tablet_manager.cc:1434] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.001s
I20260709 13:23:18.490849 24437 consensus_queue.cc:260] T 0689d527efd74dc1a43383e7f3034595 P 3aa42500bbcd4121a43413ecd5be246d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.491684 24436 tablet_bootstrap.cc:492] T 05cc80ee32e0409c959205fbbcf09e58 P 66e4fe095997420ba9949bad99bf4d50: Bootstrap starting.
I20260709 13:23:18.492403 24439 raft_consensus.cc:359] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.493199 24439 raft_consensus.cc:385] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.493592 24439 raft_consensus.cc:740] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3ac721977e3467892cbf730c5262219, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.494567 24439 consensus_queue.cc:260] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.496140 24437 ts_tablet_manager.cc:1434] T 0689d527efd74dc1a43383e7f3034595 P 3aa42500bbcd4121a43413ecd5be246d: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:23:18.497643 24440 raft_consensus.cc:493] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:18.498121 24439 ts_tablet_manager.cc:1434] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:23:18.498163 24440 raft_consensus.cc:515] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.499469 24439 tablet_bootstrap.cc:492] T a91cd6890c73435e9b43afc4262d4f01 P d3ac721977e3467892cbf730c5262219: Bootstrap starting.
I20260709 13:23:18.500185 24436 tablet_bootstrap.cc:654] T 05cc80ee32e0409c959205fbbcf09e58 P 66e4fe095997420ba9949bad99bf4d50: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.500567 24440 leader_election.cc:290] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:18.503319 24437 tablet_bootstrap.cc:492] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d: Bootstrap starting.
I20260709 13:23:18.506349 24445 raft_consensus.cc:493] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:18.506880 24439 tablet_bootstrap.cc:654] T a91cd6890c73435e9b43afc4262d4f01 P d3ac721977e3467892cbf730c5262219: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.506875 24445 raft_consensus.cc:515] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } }
I20260709 13:23:18.513742 24437 tablet_bootstrap.cc:654] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.514400 24445 leader_election.cc:290] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237)
I20260709 13:23:18.514927 24322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33f74f7b25d0436fa15cc6053960e3a3" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219" is_pre_election: true
I20260709 13:23:18.519977 24322 raft_consensus.cc:2468] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 0.
I20260709 13:23:18.515872 24321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0713792c0cb2436b932fadc8d1f428a2" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219" is_pre_election: true
I20260709 13:23:18.521051 24321 raft_consensus.cc:2468] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 0.
I20260709 13:23:18.521483 24436 tablet_bootstrap.cc:492] T 05cc80ee32e0409c959205fbbcf09e58 P 66e4fe095997420ba9949bad99bf4d50: No bootstrap required, opened a new log
I20260709 13:23:18.522040 24436 ts_tablet_manager.cc:1403] T 05cc80ee32e0409c959205fbbcf09e58 P 66e4fe095997420ba9949bad99bf4d50: Time spent bootstrapping tablet: real 0.031s	user 0.008s	sys 0.004s
I20260709 13:23:18.522547 24212 leader_election.cc:304] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:18.523599 24212 leader_election.cc:304] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:18.524513 24440 raft_consensus.cc:2804] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:18.524887 24440 raft_consensus.cc:493] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:18.525213 24440 raft_consensus.cc:3060] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.525374 24436 raft_consensus.cc:359] T 05cc80ee32e0409c959205fbbcf09e58 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.526458 24436 raft_consensus.cc:385] T 05cc80ee32e0409c959205fbbcf09e58 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.526810 24436 raft_consensus.cc:740] T 05cc80ee32e0409c959205fbbcf09e58 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.527752 24436 consensus_queue.cc:260] T 05cc80ee32e0409c959205fbbcf09e58 P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.531901 24439 tablet_bootstrap.cc:492] T a91cd6890c73435e9b43afc4262d4f01 P d3ac721977e3467892cbf730c5262219: No bootstrap required, opened a new log
I20260709 13:23:18.532743 24396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33f74f7b25d0436fa15cc6053960e3a3" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d" is_pre_election: true
I20260709 13:23:18.534312 24395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0713792c0cb2436b932fadc8d1f428a2" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d" is_pre_election: true
I20260709 13:23:18.534549 24437 tablet_bootstrap.cc:492] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d: No bootstrap required, opened a new log
I20260709 13:23:18.535072 24395 raft_consensus.cc:2468] T 0713792c0cb2436b932fadc8d1f428a2 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 0.
I20260709 13:23:18.535317 24437 ts_tablet_manager.cc:1403] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d: Time spent bootstrapping tablet: real 0.032s	user 0.000s	sys 0.014s
I20260709 13:23:18.535851 24247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f2735d0953445d8b941fd6ad09e76fc" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50" is_pre_election: true
I20260709 13:23:18.536578 24247 raft_consensus.cc:2468] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 0.
W20260709 13:23:18.537863 24213 leader_election.cc:343] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:18.536149 24463 raft_consensus.cc:2804] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:18.538550 24463 raft_consensus.cc:493] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:18.537955 24436 ts_tablet_manager.cc:1434] T 05cc80ee32e0409c959205fbbcf09e58 P 66e4fe095997420ba9949bad99bf4d50: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.008s
I20260709 13:23:18.538607 24437 raft_consensus.cc:359] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.539234 24361 leader_election.cc:304] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, 66e4fe095997420ba9949bad99bf4d50; no voters: 
I20260709 13:23:18.540217 24445 raft_consensus.cc:2804] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:18.540524 24437 raft_consensus.cc:385] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.540596 24445 raft_consensus.cc:493] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:18.540927 24437 raft_consensus.cc:740] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.541085 24445 raft_consensus.cc:3060] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.541608 24440 raft_consensus.cc:515] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.541824 24437 consensus_queue.cc:260] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.543274 24436 tablet_bootstrap.cc:492] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50: Bootstrap starting.
I20260709 13:23:18.543812 24440 leader_election.cc:290] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 election: Requested vote from peers d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:18.532778 24439 ts_tablet_manager.cc:1403] T a91cd6890c73435e9b43afc4262d4f01 P d3ac721977e3467892cbf730c5262219: Time spent bootstrapping tablet: real 0.034s	user 0.001s	sys 0.013s
I20260709 13:23:18.545382 24395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33f74f7b25d0436fa15cc6053960e3a3" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
W20260709 13:23:18.547206 24213 leader_election.cc:343] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:18.548071 24439 raft_consensus.cc:359] T a91cd6890c73435e9b43afc4262d4f01 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.548498 24321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33f74f7b25d0436fa15cc6053960e3a3" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219"
I20260709 13:23:18.552731 24321 raft_consensus.cc:3060] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.556713 24436 tablet_bootstrap.cc:654] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50: Neither blocks nor log segments found. Creating new log.
W20260709 13:23:18.560235 24193 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:23:18.549983 24463 raft_consensus.cc:3060] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.560917 24321 raft_consensus.cc:2468] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 1.
I20260709 13:23:18.561859 24439 raft_consensus.cc:385] T a91cd6890c73435e9b43afc4262d4f01 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.562189 24439 raft_consensus.cc:740] T a91cd6890c73435e9b43afc4262d4f01 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3ac721977e3467892cbf730c5262219, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.563301 24212 leader_election.cc:304] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 3aa42500bbcd4121a43413ecd5be246d
I20260709 13:23:18.563081 24445 raft_consensus.cc:515] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } }
I20260709 13:23:18.564698 24462 raft_consensus.cc:2804] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:18.565203 24462 raft_consensus.cc:697] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Becoming Leader. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Running, Role: LEADER
I20260709 13:23:18.566152 24462 consensus_queue.cc:237] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.566421 24247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f2735d0953445d8b941fd6ad09e76fc" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
I20260709 13:23:18.568140 24445 leader_election.cc:290] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 election: Requested vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237)
I20260709 13:23:18.568235 24247 raft_consensus.cc:3060] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.566414 24439 consensus_queue.cc:260] T a91cd6890c73435e9b43afc4262d4f01 P d3ac721977e3467892cbf730c5262219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.573236 24439 ts_tablet_manager.cc:1434] T a91cd6890c73435e9b43afc4262d4f01 P d3ac721977e3467892cbf730c5262219: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.001s
I20260709 13:23:18.574674 24439 tablet_bootstrap.cc:492] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219: Bootstrap starting.
I20260709 13:23:18.584676 24437 ts_tablet_manager.cc:1434] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d: Time spent starting tablet: real 0.049s	user 0.017s	sys 0.018s
I20260709 13:23:18.589166 24437 tablet_bootstrap.cc:492] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d: Bootstrap starting.
I20260709 13:23:18.592341 24322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f2735d0953445d8b941fd6ad09e76fc" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219"
I20260709 13:23:18.593075 24322 raft_consensus.cc:3060] T 7f2735d0953445d8b941fd6ad09e76fc P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.594290 24247 raft_consensus.cc:2468] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 1.
I20260709 13:23:18.596278 24439 tablet_bootstrap.cc:654] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.596171 24321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f2735d0953445d8b941fd6ad09e76fc" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219" is_pre_election: true
I20260709 13:23:18.596692 24437 tablet_bootstrap.cc:654] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.598546 24361 leader_election.cc:304] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, 66e4fe095997420ba9949bad99bf4d50; no voters: 
I20260709 13:23:18.600025 24445 raft_consensus.cc:2804] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:18.600571 24445 raft_consensus.cc:697] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [term 1 LEADER]: Becoming Leader. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Running, Role: LEADER
I20260709 13:23:18.601436 24445 consensus_queue.cc:237] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } }
I20260709 13:23:18.602113 24144 catalog_manager.cc:5697] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "66e4fe095997420ba9949bad99bf4d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:18.605275 24463 raft_consensus.cc:515] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.612668 24463 leader_election.cc:290] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 election: Requested vote from peers d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:18.613677 24321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0713792c0cb2436b932fadc8d1f428a2" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219"
I20260709 13:23:18.614588 24321 raft_consensus.cc:3060] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.615008 24395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0713792c0cb2436b932fadc8d1f428a2" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
I20260709 13:23:18.615826 24395 raft_consensus.cc:3060] T 0713792c0cb2436b932fadc8d1f428a2 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.624369 24436 tablet_bootstrap.cc:492] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50: No bootstrap required, opened a new log
I20260709 13:23:18.625247 24436 ts_tablet_manager.cc:1403] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50: Time spent bootstrapping tablet: real 0.082s	user 0.012s	sys 0.004s
I20260709 13:23:18.625672 24321 raft_consensus.cc:2468] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 1.
I20260709 13:23:18.627601 24212 leader_election.cc:304] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:18.629094 24463 raft_consensus.cc:2804] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:18.630111 24463 raft_consensus.cc:697] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Becoming Leader. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Running, Role: LEADER
I20260709 13:23:18.631390 24463 consensus_queue.cc:237] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.633378 24322 raft_consensus.cc:2468] T 7f2735d0953445d8b941fd6ad09e76fc P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 1.
I20260709 13:23:18.629360 24436 raft_consensus.cc:359] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.635965 24436 raft_consensus.cc:385] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.636154 24439 tablet_bootstrap.cc:492] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219: No bootstrap required, opened a new log
I20260709 13:23:18.636478 24436 raft_consensus.cc:740] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.641506 24437 tablet_bootstrap.cc:492] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d: No bootstrap required, opened a new log
I20260709 13:23:18.644086 24437 ts_tablet_manager.cc:1403] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d: Time spent bootstrapping tablet: real 0.056s	user 0.031s	sys 0.012s
I20260709 13:23:18.644604 24439 ts_tablet_manager.cc:1403] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219: Time spent bootstrapping tablet: real 0.070s	user 0.008s	sys 0.014s
I20260709 13:23:18.642673 24436 consensus_queue.cc:260] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.645846 24395 raft_consensus.cc:2468] T 0713792c0cb2436b932fadc8d1f428a2 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 1.
I20260709 13:23:18.647579 24436 ts_tablet_manager.cc:1434] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.001s
I20260709 13:23:18.648672 24436 tablet_bootstrap.cc:492] T 32b70a25b0264a8a96170bbecee4837d P 66e4fe095997420ba9949bad99bf4d50: Bootstrap starting.
I20260709 13:23:18.648371 24437 raft_consensus.cc:359] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.649138 24437 raft_consensus.cc:385] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.649449 24437 raft_consensus.cc:740] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.653208 24437 consensus_queue.cc:260] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.655025 24437 ts_tablet_manager.cc:1434] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:23:18.655872 24437 tablet_bootstrap.cc:492] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d: Bootstrap starting.
I20260709 13:23:18.657222 24436 tablet_bootstrap.cc:654] T 32b70a25b0264a8a96170bbecee4837d P 66e4fe095997420ba9949bad99bf4d50: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.656467 24145 catalog_manager.cc:5697] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67). New cstate: current_term: 1 leader_uuid: "3aa42500bbcd4121a43413ecd5be246d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:18.659704 24439 raft_consensus.cc:359] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.664294 24437 tablet_bootstrap.cc:654] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.664740 24439 raft_consensus.cc:385] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.665299 24439 raft_consensus.cc:740] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3ac721977e3467892cbf730c5262219, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.666407 24439 consensus_queue.cc:260] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.667335 24444 raft_consensus.cc:493] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:18.668007 24444 raft_consensus.cc:515] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.669371 24436 tablet_bootstrap.cc:492] T 32b70a25b0264a8a96170bbecee4837d P 66e4fe095997420ba9949bad99bf4d50: No bootstrap required, opened a new log
I20260709 13:23:18.670058 24436 ts_tablet_manager.cc:1403] T 32b70a25b0264a8a96170bbecee4837d P 66e4fe095997420ba9949bad99bf4d50: Time spent bootstrapping tablet: real 0.022s	user 0.008s	sys 0.011s
I20260709 13:23:18.676846 24444 leader_election.cc:290] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:18.673449 24436 raft_consensus.cc:359] T 32b70a25b0264a8a96170bbecee4837d P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.682515 24436 raft_consensus.cc:385] T 32b70a25b0264a8a96170bbecee4837d P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.682864 24436 raft_consensus.cc:740] T 32b70a25b0264a8a96170bbecee4837d P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.683809 24436 consensus_queue.cc:260] T 32b70a25b0264a8a96170bbecee4837d P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.686542 24436 ts_tablet_manager.cc:1434] T 32b70a25b0264a8a96170bbecee4837d P 66e4fe095997420ba9949bad99bf4d50: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:23:18.687718 24436 tablet_bootstrap.cc:492] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50: Bootstrap starting.
I20260709 13:23:18.695516 24436 tablet_bootstrap.cc:654] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.708051 24436 tablet_bootstrap.cc:492] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50: No bootstrap required, opened a new log
I20260709 13:23:18.708635 24436 ts_tablet_manager.cc:1403] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50: Time spent bootstrapping tablet: real 0.021s	user 0.018s	sys 0.001s
I20260709 13:23:18.711529 24436 raft_consensus.cc:359] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.712350 24436 raft_consensus.cc:385] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.712687 24436 raft_consensus.cc:740] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.713375 24436 consensus_queue.cc:260] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.715298 24437 tablet_bootstrap.cc:492] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d: No bootstrap required, opened a new log
I20260709 13:23:18.715806 24436 ts_tablet_manager.cc:1434] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:23:18.715973 24437 ts_tablet_manager.cc:1403] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d: Time spent bootstrapping tablet: real 0.060s	user 0.009s	sys 0.007s
I20260709 13:23:18.715444 24143 catalog_manager.cc:5697] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "66e4fe095997420ba9949bad99bf4d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:18.718437 24247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0689d527efd74dc1a43383e7f3034595" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50" is_pre_election: true
I20260709 13:23:18.719126 24247 raft_consensus.cc:2468] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 0.
I20260709 13:23:18.719218 24437 raft_consensus.cc:359] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.724097 24437 raft_consensus.cc:385] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.724433 24437 raft_consensus.cc:740] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.720086 24439 ts_tablet_manager.cc:1434] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219: Time spent starting tablet: real 0.070s	user 0.015s	sys 0.023s
I20260709 13:23:18.720683 24287 leader_election.cc:304] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:18.721742 24463 raft_consensus.cc:493] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:18.727212 24439 tablet_bootstrap.cc:492] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219: Bootstrap starting.
I20260709 13:23:18.726980 24463 raft_consensus.cc:515] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.728827 24463 leader_election.cc:290] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:18.729764 24322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a91cd6890c73435e9b43afc4262d4f01" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219" is_pre_election: true
I20260709 13:23:18.730481 24322 raft_consensus.cc:2468] T a91cd6890c73435e9b43afc4262d4f01 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 0.
I20260709 13:23:18.732023 24212 leader_election.cc:304] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:18.732962 24463 raft_consensus.cc:2804] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:18.733369 24463 raft_consensus.cc:493] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:18.725216 24395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0689d527efd74dc1a43383e7f3034595" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d" is_pre_election: true
I20260709 13:23:18.733744 24463 raft_consensus.cc:3060] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.733994 24439 tablet_bootstrap.cc:654] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.734002 24395 raft_consensus.cc:2468] T 0689d527efd74dc1a43383e7f3034595 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 0.
I20260709 13:23:18.733109 24396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a91cd6890c73435e9b43afc4262d4f01" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d" is_pre_election: true
I20260709 13:23:18.736764 24476 raft_consensus.cc:2804] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:18.737231 24476 raft_consensus.cc:493] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:18.737586 24476 raft_consensus.cc:3060] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.740584 24445 raft_consensus.cc:493] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:18.741130 24445 raft_consensus.cc:515] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.743340 24445 leader_election.cc:290] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237)
I20260709 13:23:18.725169 24437 consensus_queue.cc:260] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.745239 24322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33f74f7b25d0436fa15cc6053960e3a3" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219" is_pre_election: true
I20260709 13:23:18.746096 24322 raft_consensus.cc:2393] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3aa42500bbcd4121a43413ecd5be246d in current term 1: Already voted for candidate 66e4fe095997420ba9949bad99bf4d50 in this term.
I20260709 13:23:18.746347 24396 raft_consensus.cc:2468] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 0.
I20260709 13:23:18.746590 24247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33f74f7b25d0436fa15cc6053960e3a3" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50" is_pre_election: true
I20260709 13:23:18.751582 24361 leader_election.cc:304] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d; no voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219
I20260709 13:23:18.754792 24463 raft_consensus.cc:515] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.755357 24464 raft_consensus.cc:3060] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.745530 24476 raft_consensus.cc:515] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.758988 24247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0689d527efd74dc1a43383e7f3034595" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
I20260709 13:23:18.759603 24247 raft_consensus.cc:3060] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.767485 24464 raft_consensus.cc:2749] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:18.767769 24247 raft_consensus.cc:2468] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 1.
I20260709 13:23:18.768340 24437 ts_tablet_manager.cc:1434] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.007s
I20260709 13:23:18.771348 24322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a91cd6890c73435e9b43afc4262d4f01" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219"
I20260709 13:23:18.771997 24322 raft_consensus.cc:3060] T a91cd6890c73435e9b43afc4262d4f01 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.772126 24439 tablet_bootstrap.cc:492] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219: No bootstrap required, opened a new log
I20260709 13:23:18.772688 24439 ts_tablet_manager.cc:1403] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219: Time spent bootstrapping tablet: real 0.046s	user 0.019s	sys 0.024s
I20260709 13:23:18.774363 24437 tablet_bootstrap.cc:492] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d: Bootstrap starting.
I20260709 13:23:18.775621 24439 raft_consensus.cc:359] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.776352 24439 raft_consensus.cc:385] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.776762 24439 raft_consensus.cc:740] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3ac721977e3467892cbf730c5262219, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.777895 24395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a91cd6890c73435e9b43afc4262d4f01" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
I20260709 13:23:18.778462 24395 raft_consensus.cc:3060] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.769065 24476 leader_election.cc:290] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 1 election: Requested vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:18.771082 24463 leader_election.cc:290] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 election: Requested vote from peers d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:18.777530 24439 consensus_queue.cc:260] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.780076 24287 leader_election.cc:304] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:18.780934 24476 raft_consensus.cc:2804] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:18.781061 24439 ts_tablet_manager.cc:1434] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:23:18.781850 24476 raft_consensus.cc:697] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [term 1 LEADER]: Becoming Leader. State: Replica: d3ac721977e3467892cbf730c5262219, State: Running, Role: LEADER
I20260709 13:23:18.782399 24439 tablet_bootstrap.cc:492] T 32b70a25b0264a8a96170bbecee4837d P d3ac721977e3467892cbf730c5262219: Bootstrap starting.
I20260709 13:23:18.782711 24476 consensus_queue.cc:237] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.770623 24396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0689d527efd74dc1a43383e7f3034595" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
I20260709 13:23:18.784334 24396 raft_consensus.cc:3060] T 0689d527efd74dc1a43383e7f3034595 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.792133 24322 raft_consensus.cc:2468] T a91cd6890c73435e9b43afc4262d4f01 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 1.
I20260709 13:23:18.793396 24212 leader_election.cc:304] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:18.794086 24437 tablet_bootstrap.cc:654] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.794258 24463 raft_consensus.cc:2804] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:18.794445 24464 raft_consensus.cc:493] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:18.794857 24463 raft_consensus.cc:697] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Becoming Leader. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Running, Role: LEADER
I20260709 13:23:18.795002 24464 raft_consensus.cc:515] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.795720 24463 consensus_queue.cc:237] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.797976 24464 leader_election.cc:290] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237)
I20260709 13:23:18.798261 24247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05cc80ee32e0409c959205fbbcf09e58" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50" is_pre_election: true
I20260709 13:23:18.798956 24247 raft_consensus.cc:2468] T 05cc80ee32e0409c959205fbbcf09e58 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 0.
I20260709 13:23:18.799743 24322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05cc80ee32e0409c959205fbbcf09e58" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219" is_pre_election: true
I20260709 13:23:18.800122 24361 leader_election.cc:304] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, 66e4fe095997420ba9949bad99bf4d50; no voters: 
I20260709 13:23:18.800498 24444 raft_consensus.cc:493] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:18.801028 24444 raft_consensus.cc:515] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.801266 24464 raft_consensus.cc:2804] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:18.801864 24464 raft_consensus.cc:493] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:18.802294 24464 raft_consensus.cc:3060] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.802947 24395 raft_consensus.cc:2468] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 1.
I20260709 13:23:18.803184 24444 leader_election.cc:290] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:18.806082 24322 raft_consensus.cc:2468] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 0.
I20260709 13:23:18.806644 24437 tablet_bootstrap.cc:492] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d: No bootstrap required, opened a new log
I20260709 13:23:18.807210 24437 ts_tablet_manager.cc:1403] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.008s
I20260709 13:23:18.807593 24439 tablet_bootstrap.cc:654] T 32b70a25b0264a8a96170bbecee4837d P d3ac721977e3467892cbf730c5262219: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.810269 24437 raft_consensus.cc:359] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.811769 24437 raft_consensus.cc:385] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.812142 24437 raft_consensus.cc:740] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.812017 24396 raft_consensus.cc:2468] T 0689d527efd74dc1a43383e7f3034595 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 1.
I20260709 13:23:18.806681 24247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f82955421054b95ad8df878b1827aa4" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50" is_pre_election: true
I20260709 13:23:18.813874 24247 raft_consensus.cc:2468] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 0.
I20260709 13:23:18.815133 24287 leader_election.cc:304] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:18.816349 24476 raft_consensus.cc:2804] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:18.816725 24476 raft_consensus.cc:493] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:18.817199 24476 raft_consensus.cc:3060] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.816558 24437 consensus_queue.cc:260] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.817781 24445 raft_consensus.cc:493] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:18.817893 24439 tablet_bootstrap.cc:492] T 32b70a25b0264a8a96170bbecee4837d P d3ac721977e3467892cbf730c5262219: No bootstrap required, opened a new log
I20260709 13:23:18.818464 24439 ts_tablet_manager.cc:1403] T 32b70a25b0264a8a96170bbecee4837d P d3ac721977e3467892cbf730c5262219: Time spent bootstrapping tablet: real 0.036s	user 0.010s	sys 0.002s
I20260709 13:23:18.818259 24445 raft_consensus.cc:515] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.819545 24145 catalog_manager.cc:5697] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3ac721977e3467892cbf730c5262219 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "d3ac721977e3467892cbf730c5262219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:18.821298 24439 raft_consensus.cc:359] T 32b70a25b0264a8a96170bbecee4837d P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.822046 24439 raft_consensus.cc:385] T 32b70a25b0264a8a96170bbecee4837d P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.822291 24439 raft_consensus.cc:740] T 32b70a25b0264a8a96170bbecee4837d P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3ac721977e3467892cbf730c5262219, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.822558 24445 leader_election.cc:290] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237)
I20260709 13:23:18.823107 24439 consensus_queue.cc:260] T 32b70a25b0264a8a96170bbecee4837d P d3ac721977e3467892cbf730c5262219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.824235 24322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b70a25b0264a8a96170bbecee4837d" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219" is_pre_election: true
I20260709 13:23:18.824047 24143 catalog_manager.cc:5697] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "66e4fe095997420ba9949bad99bf4d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:18.824962 24322 raft_consensus.cc:2468] T 32b70a25b0264a8a96170bbecee4837d P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 0.
I20260709 13:23:18.825886 24439 ts_tablet_manager.cc:1434] T 32b70a25b0264a8a96170bbecee4837d P d3ac721977e3467892cbf730c5262219: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:23:18.827111 24439 tablet_bootstrap.cc:492] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219: Bootstrap starting.
I20260709 13:23:18.827464 24361 leader_election.cc:304] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:18.829084 24247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b70a25b0264a8a96170bbecee4837d" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50" is_pre_election: true
I20260709 13:23:18.829799 24247 raft_consensus.cc:2468] T 32b70a25b0264a8a96170bbecee4837d P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 0.
I20260709 13:23:18.832682 24445 raft_consensus.cc:2804] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:18.833300 24445 raft_consensus.cc:493] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:18.833770 24445 raft_consensus.cc:3060] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.834506 24439 tablet_bootstrap.cc:654] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.838173 24476 raft_consensus.cc:515] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.841588 24437 ts_tablet_manager.cc:1434] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d: Time spent starting tablet: real 0.034s	user 0.003s	sys 0.003s
I20260709 13:23:18.843335 24437 tablet_bootstrap.cc:492] T b027fa0920c8440fab0837dd1142fd57 P 3aa42500bbcd4121a43413ecd5be246d: Bootstrap starting.
I20260709 13:23:18.846092 24439 tablet_bootstrap.cc:492] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219: No bootstrap required, opened a new log
I20260709 13:23:18.846747 24439 ts_tablet_manager.cc:1403] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.007s
I20260709 13:23:18.846706 24247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f82955421054b95ad8df878b1827aa4" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
I20260709 13:23:18.847528 24247 raft_consensus.cc:3060] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.847399 24396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f82955421054b95ad8df878b1827aa4" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
I20260709 13:23:18.848083 24396 raft_consensus.cc:3060] T 0f82955421054b95ad8df878b1827aa4 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.849401 24445 raft_consensus.cc:515] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.857869 24246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b70a25b0264a8a96170bbecee4837d" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
I20260709 13:23:18.858454 24437 tablet_bootstrap.cc:654] T b027fa0920c8440fab0837dd1142fd57 P 3aa42500bbcd4121a43413ecd5be246d: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:18.858590 24246 raft_consensus.cc:3060] T 32b70a25b0264a8a96170bbecee4837d P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.859911 24396 raft_consensus.cc:2468] T 0f82955421054b95ad8df878b1827aa4 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 1.
I20260709 13:23:18.861979 24395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f82955421054b95ad8df878b1827aa4" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d" is_pre_election: true
I20260709 13:23:18.862736 24288 leader_election.cc:304] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:18.862977 24476 leader_election.cc:290] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 1 election: Requested vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:18.863498 24464 raft_consensus.cc:515] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.864702 24444 raft_consensus.cc:2804] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:18.865227 24247 raft_consensus.cc:2468] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 1.
I20260709 13:23:18.862502 24439 raft_consensus.cc:359] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.866997 24439 raft_consensus.cc:385] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.867761 24464 leader_election.cc:290] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 election: Requested vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237)
I20260709 13:23:18.868610 24247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05cc80ee32e0409c959205fbbcf09e58" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
I20260709 13:23:18.868634 24322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05cc80ee32e0409c959205fbbcf09e58" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219"
I20260709 13:23:18.869525 24247 raft_consensus.cc:3060] T 05cc80ee32e0409c959205fbbcf09e58 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.869632 24322 raft_consensus.cc:3060] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.874362 24246 raft_consensus.cc:2468] T 32b70a25b0264a8a96170bbecee4837d P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 1.
I20260709 13:23:18.875758 24361 leader_election.cc:304] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, 66e4fe095997420ba9949bad99bf4d50; no voters: 
I20260709 13:23:18.876613 24464 raft_consensus.cc:2804] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:18.877108 24464 raft_consensus.cc:697] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d [term 1 LEADER]: Becoming Leader. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Running, Role: LEADER
I20260709 13:23:18.877662 24445 leader_election.cc:290] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 election: Requested vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237)
I20260709 13:23:18.870085 24444 raft_consensus.cc:697] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [term 1 LEADER]: Becoming Leader. State: Replica: d3ac721977e3467892cbf730c5262219, State: Running, Role: LEADER
I20260709 13:23:18.878017 24464 consensus_queue.cc:237] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.878553 24321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b70a25b0264a8a96170bbecee4837d" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219"
I20260709 13:23:18.879457 24321 raft_consensus.cc:3060] T 32b70a25b0264a8a96170bbecee4837d P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.862814 24395 raft_consensus.cc:2376] T 0f82955421054b95ad8df878b1827aa4 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 1. Re-sending same reply.
I20260709 13:23:18.879211 24444 consensus_queue.cc:237] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.885293 24247 raft_consensus.cc:2468] T 05cc80ee32e0409c959205fbbcf09e58 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 1.
I20260709 13:23:18.886893 24361 leader_election.cc:304] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, 66e4fe095997420ba9949bad99bf4d50; no voters: 
I20260709 13:23:18.887924 24464 raft_consensus.cc:2804] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:18.888676 24464 raft_consensus.cc:697] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [term 1 LEADER]: Becoming Leader. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Running, Role: LEADER
I20260709 13:23:18.890060 24321 raft_consensus.cc:2468] T 32b70a25b0264a8a96170bbecee4837d P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 1.
I20260709 13:23:18.889577 24464 consensus_queue.cc:237] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.891444 24143 catalog_manager.cc:5697] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67). New cstate: current_term: 1 leader_uuid: "3aa42500bbcd4121a43413ecd5be246d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:18.869746 24439 raft_consensus.cc:740] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3ac721977e3467892cbf730c5262219, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.894966 24439 consensus_queue.cc:260] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.899582 24437 tablet_bootstrap.cc:492] T b027fa0920c8440fab0837dd1142fd57 P 3aa42500bbcd4121a43413ecd5be246d: No bootstrap required, opened a new log
I20260709 13:23:18.901118 24437 ts_tablet_manager.cc:1403] T b027fa0920c8440fab0837dd1142fd57 P 3aa42500bbcd4121a43413ecd5be246d: Time spent bootstrapping tablet: real 0.058s	user 0.018s	sys 0.016s
I20260709 13:23:18.903156 24322 raft_consensus.cc:2468] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 1.
I20260709 13:23:18.902834 24145 catalog_manager.cc:5697] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3ac721977e3467892cbf730c5262219 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "d3ac721977e3467892cbf730c5262219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:18.903949 24437 raft_consensus.cc:359] T b027fa0920c8440fab0837dd1142fd57 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.904631 24437 raft_consensus.cc:385] T b027fa0920c8440fab0837dd1142fd57 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:18.904897 24437 raft_consensus.cc:740] T b027fa0920c8440fab0837dd1142fd57 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Initialized, Role: FOLLOWER
I20260709 13:23:18.897861 24439 ts_tablet_manager.cc:1434] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.000s
I20260709 13:23:18.905602 24437 consensus_queue.cc:260] T b027fa0920c8440fab0837dd1142fd57 P 3aa42500bbcd4121a43413ecd5be246d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.907641 24437 ts_tablet_manager.cc:1434] T b027fa0920c8440fab0837dd1142fd57 P 3aa42500bbcd4121a43413ecd5be246d: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 13:23:18.914669 24145 catalog_manager.cc:5697] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67). New cstate: current_term: 1 leader_uuid: "3aa42500bbcd4121a43413ecd5be246d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:18.932883 24463 raft_consensus.cc:493] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:18.933476 24463 raft_consensus.cc:515] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.935350 24463 leader_election.cc:290] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:18.936203 24322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b027fa0920c8440fab0837dd1142fd57" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219" is_pre_election: true
I20260709 13:23:18.936412 24395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b027fa0920c8440fab0837dd1142fd57" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d" is_pre_election: true
I20260709 13:23:18.937080 24322 raft_consensus.cc:2468] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 0.
I20260709 13:23:18.937183 24395 raft_consensus.cc:2468] T b027fa0920c8440fab0837dd1142fd57 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 0.
I20260709 13:23:18.938133 24212 leader_election.cc:304] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:18.938974 24463 raft_consensus.cc:2804] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:18.939246 24463 raft_consensus.cc:493] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:18.939473 24463 raft_consensus.cc:3060] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.944553 24463 raft_consensus.cc:515] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.946400 24463 leader_election.cc:290] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 election: Requested vote from peers d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:18.947182 24322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b027fa0920c8440fab0837dd1142fd57" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219"
I20260709 13:23:18.947356 24395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b027fa0920c8440fab0837dd1142fd57" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
I20260709 13:23:18.948010 24395 raft_consensus.cc:3060] T b027fa0920c8440fab0837dd1142fd57 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.948515 24322 raft_consensus.cc:3060] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:18.953651 24322 raft_consensus.cc:2468] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 1.
I20260709 13:23:18.954317 24395 raft_consensus.cc:2468] T b027fa0920c8440fab0837dd1142fd57 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 1.
I20260709 13:23:18.954686 24212 leader_election.cc:304] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:18.955518 24463 raft_consensus.cc:2804] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:18.955989 24463 raft_consensus.cc:697] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Becoming Leader. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Running, Role: LEADER
I20260709 13:23:18.956696 24463 consensus_queue.cc:237] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:18.963913 24143 catalog_manager.cc:5697] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "66e4fe095997420ba9949bad99bf4d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:19.009188 24464 consensus_queue.cc:1048] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [LEADER]: Connected to new peer: Peer: permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:19.017311 24395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f2735d0953445d8b941fd6ad09e76fc"
dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
mode: GRACEFUL
new_leader_uuid: "66e4fe095997420ba9949bad99bf4d50"
 from {username='slave'} at 127.0.0.1:56354
I20260709 13:23:19.018133 24395 raft_consensus.cc:606] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [term 1 LEADER]: Received request to transfer leadership to TS 66e4fe095997420ba9949bad99bf4d50
I20260709 13:23:19.022447 24486 consensus_queue.cc:1048] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:19.026248 24464 raft_consensus.cc:993] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d: : Instructing follower 66e4fe095997420ba9949bad99bf4d50 to start an election
I20260709 13:23:19.026706 24464 raft_consensus.cc:1081] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [term 1 LEADER]: Signalling peer 66e4fe095997420ba9949bad99bf4d50 to start an election
I20260709 13:23:19.030555 24247 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7f2735d0953445d8b941fd6ad09e76fc"
dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
 from {username='slave'} at 127.0.0.1:56886
I20260709 13:23:19.031255 24247 raft_consensus.cc:493] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:19.031699 24247 raft_consensus.cc:3060] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:19.038587 24247 raft_consensus.cc:515] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } }
I20260709 13:23:19.041420 24395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f2735d0953445d8b941fd6ad09e76fc" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
I20260709 13:23:19.041623 24247 leader_election.cc:290] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 2 election: Requested vote from peers 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619), d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237)
I20260709 13:23:19.042083 24395 raft_consensus.cc:3055] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:19.042496 24395 raft_consensus.cc:740] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Running, Role: LEADER
I20260709 13:23:19.042575 24321 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0f82955421054b95ad8df878b1827aa4"
dest_uuid: "d3ac721977e3467892cbf730c5262219"
mode: GRACEFUL
new_leader_uuid: "66e4fe095997420ba9949bad99bf4d50"
 from {username='slave'} at 127.0.0.1:48144
I20260709 13:23:19.042339 24322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f2735d0953445d8b941fd6ad09e76fc" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3ac721977e3467892cbf730c5262219"
I20260709 13:23:19.043351 24321 raft_consensus.cc:606] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [term 1 LEADER]: Received request to transfer leadership to TS 66e4fe095997420ba9949bad99bf4d50
I20260709 13:23:19.043543 24322 raft_consensus.cc:3060] T 7f2735d0953445d8b941fd6ad09e76fc P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:19.043471 24395 consensus_queue.cc:260] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } }
I20260709 13:23:19.045015 24395 raft_consensus.cc:3060] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:19.051462 24322 raft_consensus.cc:2468] T 7f2735d0953445d8b941fd6ad09e76fc P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 2.
I20260709 13:23:19.052675 24212 leader_election.cc:304] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:19.053428 24463 raft_consensus.cc:2804] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:19.053893 24462 consensus_queue.cc:1048] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:19.054565 24395 raft_consensus.cc:2468] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 2.
I20260709 13:23:19.055157 24321 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0689d527efd74dc1a43383e7f3034595"
dest_uuid: "d3ac721977e3467892cbf730c5262219"
mode: GRACEFUL
new_leader_uuid: "66e4fe095997420ba9949bad99bf4d50"
 from {username='slave'} at 127.0.0.1:48144
I20260709 13:23:19.053982 24463 raft_consensus.cc:697] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50 [term 2 LEADER]: Becoming Leader. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Running, Role: LEADER
I20260709 13:23:19.055891 24321 raft_consensus.cc:606] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [term 1 LEADER]: Received request to transfer leadership to TS 66e4fe095997420ba9949bad99bf4d50
I20260709 13:23:19.056753 24463 consensus_queue.cc:237] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } }
I20260709 13:23:19.064904 24463 consensus_queue.cc:1048] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:19.069167 24395 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05cc80ee32e0409c959205fbbcf09e58"
dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
mode: GRACEFUL
new_leader_uuid: "d3ac721977e3467892cbf730c5262219"
 from {username='slave'} at 127.0.0.1:56354
I20260709 13:23:19.070142 24395 raft_consensus.cc:606] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [term 1 LEADER]: Received request to transfer leadership to TS d3ac721977e3467892cbf730c5262219
I20260709 13:23:19.070209 24145 catalog_manager.cc:5697] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50 reported cstate change: term changed from 1 to 2, leader changed from 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67) to 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65). New cstate: current_term: 2 leader_uuid: "66e4fe095997420ba9949bad99bf4d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:19.084337 24440 consensus_queue.cc:1048] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:19.091575 24247 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0713792c0cb2436b932fadc8d1f428a2"
dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
mode: GRACEFUL
new_leader_uuid: "d3ac721977e3467892cbf730c5262219"
 from {username='slave'} at 127.0.0.1:56902
I20260709 13:23:19.092412 24247 raft_consensus.cc:606] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Received request to transfer leadership to TS d3ac721977e3467892cbf730c5262219
I20260709 13:23:19.102960 24463 raft_consensus.cc:993] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50: : Instructing follower d3ac721977e3467892cbf730c5262219 to start an election
I20260709 13:23:19.103098 24247 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "33f74f7b25d0436fa15cc6053960e3a3"
dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
mode: GRACEFUL
new_leader_uuid: "d3ac721977e3467892cbf730c5262219"
 from {username='slave'} at 127.0.0.1:56902
I20260709 13:23:19.103453 24463 raft_consensus.cc:1081] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Signalling peer d3ac721977e3467892cbf730c5262219 to start an election
I20260709 13:23:19.103809 24247 raft_consensus.cc:606] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Received request to transfer leadership to TS d3ac721977e3467892cbf730c5262219
I20260709 13:23:19.106132 24320 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0713792c0cb2436b932fadc8d1f428a2"
dest_uuid: "d3ac721977e3467892cbf730c5262219"
 from {username='slave'} at 127.0.0.1:48128
I20260709 13:23:19.106747 24320 raft_consensus.cc:493] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:19.107139 24320 raft_consensus.cc:3060] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:19.110850 24247 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b027fa0920c8440fab0837dd1142fd57"
dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
mode: GRACEFUL
new_leader_uuid: "d3ac721977e3467892cbf730c5262219"
 from {username='slave'} at 127.0.0.1:56902
I20260709 13:23:19.111898 24247 raft_consensus.cc:606] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Received request to transfer leadership to TS d3ac721977e3467892cbf730c5262219
I20260709 13:23:19.114526 24462 consensus_queue.cc:1048] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:19.121681 24320 raft_consensus.cc:515] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:19.124737 24247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0713792c0cb2436b932fadc8d1f428a2" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
I20260709 13:23:19.125299 24247 raft_consensus.cc:3055] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:19.125655 24247 raft_consensus.cc:740] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Running, Role: LEADER
I20260709 13:23:19.125790 24395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0713792c0cb2436b932fadc8d1f428a2" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
I20260709 13:23:19.126432 24395 raft_consensus.cc:3060] T 0713792c0cb2436b932fadc8d1f428a2 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:19.126502 24247 consensus_queue.cc:260] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:19.127384 24247 raft_consensus.cc:3060] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:19.132580 24320 leader_election.cc:290] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 2 election: Requested vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:19.132552 24247 raft_consensus.cc:2468] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 2.
I20260709 13:23:19.134886 24287 leader_election.cc:304] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:19.135703 24444 raft_consensus.cc:2804] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:19.136014 24395 raft_consensus.cc:2468] T 0713792c0cb2436b932fadc8d1f428a2 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 2.
I20260709 13:23:19.136209 24444 raft_consensus.cc:697] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219 [term 2 LEADER]: Becoming Leader. State: Replica: d3ac721977e3467892cbf730c5262219, State: Running, Role: LEADER
I20260709 13:23:19.137101 24444 consensus_queue.cc:237] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:19.143905 24144 catalog_manager.cc:5697] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219 reported cstate change: term changed from 1 to 2, leader changed from 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65) to d3ac721977e3467892cbf730c5262219 (127.11.63.66). New cstate: current_term: 2 leader_uuid: "d3ac721977e3467892cbf730c5262219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:19.192643 24440 consensus_queue.cc:1048] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:19.205253 24462 consensus_queue.cc:1048] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:19.313269 24444 consensus_queue.cc:1048] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:19.325227 24489 consensus_queue.cc:1048] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:19.327461 24486 consensus_queue.cc:1048] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [LEADER]: Connected to new peer: Peer: permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:19.336273 24489 raft_consensus.cc:993] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219: : Instructing follower 66e4fe095997420ba9949bad99bf4d50 to start an election
I20260709 13:23:19.336767 24444 raft_consensus.cc:1081] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [term 1 LEADER]: Signalling peer 66e4fe095997420ba9949bad99bf4d50 to start an election
I20260709 13:23:19.340790 24247 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0f82955421054b95ad8df878b1827aa4"
dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
 from {username='slave'} at 127.0.0.1:56898
I20260709 13:23:19.341365 24247 raft_consensus.cc:493] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:19.341791 24247 raft_consensus.cc:3060] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:19.342447 24486 consensus_queue.cc:1048] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:19.348003 24464 consensus_queue.cc:1048] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:19.351516 24247 raft_consensus.cc:515] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:19.357925 24486 raft_consensus.cc:993] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d: : Instructing follower d3ac721977e3467892cbf730c5262219 to start an election
I20260709 13:23:19.358417 24464 raft_consensus.cc:1081] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [term 1 LEADER]: Signalling peer d3ac721977e3467892cbf730c5262219 to start an election
I20260709 13:23:19.358803 24247 leader_election.cc:290] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 2 election: Requested vote from peers d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:19.359939 24320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f82955421054b95ad8df878b1827aa4" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3ac721977e3467892cbf730c5262219"
I20260709 13:23:19.360585 24320 raft_consensus.cc:3055] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:19.360908 24320 raft_consensus.cc:740] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3ac721977e3467892cbf730c5262219, State: Running, Role: LEADER
I20260709 13:23:19.361255 24395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f82955421054b95ad8df878b1827aa4" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
I20260709 13:23:19.361914 24395 raft_consensus.cc:3060] T 0f82955421054b95ad8df878b1827aa4 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:19.361898 24320 consensus_queue.cc:260] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:19.363679 24321 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "05cc80ee32e0409c959205fbbcf09e58"
dest_uuid: "d3ac721977e3467892cbf730c5262219"
 from {username='slave'} at 127.0.0.1:48130
I20260709 13:23:19.371008 24320 raft_consensus.cc:3060] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:19.371027 24321 raft_consensus.cc:493] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:19.371747 24321 raft_consensus.cc:3060] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:19.369838 24395 raft_consensus.cc:2468] T 0f82955421054b95ad8df878b1827aa4 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 2.
I20260709 13:23:19.375149 24213 leader_election.cc:304] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, 66e4fe095997420ba9949bad99bf4d50; no voters: 
I20260709 13:23:19.376121 24462 raft_consensus.cc:2804] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:19.376591 24462 raft_consensus.cc:697] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50 [term 2 LEADER]: Becoming Leader. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Running, Role: LEADER
I20260709 13:23:19.377434 24462 consensus_queue.cc:237] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:19.379918 24320 raft_consensus.cc:2468] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 2.
I20260709 13:23:19.381083 24509 consensus_queue.cc:1048] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d [LEADER]: Connected to new peer: Peer: permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:19.382766 24321 raft_consensus.cc:515] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:19.385720 24444 consensus_queue.cc:1048] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:19.386186 24321 leader_election.cc:290] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 2 election: Requested vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:19.389175 24395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05cc80ee32e0409c959205fbbcf09e58" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
I20260709 13:23:19.388782 24143 catalog_manager.cc:5697] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50 reported cstate change: term changed from 1 to 2, leader changed from d3ac721977e3467892cbf730c5262219 (127.11.63.66) to 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65). New cstate: current_term: 2 leader_uuid: "66e4fe095997420ba9949bad99bf4d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:19.389891 24395 raft_consensus.cc:3055] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:19.390305 24395 raft_consensus.cc:740] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Running, Role: LEADER
I20260709 13:23:19.391248 24395 consensus_queue.cc:260] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:19.392676 24395 raft_consensus.cc:3060] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:19.393309 24444 consensus_queue.cc:1048] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:19.407743 24247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05cc80ee32e0409c959205fbbcf09e58" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
I20260709 13:23:19.407995 24462 consensus_queue.cc:1048] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:19.408752 24247 raft_consensus.cc:3060] T 05cc80ee32e0409c959205fbbcf09e58 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:19.411147 24395 raft_consensus.cc:2468] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 2.
I20260709 13:23:19.417654 24247 raft_consensus.cc:2468] T 05cc80ee32e0409c959205fbbcf09e58 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 2.
I20260709 13:23:19.418186 24288 leader_election.cc:304] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:19.419137 24489 raft_consensus.cc:2804] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:19.419631 24489 raft_consensus.cc:697] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219 [term 2 LEADER]: Becoming Leader. State: Replica: d3ac721977e3467892cbf730c5262219, State: Running, Role: LEADER
I20260709 13:23:19.420537 24489 consensus_queue.cc:237] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:19.424381 24463 raft_consensus.cc:993] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50: : Instructing follower d3ac721977e3467892cbf730c5262219 to start an election
I20260709 13:23:19.424834 24463 raft_consensus.cc:1081] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Signalling peer d3ac721977e3467892cbf730c5262219 to start an election
I20260709 13:23:19.432721 24440 consensus_queue.cc:1048] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:19.433518 24320 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b027fa0920c8440fab0837dd1142fd57"
dest_uuid: "d3ac721977e3467892cbf730c5262219"
 from {username='slave'} at 127.0.0.1:48128
I20260709 13:23:19.434233 24320 raft_consensus.cc:493] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:19.437188 24320 raft_consensus.cc:3060] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:19.438133 24145 catalog_manager.cc:5697] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219 reported cstate change: term changed from 1 to 2, leader changed from 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67) to d3ac721977e3467892cbf730c5262219 (127.11.63.66). New cstate: current_term: 2 leader_uuid: "d3ac721977e3467892cbf730c5262219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:19.447901 24320 raft_consensus.cc:515] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:19.451212 24246 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b027fa0920c8440fab0837dd1142fd57" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
I20260709 13:23:19.451969 24246 raft_consensus.cc:3055] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:19.452314 24246 raft_consensus.cc:740] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Running, Role: LEADER
I20260709 13:23:19.453193 24246 consensus_queue.cc:260] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
W20260709 13:23:19.454263 24462 consensus_queue.cc:1175] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3aa42500bbcd4121a43413ecd5be246d" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 13:23:19.457888 24320 leader_election.cc:290] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 2 election: Requested vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:19.454265 24246 raft_consensus.cc:3060] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:19.460067 24395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b027fa0920c8440fab0837dd1142fd57" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
I20260709 13:23:19.460845 24395 raft_consensus.cc:3060] T b027fa0920c8440fab0837dd1142fd57 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:19.467411 24246 raft_consensus.cc:2468] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 2.
I20260709 13:23:19.468125 24476 raft_consensus.cc:993] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219: : Instructing follower 66e4fe095997420ba9949bad99bf4d50 to start an election
I20260709 13:23:19.468698 24476 raft_consensus.cc:1081] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [term 1 LEADER]: Signalling peer 66e4fe095997420ba9949bad99bf4d50 to start an election
I20260709 13:23:19.469617 24287 leader_election.cc:304] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:19.471298 24444 raft_consensus.cc:2804] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:19.471885 24444 raft_consensus.cc:697] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219 [term 2 LEADER]: Becoming Leader. State: Replica: d3ac721977e3467892cbf730c5262219, State: Running, Role: LEADER
I20260709 13:23:19.472884 24444 consensus_queue.cc:237] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:19.473727 24247 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0689d527efd74dc1a43383e7f3034595"
dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
 from {username='slave'} at 127.0.0.1:56898
I20260709 13:23:19.474395 24247 raft_consensus.cc:493] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:19.474798 24247 raft_consensus.cc:3060] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:19.480451 24395 raft_consensus.cc:2468] T b027fa0920c8440fab0837dd1142fd57 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 2.
I20260709 13:23:19.482649 24247 raft_consensus.cc:515] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:19.485765 24247 leader_election.cc:290] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 2 election: Requested vote from peers d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:19.486389 24395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0689d527efd74dc1a43383e7f3034595" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
I20260709 13:23:19.487347 24395 raft_consensus.cc:3060] T 0689d527efd74dc1a43383e7f3034595 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:19.487910 24320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0689d527efd74dc1a43383e7f3034595" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3ac721977e3467892cbf730c5262219"
I20260709 13:23:19.486418 24144 catalog_manager.cc:5697] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219 reported cstate change: term changed from 1 to 2, leader changed from 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65) to d3ac721977e3467892cbf730c5262219 (127.11.63.66). New cstate: current_term: 2 leader_uuid: "d3ac721977e3467892cbf730c5262219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:19.488619 24320 raft_consensus.cc:3055] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:19.489003 24320 raft_consensus.cc:740] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d3ac721977e3467892cbf730c5262219, State: Running, Role: LEADER
I20260709 13:23:19.489905 24320 consensus_queue.cc:260] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:19.491438 24320 raft_consensus.cc:3060] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:19.495517 24395 raft_consensus.cc:2468] T 0689d527efd74dc1a43383e7f3034595 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 2.
I20260709 13:23:19.497017 24213 leader_election.cc:304] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, 66e4fe095997420ba9949bad99bf4d50; no voters: 
I20260709 13:23:19.498108 24462 raft_consensus.cc:2804] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:19.498607 24462 raft_consensus.cc:697] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50 [term 2 LEADER]: Becoming Leader. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Running, Role: LEADER
I20260709 13:23:19.499527 24320 raft_consensus.cc:2468] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 2.
I20260709 13:23:19.499536 24462 consensus_queue.cc:237] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:19.508131 24144 catalog_manager.cc:5697] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50 reported cstate change: term changed from 1 to 2, leader changed from d3ac721977e3467892cbf730c5262219 (127.11.63.66) to 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65). New cstate: current_term: 2 leader_uuid: "66e4fe095997420ba9949bad99bf4d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:19.555716 24320 raft_consensus.cc:1275] T 7f2735d0953445d8b941fd6ad09e76fc P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Refusing update from remote peer 66e4fe095997420ba9949bad99bf4d50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:19.557022 24462 consensus_queue.cc:1048] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:19.566347 24395 raft_consensus.cc:1275] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Refusing update from remote peer 66e4fe095997420ba9949bad99bf4d50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:19.570865 24463 consensus_queue.cc:1048] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:19.629714 24462 raft_consensus.cc:993] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50: : Instructing follower d3ac721977e3467892cbf730c5262219 to start an election
I20260709 13:23:19.630240 24463 raft_consensus.cc:1081] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Signalling peer d3ac721977e3467892cbf730c5262219 to start an election
I20260709 13:23:19.632427 24320 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "33f74f7b25d0436fa15cc6053960e3a3"
dest_uuid: "d3ac721977e3467892cbf730c5262219"
 from {username='slave'} at 127.0.0.1:48128
I20260709 13:23:19.632951 24320 raft_consensus.cc:493] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:19.633250 24320 raft_consensus.cc:3060] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:19.638900 24320 raft_consensus.cc:515] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:19.640990 24320 leader_election.cc:290] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 2 election: Requested vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:19.641721 24247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33f74f7b25d0436fa15cc6053960e3a3" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
I20260709 13:23:19.642093 24395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33f74f7b25d0436fa15cc6053960e3a3" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
I20260709 13:23:19.642467 24247 raft_consensus.cc:3055] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:19.642685 24395 raft_consensus.cc:3060] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:19.642858 24247 raft_consensus.cc:740] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Running, Role: LEADER
I20260709 13:23:19.643736 24247 consensus_queue.cc:260] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:19.644964 24247 raft_consensus.cc:3060] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:19.650331 24395 raft_consensus.cc:2468] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 2.
I20260709 13:23:19.651818 24288 leader_election.cc:304] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:19.652364 24247 raft_consensus.cc:2468] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 2.
I20260709 13:23:19.652696 24444 raft_consensus.cc:2804] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:19.653276 24444 raft_consensus.cc:697] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [term 2 LEADER]: Becoming Leader. State: Replica: d3ac721977e3467892cbf730c5262219, State: Running, Role: LEADER
I20260709 13:23:19.654183 24444 consensus_queue.cc:237] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:19.660684 24144 catalog_manager.cc:5697] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 reported cstate change: term changed from 1 to 2, leader changed from 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65) to d3ac721977e3467892cbf730c5262219 (127.11.63.66). New cstate: current_term: 2 leader_uuid: "d3ac721977e3467892cbf730c5262219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:19.708161 24247 raft_consensus.cc:1275] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Refusing update from remote peer d3ac721977e3467892cbf730c5262219: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:19.709506 24444 consensus_queue.cc:1048] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:19.718107 24395 raft_consensus.cc:1275] T 0713792c0cb2436b932fadc8d1f428a2 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Refusing update from remote peer d3ac721977e3467892cbf730c5262219: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:19.720544 24444 consensus_queue.cc:1048] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:19.775166 24395 raft_consensus.cc:1275] T 0f82955421054b95ad8df878b1827aa4 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Refusing update from remote peer 66e4fe095997420ba9949bad99bf4d50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:19.776572 24463 consensus_queue.cc:1048] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:19.784670 24320 raft_consensus.cc:1275] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Refusing update from remote peer 66e4fe095997420ba9949bad99bf4d50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:19.786361 24440 consensus_queue.cc:1048] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:19.874183 24395 raft_consensus.cc:1275] T b027fa0920c8440fab0837dd1142fd57 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Refusing update from remote peer d3ac721977e3467892cbf730c5262219: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:19.875566 24444 consensus_queue.cc:1048] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:19.883652 24247 raft_consensus.cc:1275] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Refusing update from remote peer d3ac721977e3467892cbf730c5262219: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:19.885414 24476 consensus_queue.cc:1048] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:19.931216 24395 raft_consensus.cc:1275] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Refusing update from remote peer d3ac721977e3467892cbf730c5262219: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:19.932654 24516 consensus_queue.cc:1048] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:19.940711 24247 raft_consensus.cc:1275] T 05cc80ee32e0409c959205fbbcf09e58 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Refusing update from remote peer d3ac721977e3467892cbf730c5262219: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:19.942288 24489 consensus_queue.cc:1048] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:19.944855 24320 raft_consensus.cc:1275] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Refusing update from remote peer 66e4fe095997420ba9949bad99bf4d50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:19.947072 24440 consensus_queue.cc:1048] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:19.955725 24395 raft_consensus.cc:1275] T 0689d527efd74dc1a43383e7f3034595 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Refusing update from remote peer 66e4fe095997420ba9949bad99bf4d50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:19.957178 24462 consensus_queue.cc:1048] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:20.094758 24247 raft_consensus.cc:1275] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Refusing update from remote peer d3ac721977e3467892cbf730c5262219: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:20.096382 24489 consensus_queue.cc:1048] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:20.106539 24395 raft_consensus.cc:1275] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Refusing update from remote peer d3ac721977e3467892cbf730c5262219: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:20.108750 24444 consensus_queue.cc:1048] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:23:22.459715 24268 debug-util.cc:398] Leaking SignalData structure 0x7b0800247bc0 after lost signal to thread 11520
W20260709 13:23:22.461270 24268 debug-util.cc:398] Leaking SignalData structure 0x7b080036dfc0 after lost signal to thread 24179
W20260709 13:23:22.462496 24268 debug-util.cc:398] Leaking SignalData structure 0x7b08003b2c20 after lost signal to thread 24271
W20260709 13:23:22.463559 24268 debug-util.cc:398] Leaking SignalData structure 0x7b08001ddd20 after lost signal to thread 24346
W20260709 13:23:22.464532 24268 debug-util.cc:398] Leaking SignalData structure 0x7b0800130ec0 after lost signal to thread 24420
I20260709 13:23:22.790890 24145 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49776:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:23:22.794569 24145 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:23:22.876216 24233 tablet_service.cc:1511] Processing CreateTablet for tablet 48b0cd91c4e04dc7afb4f78ceeebb0b2 (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:23:22.877864 24310 tablet_service.cc:1511] Processing CreateTablet for tablet 48b0cd91c4e04dc7afb4f78ceeebb0b2 (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:23:22.879912 24310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48b0cd91c4e04dc7afb4f78ceeebb0b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:22.879796 24381 tablet_service.cc:1511] Processing CreateTablet for tablet 48b0cd91c4e04dc7afb4f78ceeebb0b2 (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:23:22.881911 24381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48b0cd91c4e04dc7afb4f78ceeebb0b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:22.878468 24233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48b0cd91c4e04dc7afb4f78ceeebb0b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:22.886153 24382 tablet_service.cc:1511] Processing CreateTablet for tablet 337bdcde8c194dccb8aee8a99e36a997 (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:23:22.887683 24236 tablet_service.cc:1511] Processing CreateTablet for tablet 337bdcde8c194dccb8aee8a99e36a997 (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:23:22.888844 24309 tablet_service.cc:1511] Processing CreateTablet for tablet 337bdcde8c194dccb8aee8a99e36a997 (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:23:22.889586 24236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 337bdcde8c194dccb8aee8a99e36a997. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:22.890853 24382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 337bdcde8c194dccb8aee8a99e36a997. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:22.894613 24309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 337bdcde8c194dccb8aee8a99e36a997. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:22.904908 24384 tablet_service.cc:1511] Processing CreateTablet for tablet 5a467f6687c5464e8c8ea4429a2f0e4b (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:23:22.906607 24384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a467f6687c5464e8c8ea4429a2f0e4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:22.924587 24235 tablet_service.cc:1511] Processing CreateTablet for tablet 5a467f6687c5464e8c8ea4429a2f0e4b (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:23:22.927064 24235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a467f6687c5464e8c8ea4429a2f0e4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:22.945947 24311 tablet_service.cc:1511] Processing CreateTablet for tablet 5a467f6687c5464e8c8ea4429a2f0e4b (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:23:22.947666 24311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a467f6687c5464e8c8ea4429a2f0e4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:22.976043 24235 tablet_service.cc:1511] Processing CreateTablet for tablet 3a38a604d77f4400b910e48e3e9adc54 (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:23:22.977269 24309 tablet_service.cc:1511] Processing CreateTablet for tablet 3a38a604d77f4400b910e48e3e9adc54 (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:23:22.977734 24541 tablet_bootstrap.cc:492] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50: Bootstrap starting.
I20260709 13:23:22.977795 24235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a38a604d77f4400b910e48e3e9adc54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:22.986515 24542 tablet_bootstrap.cc:492] T 337bdcde8c194dccb8aee8a99e36a997 P 3aa42500bbcd4121a43413ecd5be246d: Bootstrap starting.
I20260709 13:23:22.988559 24309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a38a604d77f4400b910e48e3e9adc54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:23.000787 24543 tablet_bootstrap.cc:492] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219: Bootstrap starting.
I20260709 13:23:23.001228 24382 tablet_service.cc:1511] Processing CreateTablet for tablet 3a38a604d77f4400b910e48e3e9adc54 (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:23:23.002862 24382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a38a604d77f4400b910e48e3e9adc54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:23.006311 24542 tablet_bootstrap.cc:654] T 337bdcde8c194dccb8aee8a99e36a997 P 3aa42500bbcd4121a43413ecd5be246d: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.012728 24541 tablet_bootstrap.cc:654] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.013347 24235 tablet_service.cc:1511] Processing CreateTablet for tablet 32ccadb88b4348f7ab9d7a50b967c3d6 (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:23:23.018585 24235 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32ccadb88b4348f7ab9d7a50b967c3d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:23.016858 24310 tablet_service.cc:1511] Processing CreateTablet for tablet 32ccadb88b4348f7ab9d7a50b967c3d6 (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:23:23.019608 24543 tablet_bootstrap.cc:654] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.019778 24310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32ccadb88b4348f7ab9d7a50b967c3d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:23.023900 24384 tablet_service.cc:1511] Processing CreateTablet for tablet 32ccadb88b4348f7ab9d7a50b967c3d6 (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:23:23.025229 24236 tablet_service.cc:1511] Processing CreateTablet for tablet 59ddbccb19fc424c95e59a4455cea713 (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:23:23.026811 24236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59ddbccb19fc424c95e59a4455cea713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:23.038739 24381 tablet_service.cc:1511] Processing CreateTablet for tablet 59ddbccb19fc424c95e59a4455cea713 (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:23:23.025364 24384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32ccadb88b4348f7ab9d7a50b967c3d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:23.042882 24233 tablet_service.cc:1511] Processing CreateTablet for tablet 192a3efb7b804c38bca2ad2df1ed1188 (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:23:23.044070 24381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59ddbccb19fc424c95e59a4455cea713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:23.044441 24233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 192a3efb7b804c38bca2ad2df1ed1188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:23.055181 24311 tablet_service.cc:1511] Processing CreateTablet for tablet 192a3efb7b804c38bca2ad2df1ed1188 (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:23:23.056728 24311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 192a3efb7b804c38bca2ad2df1ed1188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:23.060298 24382 tablet_service.cc:1511] Processing CreateTablet for tablet 192a3efb7b804c38bca2ad2df1ed1188 (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:23:23.061817 24382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 192a3efb7b804c38bca2ad2df1ed1188. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:23.062575 24309 tablet_service.cc:1511] Processing CreateTablet for tablet 59ddbccb19fc424c95e59a4455cea713 (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:23:23.063997 24309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59ddbccb19fc424c95e59a4455cea713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:23.063709 24231 tablet_service.cc:1511] Processing CreateTablet for tablet ef9538c5bb80422eb571d567b173d003 (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:23:23.064469 24541 tablet_bootstrap.cc:492] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50: No bootstrap required, opened a new log
I20260709 13:23:23.065084 24541 ts_tablet_manager.cc:1403] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50: Time spent bootstrapping tablet: real 0.088s	user 0.006s	sys 0.008s
I20260709 13:23:23.065315 24231 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef9538c5bb80422eb571d567b173d003. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:23.068665 24543 tablet_bootstrap.cc:492] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219: No bootstrap required, opened a new log
I20260709 13:23:23.069305 24543 ts_tablet_manager.cc:1403] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219: Time spent bootstrapping tablet: real 0.069s	user 0.012s	sys 0.004s
I20260709 13:23:23.073601 24543 raft_consensus.cc:359] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.074900 24543 raft_consensus.cc:385] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:23.075343 24543 raft_consensus.cc:740] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3ac721977e3467892cbf730c5262219, State: Initialized, Role: FOLLOWER
I20260709 13:23:23.076509 24543 consensus_queue.cc:260] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.080510 24543 ts_tablet_manager.cc:1434] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:23:23.081784 24543 tablet_bootstrap.cc:492] T 337bdcde8c194dccb8aee8a99e36a997 P d3ac721977e3467892cbf730c5262219: Bootstrap starting.
I20260709 13:23:23.085733 24541 raft_consensus.cc:359] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.086817 24541 raft_consensus.cc:385] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:23.087179 24541 raft_consensus.cc:740] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Initialized, Role: FOLLOWER
I20260709 13:23:23.088054 24541 consensus_queue.cc:260] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.088639 24542 tablet_bootstrap.cc:492] T 337bdcde8c194dccb8aee8a99e36a997 P 3aa42500bbcd4121a43413ecd5be246d: No bootstrap required, opened a new log
I20260709 13:23:23.089272 24542 ts_tablet_manager.cc:1403] T 337bdcde8c194dccb8aee8a99e36a997 P 3aa42500bbcd4121a43413ecd5be246d: Time spent bootstrapping tablet: real 0.103s	user 0.014s	sys 0.008s
I20260709 13:23:23.091164 24541 ts_tablet_manager.cc:1434] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50: Time spent starting tablet: real 0.026s	user 0.010s	sys 0.000s
I20260709 13:23:23.091358 24543 tablet_bootstrap.cc:654] T 337bdcde8c194dccb8aee8a99e36a997 P d3ac721977e3467892cbf730c5262219: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.092447 24541 tablet_bootstrap.cc:492] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50: Bootstrap starting.
I20260709 13:23:23.103660 24542 raft_consensus.cc:359] T 337bdcde8c194dccb8aee8a99e36a997 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.117743 24542 raft_consensus.cc:385] T 337bdcde8c194dccb8aee8a99e36a997 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:23.121680 24542 raft_consensus.cc:740] T 337bdcde8c194dccb8aee8a99e36a997 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Initialized, Role: FOLLOWER
I20260709 13:23:23.122742 24542 consensus_queue.cc:260] T 337bdcde8c194dccb8aee8a99e36a997 P 3aa42500bbcd4121a43413ecd5be246d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.125831 24310 tablet_service.cc:1511] Processing CreateTablet for tablet ef9538c5bb80422eb571d567b173d003 (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:23:23.133483 24310 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef9538c5bb80422eb571d567b173d003. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:23.130466 24383 tablet_service.cc:1511] Processing CreateTablet for tablet ef9538c5bb80422eb571d567b173d003 (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:23:23.139835 24383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef9538c5bb80422eb571d567b173d003. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:23.147178 24541 tablet_bootstrap.cc:654] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.155103 24231 tablet_service.cc:1511] Processing CreateTablet for tablet 9de1041b009d4b3195afb70e0852c742 (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:23:23.156548 24542 ts_tablet_manager.cc:1434] T 337bdcde8c194dccb8aee8a99e36a997 P 3aa42500bbcd4121a43413ecd5be246d: Time spent starting tablet: real 0.063s	user 0.007s	sys 0.002s
I20260709 13:23:23.157178 24231 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9de1041b009d4b3195afb70e0852c742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:23.157889 24542 tablet_bootstrap.cc:492] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d: Bootstrap starting.
I20260709 13:23:23.198297 24309 tablet_service.cc:1511] Processing CreateTablet for tablet 9de1041b009d4b3195afb70e0852c742 (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:23:23.200316 24309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9de1041b009d4b3195afb70e0852c742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:23.202925 24382 tablet_service.cc:1511] Processing CreateTablet for tablet 9de1041b009d4b3195afb70e0852c742 (DEFAULT_TABLE table=soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:23:23.204607 24382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9de1041b009d4b3195afb70e0852c742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:23:23.210582 24543 tablet_bootstrap.cc:492] T 337bdcde8c194dccb8aee8a99e36a997 P d3ac721977e3467892cbf730c5262219: No bootstrap required, opened a new log
I20260709 13:23:23.211494 24543 ts_tablet_manager.cc:1403] T 337bdcde8c194dccb8aee8a99e36a997 P d3ac721977e3467892cbf730c5262219: Time spent bootstrapping tablet: real 0.130s	user 0.046s	sys 0.033s
I20260709 13:23:23.214888 24543 raft_consensus.cc:359] T 337bdcde8c194dccb8aee8a99e36a997 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.215665 24543 raft_consensus.cc:385] T 337bdcde8c194dccb8aee8a99e36a997 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:23.216001 24543 raft_consensus.cc:740] T 337bdcde8c194dccb8aee8a99e36a997 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3ac721977e3467892cbf730c5262219, State: Initialized, Role: FOLLOWER
I20260709 13:23:23.216820 24543 consensus_queue.cc:260] T 337bdcde8c194dccb8aee8a99e36a997 P d3ac721977e3467892cbf730c5262219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.219623 24543 ts_tablet_manager.cc:1434] T 337bdcde8c194dccb8aee8a99e36a997 P d3ac721977e3467892cbf730c5262219: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:23:23.220666 24543 tablet_bootstrap.cc:492] T 5a467f6687c5464e8c8ea4429a2f0e4b P d3ac721977e3467892cbf730c5262219: Bootstrap starting.
I20260709 13:23:23.222172 24549 raft_consensus.cc:493] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:23.222779 24549 raft_consensus.cc:515] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.226058 24549 leader_election.cc:290] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:23.229036 24396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48b0cd91c4e04dc7afb4f78ceeebb0b2" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d" is_pre_election: true
W20260709 13:23:23.234148 24213 leader_election.cc:343] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:23.235185 24320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48b0cd91c4e04dc7afb4f78ceeebb0b2" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219" is_pre_election: true
I20260709 13:23:23.236075 24320 raft_consensus.cc:2468] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 0.
I20260709 13:23:23.236788 24543 tablet_bootstrap.cc:654] T 5a467f6687c5464e8c8ea4429a2f0e4b P d3ac721977e3467892cbf730c5262219: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.238487 24212 leader_election.cc:304] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 3aa42500bbcd4121a43413ecd5be246d
I20260709 13:23:23.239552 24549 raft_consensus.cc:2804] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:23.239966 24549 raft_consensus.cc:493] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:23.240324 24549 raft_consensus.cc:3060] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:23.245263 24542 tablet_bootstrap.cc:654] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.252800 24549 raft_consensus.cc:515] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.249514 24541 tablet_bootstrap.cc:492] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50: No bootstrap required, opened a new log
I20260709 13:23:23.254097 24541 ts_tablet_manager.cc:1403] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50: Time spent bootstrapping tablet: real 0.162s	user 0.039s	sys 0.032s
I20260709 13:23:23.256139 24320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48b0cd91c4e04dc7afb4f78ceeebb0b2" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219"
I20260709 13:23:23.256794 24320 raft_consensus.cc:3060] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:23.258229 24543 tablet_bootstrap.cc:492] T 5a467f6687c5464e8c8ea4429a2f0e4b P d3ac721977e3467892cbf730c5262219: No bootstrap required, opened a new log
I20260709 13:23:23.258812 24543 ts_tablet_manager.cc:1403] T 5a467f6687c5464e8c8ea4429a2f0e4b P d3ac721977e3467892cbf730c5262219: Time spent bootstrapping tablet: real 0.038s	user 0.011s	sys 0.003s
I20260709 13:23:23.261705 24396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48b0cd91c4e04dc7afb4f78ceeebb0b2" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
W20260709 13:23:23.263218 24213 leader_election.cc:343] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:23.263777 24549 leader_election.cc:290] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 election: Requested vote from peers d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:23.265048 24541 raft_consensus.cc:359] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.265991 24541 raft_consensus.cc:385] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:23.266332 24541 raft_consensus.cc:740] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Initialized, Role: FOLLOWER
I20260709 13:23:23.266324 24543 raft_consensus.cc:359] T 5a467f6687c5464e8c8ea4429a2f0e4b P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.268566 24543 raft_consensus.cc:385] T 5a467f6687c5464e8c8ea4429a2f0e4b P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:23.268566 24541 consensus_queue.cc:260] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.269121 24543 raft_consensus.cc:740] T 5a467f6687c5464e8c8ea4429a2f0e4b P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3ac721977e3467892cbf730c5262219, State: Initialized, Role: FOLLOWER
I20260709 13:23:23.270012 24543 consensus_queue.cc:260] T 5a467f6687c5464e8c8ea4429a2f0e4b P d3ac721977e3467892cbf730c5262219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.270727 24320 raft_consensus.cc:2468] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 1.
I20260709 13:23:23.272444 24212 leader_election.cc:304] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 3aa42500bbcd4121a43413ecd5be246d
I20260709 13:23:23.273598 24541 ts_tablet_manager.cc:1434] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:23:23.273768 24549 raft_consensus.cc:2804] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:23.275003 24541 tablet_bootstrap.cc:492] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50: Bootstrap starting.
I20260709 13:23:23.275570 24542 tablet_bootstrap.cc:492] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d: No bootstrap required, opened a new log
I20260709 13:23:23.276176 24542 ts_tablet_manager.cc:1403] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d: Time spent bootstrapping tablet: real 0.119s	user 0.014s	sys 0.000s
I20260709 13:23:23.274406 24549 raft_consensus.cc:697] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Becoming Leader. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Running, Role: LEADER
I20260709 13:23:23.278061 24549 consensus_queue.cc:237] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.280118 24529 raft_consensus.cc:493] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:23.280920 24529 raft_consensus.cc:515] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.283905 24543 ts_tablet_manager.cc:1434] T 5a467f6687c5464e8c8ea4429a2f0e4b P d3ac721977e3467892cbf730c5262219: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.003s
I20260709 13:23:23.285820 24396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "337bdcde8c194dccb8aee8a99e36a997" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d" is_pre_election: true
I20260709 13:23:23.286477 24396 raft_consensus.cc:2468] T 337bdcde8c194dccb8aee8a99e36a997 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 0.
I20260709 13:23:23.287664 24213 leader_election.cc:304] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, 66e4fe095997420ba9949bad99bf4d50; no voters: 
I20260709 13:23:23.288179 24321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "337bdcde8c194dccb8aee8a99e36a997" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219" is_pre_election: true
I20260709 13:23:23.289252 24529 leader_election.cc:290] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:23.280148 24542 raft_consensus.cc:359] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.290375 24321 raft_consensus.cc:2468] T 337bdcde8c194dccb8aee8a99e36a997 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 0.
I20260709 13:23:23.291855 24529 raft_consensus.cc:2804] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:23.292307 24529 raft_consensus.cc:493] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:23.292722 24529 raft_consensus.cc:3060] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:23.293354 24541 tablet_bootstrap.cc:654] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.296715 24542 raft_consensus.cc:385] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:23.299326 24542 raft_consensus.cc:740] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Initialized, Role: FOLLOWER
I20260709 13:23:23.301961 24542 consensus_queue.cc:260] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.310429 24542 ts_tablet_manager.cc:1434] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.005s
I20260709 13:23:23.310429 24543 tablet_bootstrap.cc:492] T 3a38a604d77f4400b910e48e3e9adc54 P d3ac721977e3467892cbf730c5262219: Bootstrap starting.
I20260709 13:23:23.313961 24529 raft_consensus.cc:515] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.319164 24542 tablet_bootstrap.cc:492] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d: Bootstrap starting.
I20260709 13:23:23.319655 24541 tablet_bootstrap.cc:492] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50: No bootstrap required, opened a new log
I20260709 13:23:23.329398 24529 leader_election.cc:290] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 election: Requested vote from peers d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:23.327076 24543 tablet_bootstrap.cc:654] T 3a38a604d77f4400b910e48e3e9adc54 P d3ac721977e3467892cbf730c5262219: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.330036 24541 ts_tablet_manager.cc:1403] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50: Time spent bootstrapping tablet: real 0.055s	user 0.015s	sys 0.003s
I20260709 13:23:23.330366 24396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "337bdcde8c194dccb8aee8a99e36a997" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
I20260709 13:23:23.330547 24321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "337bdcde8c194dccb8aee8a99e36a997" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219"
I20260709 13:23:23.331056 24396 raft_consensus.cc:3060] T 337bdcde8c194dccb8aee8a99e36a997 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:23.331228 24321 raft_consensus.cc:3060] T 337bdcde8c194dccb8aee8a99e36a997 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:23.333981 24541 raft_consensus.cc:359] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.334932 24541 raft_consensus.cc:385] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:23.335254 24541 raft_consensus.cc:740] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Initialized, Role: FOLLOWER
I20260709 13:23:23.336303 24541 consensus_queue.cc:260] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.339071 24541 ts_tablet_manager.cc:1434] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:23:23.340180 24541 tablet_bootstrap.cc:492] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50: Bootstrap starting.
I20260709 13:23:23.341684 24143 catalog_manager.cc:5697] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "66e4fe095997420ba9949bad99bf4d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:23.349357 24321 raft_consensus.cc:2468] T 337bdcde8c194dccb8aee8a99e36a997 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 1.
I20260709 13:23:23.351081 24212 leader_election.cc:304] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:23.352859 24529 raft_consensus.cc:2804] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:23.353395 24529 raft_consensus.cc:697] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Becoming Leader. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Running, Role: LEADER
I20260709 13:23:23.354290 24529 consensus_queue.cc:237] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.327776 24542 tablet_bootstrap.cc:654] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.365263 24396 raft_consensus.cc:2468] T 337bdcde8c194dccb8aee8a99e36a997 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 1.
I20260709 13:23:23.367784 24541 tablet_bootstrap.cc:654] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.370388 24543 tablet_bootstrap.cc:492] T 3a38a604d77f4400b910e48e3e9adc54 P d3ac721977e3467892cbf730c5262219: No bootstrap required, opened a new log
I20260709 13:23:23.370885 24543 ts_tablet_manager.cc:1403] T 3a38a604d77f4400b910e48e3e9adc54 P d3ac721977e3467892cbf730c5262219: Time spent bootstrapping tablet: real 0.071s	user 0.023s	sys 0.024s
I20260709 13:23:23.373721 24543 raft_consensus.cc:359] T 3a38a604d77f4400b910e48e3e9adc54 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.374543 24543 raft_consensus.cc:385] T 3a38a604d77f4400b910e48e3e9adc54 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:23.377240 24543 raft_consensus.cc:740] T 3a38a604d77f4400b910e48e3e9adc54 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3ac721977e3467892cbf730c5262219, State: Initialized, Role: FOLLOWER
I20260709 13:23:23.378194 24543 consensus_queue.cc:260] T 3a38a604d77f4400b910e48e3e9adc54 P d3ac721977e3467892cbf730c5262219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.380748 24543 ts_tablet_manager.cc:1434] T 3a38a604d77f4400b910e48e3e9adc54 P d3ac721977e3467892cbf730c5262219: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:23:23.383088 24542 tablet_bootstrap.cc:492] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d: No bootstrap required, opened a new log
I20260709 13:23:23.383708 24542 ts_tablet_manager.cc:1403] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d: Time spent bootstrapping tablet: real 0.065s	user 0.012s	sys 0.006s
I20260709 13:23:23.386981 24542 raft_consensus.cc:359] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.387764 24542 raft_consensus.cc:385] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:23.387868 24543 tablet_bootstrap.cc:492] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219: Bootstrap starting.
I20260709 13:23:23.388137 24542 raft_consensus.cc:740] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Initialized, Role: FOLLOWER
I20260709 13:23:23.389199 24542 consensus_queue.cc:260] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.391106 24542 ts_tablet_manager.cc:1434] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:23:23.392252 24542 tablet_bootstrap.cc:492] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d: Bootstrap starting.
I20260709 13:23:23.381317 24143 catalog_manager.cc:5697] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "66e4fe095997420ba9949bad99bf4d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:23.398687 24543 tablet_bootstrap.cc:654] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.400471 24542 tablet_bootstrap.cc:654] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.413393 24525 raft_consensus.cc:493] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:23.413902 24542 tablet_bootstrap.cc:492] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d: No bootstrap required, opened a new log
I20260709 13:23:23.414542 24542 ts_tablet_manager.cc:1403] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.004s
I20260709 13:23:23.414161 24525 raft_consensus.cc:515] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.417274 24529 raft_consensus.cc:493] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:23.417846 24529 raft_consensus.cc:515] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.418182 24542 raft_consensus.cc:359] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.419057 24542 raft_consensus.cc:385] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:23.419188 24321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a467f6687c5464e8c8ea4429a2f0e4b" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219" is_pre_election: true
I20260709 13:23:23.419409 24542 raft_consensus.cc:740] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Initialized, Role: FOLLOWER
I20260709 13:23:23.419889 24321 raft_consensus.cc:2468] T 5a467f6687c5464e8c8ea4429a2f0e4b P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 0.
I20260709 13:23:23.420550 24542 consensus_queue.cc:260] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.421476 24361 leader_election.cc:304] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:23.422500 24247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a467f6687c5464e8c8ea4429a2f0e4b" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50" is_pre_election: true
I20260709 13:23:23.419945 24529 leader_election.cc:290] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:23.423121 24396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a467f6687c5464e8c8ea4429a2f0e4b" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d" is_pre_election: true
I20260709 13:23:23.423168 24247 raft_consensus.cc:2468] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 0.
I20260709 13:23:23.423874 24562 raft_consensus.cc:2804] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:23.426803 24396 raft_consensus.cc:2468] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 0.
I20260709 13:23:23.429312 24321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a467f6687c5464e8c8ea4429a2f0e4b" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219" is_pre_election: true
I20260709 13:23:23.430135 24321 raft_consensus.cc:2468] T 5a467f6687c5464e8c8ea4429a2f0e4b P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 0.
I20260709 13:23:23.431097 24213 leader_election.cc:304] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, 66e4fe095997420ba9949bad99bf4d50; no voters: 
I20260709 13:23:23.432176 24529 raft_consensus.cc:2804] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:23.432446 24562 raft_consensus.cc:493] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:23.432559 24529 raft_consensus.cc:493] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:23.433094 24562 raft_consensus.cc:3060] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:23.433193 24529 raft_consensus.cc:3060] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:23.434123 24525 leader_election.cc:290] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237)
I20260709 13:23:23.443331 24529 raft_consensus.cc:515] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.444218 24541 tablet_bootstrap.cc:492] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50: No bootstrap required, opened a new log
I20260709 13:23:23.444880 24541 ts_tablet_manager.cc:1403] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50: Time spent bootstrapping tablet: real 0.105s	user 0.008s	sys 0.004s
I20260709 13:23:23.444989 24542 ts_tablet_manager.cc:1434] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d: Time spent starting tablet: real 0.030s	user 0.011s	sys 0.003s
I20260709 13:23:23.446205 24542 tablet_bootstrap.cc:492] T 59ddbccb19fc424c95e59a4455cea713 P 3aa42500bbcd4121a43413ecd5be246d: Bootstrap starting.
I20260709 13:23:23.446311 24321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a467f6687c5464e8c8ea4429a2f0e4b" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219"
I20260709 13:23:23.446609 24396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a467f6687c5464e8c8ea4429a2f0e4b" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
I20260709 13:23:23.447052 24321 raft_consensus.cc:3060] T 5a467f6687c5464e8c8ea4429a2f0e4b P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:23.447464 24529 leader_election.cc:290] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 election: Requested vote from peers d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:23.448591 24541 raft_consensus.cc:359] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.449512 24541 raft_consensus.cc:385] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:23.449934 24541 raft_consensus.cc:740] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Initialized, Role: FOLLOWER
I20260709 13:23:23.451305 24541 consensus_queue.cc:260] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.452402 24543 tablet_bootstrap.cc:492] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219: No bootstrap required, opened a new log
I20260709 13:23:23.455477 24321 raft_consensus.cc:2468] T 5a467f6687c5464e8c8ea4429a2f0e4b P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 1.
I20260709 13:23:23.459311 24562 raft_consensus.cc:515] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.460399 24561 raft_consensus.cc:493] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:23.462215 24543 ts_tablet_manager.cc:1403] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219: Time spent bootstrapping tablet: real 0.075s	user 0.023s	sys 0.011s
I20260709 13:23:23.462443 24561 raft_consensus.cc:515] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.463213 24212 leader_election.cc:304] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:23.463685 24541 ts_tablet_manager.cc:1434] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:23:23.464502 24549 raft_consensus.cc:2804] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:23.464982 24549 raft_consensus.cc:697] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Becoming Leader. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Running, Role: LEADER
I20260709 13:23:23.465044 24541 tablet_bootstrap.cc:492] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50: Bootstrap starting.
I20260709 13:23:23.465883 24543 raft_consensus.cc:359] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.466432 24247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a38a604d77f4400b910e48e3e9adc54" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50" is_pre_election: true
I20260709 13:23:23.466715 24543 raft_consensus.cc:385] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:23.465920 24549 consensus_queue.cc:237] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.467175 24247 raft_consensus.cc:2468] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 0.
I20260709 13:23:23.467221 24543 raft_consensus.cc:740] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3ac721977e3467892cbf730c5262219, State: Initialized, Role: FOLLOWER
I20260709 13:23:23.468371 24543 consensus_queue.cc:260] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.468705 24361 leader_election.cc:304] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, 66e4fe095997420ba9949bad99bf4d50; no voters: 
I20260709 13:23:23.469856 24525 raft_consensus.cc:2804] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:23.470249 24525 raft_consensus.cc:493] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:23.470754 24525 raft_consensus.cc:3060] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:23.471438 24543 ts_tablet_manager.cc:1434] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:23:23.471587 24561 leader_election.cc:290] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237)
I20260709 13:23:23.472813 24543 tablet_bootstrap.cc:492] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219: Bootstrap starting.
I20260709 13:23:23.473738 24541 tablet_bootstrap.cc:654] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.474746 24247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a467f6687c5464e8c8ea4429a2f0e4b" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
I20260709 13:23:23.461114 24542 tablet_bootstrap.cc:654] T 59ddbccb19fc424c95e59a4455cea713 P 3aa42500bbcd4121a43413ecd5be246d: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.476751 24396 raft_consensus.cc:2393] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 66e4fe095997420ba9949bad99bf4d50 in current term 1: Already voted for candidate 3aa42500bbcd4121a43413ecd5be246d in this term.
I20260709 13:23:23.480865 24320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a467f6687c5464e8c8ea4429a2f0e4b" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219"
I20260709 13:23:23.481959 24320 raft_consensus.cc:2393] T 5a467f6687c5464e8c8ea4429a2f0e4b P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3aa42500bbcd4121a43413ecd5be246d in current term 1: Already voted for candidate 66e4fe095997420ba9949bad99bf4d50 in this term.
I20260709 13:23:23.485230 24543 tablet_bootstrap.cc:654] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.484925 24361 leader_election.cc:304] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d; no voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219
I20260709 13:23:23.486277 24561 raft_consensus.cc:2749] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:23:23.485571 24562 leader_election.cc:290] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 election: Requested vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237)
I20260709 13:23:23.492439 24543 tablet_bootstrap.cc:492] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219: No bootstrap required, opened a new log
I20260709 13:23:23.492712 24321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a38a604d77f4400b910e48e3e9adc54" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219" is_pre_election: true
I20260709 13:23:23.495198 24321 raft_consensus.cc:2468] T 3a38a604d77f4400b910e48e3e9adc54 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 0.
I20260709 13:23:23.497980 24542 tablet_bootstrap.cc:492] T 59ddbccb19fc424c95e59a4455cea713 P 3aa42500bbcd4121a43413ecd5be246d: No bootstrap required, opened a new log
I20260709 13:23:23.498545 24542 ts_tablet_manager.cc:1403] T 59ddbccb19fc424c95e59a4455cea713 P 3aa42500bbcd4121a43413ecd5be246d: Time spent bootstrapping tablet: real 0.053s	user 0.015s	sys 0.000s
I20260709 13:23:23.500543 24525 raft_consensus.cc:515] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.501048 24542 raft_consensus.cc:359] T 59ddbccb19fc424c95e59a4455cea713 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.502074 24542 raft_consensus.cc:385] T 59ddbccb19fc424c95e59a4455cea713 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:23.502489 24542 raft_consensus.cc:740] T 59ddbccb19fc424c95e59a4455cea713 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Initialized, Role: FOLLOWER
I20260709 13:23:23.503340 24247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a38a604d77f4400b910e48e3e9adc54" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
I20260709 13:23:23.503959 24247 raft_consensus.cc:3060] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:23.503484 24542 consensus_queue.cc:260] T 59ddbccb19fc424c95e59a4455cea713 P 3aa42500bbcd4121a43413ecd5be246d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.493218 24543 ts_tablet_manager.cc:1403] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.001s
I20260709 13:23:23.506634 24542 ts_tablet_manager.cc:1434] T 59ddbccb19fc424c95e59a4455cea713 P 3aa42500bbcd4121a43413ecd5be246d: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:23:23.507776 24542 tablet_bootstrap.cc:492] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d: Bootstrap starting.
I20260709 13:23:23.508795 24543 raft_consensus.cc:359] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.512485 24541 tablet_bootstrap.cc:492] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50: No bootstrap required, opened a new log
I20260709 13:23:23.514674 24541 ts_tablet_manager.cc:1403] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50: Time spent bootstrapping tablet: real 0.050s	user 0.006s	sys 0.009s
I20260709 13:23:23.515058 24321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a38a604d77f4400b910e48e3e9adc54" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219"
I20260709 13:23:23.515873 24321 raft_consensus.cc:3060] T 3a38a604d77f4400b910e48e3e9adc54 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:23.518280 24541 raft_consensus.cc:359] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.519194 24541 raft_consensus.cc:385] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:23.519531 24541 raft_consensus.cc:740] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Initialized, Role: FOLLOWER
I20260709 13:23:23.520403 24541 consensus_queue.cc:260] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.523470 24541 ts_tablet_manager.cc:1434] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:23:23.524717 24541 tablet_bootstrap.cc:492] T 192a3efb7b804c38bca2ad2df1ed1188 P 66e4fe095997420ba9949bad99bf4d50: Bootstrap starting.
I20260709 13:23:23.513056 24247 raft_consensus.cc:2468] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 1.
I20260709 13:23:23.509538 24543 raft_consensus.cc:385] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:23.531698 24361 leader_election.cc:304] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, 66e4fe095997420ba9949bad99bf4d50; no voters: 
I20260709 13:23:23.532896 24562 raft_consensus.cc:2804] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:23.533092 24525 leader_election.cc:290] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 election: Requested vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237)
I20260709 13:23:23.533486 24562 raft_consensus.cc:697] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [term 1 LEADER]: Becoming Leader. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Running, Role: LEADER
I20260709 13:23:23.534490 24542 tablet_bootstrap.cc:654] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.534593 24562 consensus_queue.cc:237] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.537271 24321 raft_consensus.cc:2468] T 3a38a604d77f4400b910e48e3e9adc54 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 1.
I20260709 13:23:23.532270 24543 raft_consensus.cc:740] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3ac721977e3467892cbf730c5262219, State: Initialized, Role: FOLLOWER
I20260709 13:23:23.540407 24543 consensus_queue.cc:260] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.547730 24542 tablet_bootstrap.cc:492] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d: No bootstrap required, opened a new log
I20260709 13:23:23.548384 24542 ts_tablet_manager.cc:1403] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d: Time spent bootstrapping tablet: real 0.041s	user 0.014s	sys 0.000s
I20260709 13:23:23.549086 24543 ts_tablet_manager.cc:1434] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219: Time spent starting tablet: real 0.043s	user 0.003s	sys 0.006s
I20260709 13:23:23.551771 24542 raft_consensus.cc:359] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.552670 24542 raft_consensus.cc:385] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:23.553028 24542 raft_consensus.cc:740] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Initialized, Role: FOLLOWER
I20260709 13:23:23.553465 24543 tablet_bootstrap.cc:492] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219: Bootstrap starting.
I20260709 13:23:23.554124 24542 consensus_queue.cc:260] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.551991 24541 tablet_bootstrap.cc:654] T 192a3efb7b804c38bca2ad2df1ed1188 P 66e4fe095997420ba9949bad99bf4d50: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.556809 24542 ts_tablet_manager.cc:1434] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:23:23.559556 24542 tablet_bootstrap.cc:492] T 192a3efb7b804c38bca2ad2df1ed1188 P 3aa42500bbcd4121a43413ecd5be246d: Bootstrap starting.
I20260709 13:23:23.566766 24542 tablet_bootstrap.cc:654] T 192a3efb7b804c38bca2ad2df1ed1188 P 3aa42500bbcd4121a43413ecd5be246d: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.547336 24143 catalog_manager.cc:5697] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "66e4fe095997420ba9949bad99bf4d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:23.566776 24543 tablet_bootstrap.cc:654] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.575265 24542 tablet_bootstrap.cc:492] T 192a3efb7b804c38bca2ad2df1ed1188 P 3aa42500bbcd4121a43413ecd5be246d: No bootstrap required, opened a new log
I20260709 13:23:23.577930 24542 ts_tablet_manager.cc:1403] T 192a3efb7b804c38bca2ad2df1ed1188 P 3aa42500bbcd4121a43413ecd5be246d: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.000s
I20260709 13:23:23.583693 24542 raft_consensus.cc:359] T 192a3efb7b804c38bca2ad2df1ed1188 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.580308 24541 tablet_bootstrap.cc:492] T 192a3efb7b804c38bca2ad2df1ed1188 P 66e4fe095997420ba9949bad99bf4d50: No bootstrap required, opened a new log
I20260709 13:23:23.584792 24541 ts_tablet_manager.cc:1403] T 192a3efb7b804c38bca2ad2df1ed1188 P 66e4fe095997420ba9949bad99bf4d50: Time spent bootstrapping tablet: real 0.060s	user 0.009s	sys 0.008s
I20260709 13:23:23.588092 24541 raft_consensus.cc:359] T 192a3efb7b804c38bca2ad2df1ed1188 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.588871 24541 raft_consensus.cc:385] T 192a3efb7b804c38bca2ad2df1ed1188 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:23.589210 24541 raft_consensus.cc:740] T 192a3efb7b804c38bca2ad2df1ed1188 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Initialized, Role: FOLLOWER
I20260709 13:23:23.590121 24541 consensus_queue.cc:260] T 192a3efb7b804c38bca2ad2df1ed1188 P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.592600 24541 ts_tablet_manager.cc:1434] T 192a3efb7b804c38bca2ad2df1ed1188 P 66e4fe095997420ba9949bad99bf4d50: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:23:23.593767 24541 tablet_bootstrap.cc:492] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50: Bootstrap starting.
I20260709 13:23:23.594110 24145 catalog_manager.cc:5697] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67). New cstate: current_term: 1 leader_uuid: "3aa42500bbcd4121a43413ecd5be246d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:23.602412 24541 tablet_bootstrap.cc:654] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.607080 24543 tablet_bootstrap.cc:492] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219: No bootstrap required, opened a new log
I20260709 13:23:23.610559 24543 ts_tablet_manager.cc:1403] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219: Time spent bootstrapping tablet: real 0.057s	user 0.018s	sys 0.011s
I20260709 13:23:23.614025 24543 raft_consensus.cc:359] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.614885 24543 raft_consensus.cc:385] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:23.614894 24541 tablet_bootstrap.cc:492] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50: No bootstrap required, opened a new log
I20260709 13:23:23.615284 24543 raft_consensus.cc:740] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3ac721977e3467892cbf730c5262219, State: Initialized, Role: FOLLOWER
I20260709 13:23:23.615769 24541 ts_tablet_manager.cc:1403] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50: Time spent bootstrapping tablet: real 0.022s	user 0.004s	sys 0.015s
I20260709 13:23:23.616240 24543 consensus_queue.cc:260] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.617467 24542 raft_consensus.cc:385] T 192a3efb7b804c38bca2ad2df1ed1188 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:23.617905 24542 raft_consensus.cc:740] T 192a3efb7b804c38bca2ad2df1ed1188 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Initialized, Role: FOLLOWER
I20260709 13:23:23.619808 24541 raft_consensus.cc:359] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.618916 24542 consensus_queue.cc:260] T 192a3efb7b804c38bca2ad2df1ed1188 P 3aa42500bbcd4121a43413ecd5be246d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.620712 24541 raft_consensus.cc:385] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:23.625317 24541 raft_consensus.cc:740] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Initialized, Role: FOLLOWER
I20260709 13:23:23.626253 24541 consensus_queue.cc:260] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.629063 24541 ts_tablet_manager.cc:1434] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:23:23.624255 24542 ts_tablet_manager.cc:1434] T 192a3efb7b804c38bca2ad2df1ed1188 P 3aa42500bbcd4121a43413ecd5be246d: Time spent starting tablet: real 0.043s	user 0.003s	sys 0.003s
W20260709 13:23:23.629829 24193 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:23:23.633335 24542 tablet_bootstrap.cc:492] T ef9538c5bb80422eb571d567b173d003 P 3aa42500bbcd4121a43413ecd5be246d: Bootstrap starting.
I20260709 13:23:23.638095 24543 ts_tablet_manager.cc:1434] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.001s
I20260709 13:23:23.639369 24543 tablet_bootstrap.cc:492] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219: Bootstrap starting.
I20260709 13:23:23.642421 24541 tablet_bootstrap.cc:492] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 66e4fe095997420ba9949bad99bf4d50: Bootstrap starting.
I20260709 13:23:23.648308 24543 tablet_bootstrap.cc:654] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.658057 24542 tablet_bootstrap.cc:654] T ef9538c5bb80422eb571d567b173d003 P 3aa42500bbcd4121a43413ecd5be246d: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.666474 24543 tablet_bootstrap.cc:492] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219: No bootstrap required, opened a new log
I20260709 13:23:23.667168 24543 ts_tablet_manager.cc:1403] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.012s
I20260709 13:23:23.666457 24549 raft_consensus.cc:493] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:23.670188 24543 raft_consensus.cc:359] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.669836 24549 raft_consensus.cc:515] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.670969 24543 raft_consensus.cc:385] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:23.666453 24541 tablet_bootstrap.cc:654] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 66e4fe095997420ba9949bad99bf4d50: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.674403 24543 raft_consensus.cc:740] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3ac721977e3467892cbf730c5262219, State: Initialized, Role: FOLLOWER
I20260709 13:23:23.674561 24321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef9538c5bb80422eb571d567b173d003" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219" is_pre_election: true
I20260709 13:23:23.675271 24543 consensus_queue.cc:260] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.675671 24321 raft_consensus.cc:2468] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 0.
I20260709 13:23:23.678047 24543 ts_tablet_manager.cc:1434] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:23:23.678172 24212 leader_election.cc:304] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:23.680555 24543 tablet_bootstrap.cc:492] T 9de1041b009d4b3195afb70e0852c742 P d3ac721977e3467892cbf730c5262219: Bootstrap starting.
I20260709 13:23:23.680394 24529 raft_consensus.cc:2804] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:23.681282 24529 raft_consensus.cc:493] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:23.681870 24529 raft_consensus.cc:3060] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:23.684175 24541 tablet_bootstrap.cc:492] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 66e4fe095997420ba9949bad99bf4d50: No bootstrap required, opened a new log
I20260709 13:23:23.684759 24541 ts_tablet_manager.cc:1403] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 66e4fe095997420ba9949bad99bf4d50: Time spent bootstrapping tablet: real 0.043s	user 0.013s	sys 0.000s
I20260709 13:23:23.688587 24543 tablet_bootstrap.cc:654] T 9de1041b009d4b3195afb70e0852c742 P d3ac721977e3467892cbf730c5262219: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.699680 24577 raft_consensus.cc:493] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:23.700930 24577 raft_consensus.cc:515] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.700980 24541 raft_consensus.cc:359] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.701984 24562 raft_consensus.cc:493] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:23.702857 24541 raft_consensus.cc:385] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:23.703193 24541 raft_consensus.cc:740] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Initialized, Role: FOLLOWER
I20260709 13:23:23.703050 24562 raft_consensus.cc:515] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.704031 24541 consensus_queue.cc:260] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.706183 24541 ts_tablet_manager.cc:1434] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 66e4fe095997420ba9949bad99bf4d50: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.003s
I20260709 13:23:23.706871 24562 leader_election.cc:290] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237)
I20260709 13:23:23.709005 24321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32ccadb88b4348f7ab9d7a50b967c3d6" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219" is_pre_election: true
I20260709 13:23:23.709400 24541 tablet_bootstrap.cc:492] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50: Bootstrap starting.
I20260709 13:23:23.709868 24321 raft_consensus.cc:2468] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 0.
I20260709 13:23:23.713917 24361 leader_election.cc:304] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:23.715121 24562 raft_consensus.cc:2804] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:23.715571 24562 raft_consensus.cc:493] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:23.715965 24562 raft_consensus.cc:3060] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:23.716032 24541 tablet_bootstrap.cc:654] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:23.719480 24577 leader_election.cc:290] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:23.721707 24321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59ddbccb19fc424c95e59a4455cea713" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219" is_pre_election: true
I20260709 13:23:23.722678 24321 raft_consensus.cc:2468] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 0.
I20260709 13:23:23.724227 24212 leader_election.cc:304] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:23.725884 24562 raft_consensus.cc:515] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.727499 24247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32ccadb88b4348f7ab9d7a50b967c3d6" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50" is_pre_election: true
I20260709 13:23:23.728185 24247 raft_consensus.cc:2468] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 0.
I20260709 13:23:23.728485 24394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59ddbccb19fc424c95e59a4455cea713" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d" is_pre_election: true
I20260709 13:23:23.729235 24394 raft_consensus.cc:2468] T 59ddbccb19fc424c95e59a4455cea713 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 0.
I20260709 13:23:23.730916 24396 raft_consensus.cc:3060] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:23.732318 24320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32ccadb88b4348f7ab9d7a50b967c3d6" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219"
I20260709 13:23:23.733309 24320 raft_consensus.cc:3060] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:23.739750 24552 raft_consensus.cc:2804] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:23.740401 24552 raft_consensus.cc:493] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:23.740813 24552 raft_consensus.cc:3060] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:23.745050 24247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32ccadb88b4348f7ab9d7a50b967c3d6" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
I20260709 13:23:23.746019 24247 raft_consensus.cc:3060] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:23.751935 24320 raft_consensus.cc:2468] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 1.
I20260709 13:23:23.753454 24361 leader_election.cc:304] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:23.754738 24525 raft_consensus.cc:2804] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:23.755402 24525 raft_consensus.cc:697] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [term 1 LEADER]: Becoming Leader. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Running, Role: LEADER
I20260709 13:23:23.756554 24525 consensus_queue.cc:237] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.758626 24562 leader_election.cc:290] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 election: Requested vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237)
I20260709 13:23:23.780649 24549 leader_election.cc:290] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:23.787716 24394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef9538c5bb80422eb571d567b173d003" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d" is_pre_election: true
W20260709 13:23:23.790961 24213 leader_election.cc:343] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:23.802006 24581 consensus_queue.cc:1048] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:23.803086 24247 raft_consensus.cc:2468] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 1.
I20260709 13:23:23.807505 24552 raft_consensus.cc:515] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.841094 24142 catalog_manager.cc:5697] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67). New cstate: current_term: 1 leader_uuid: "3aa42500bbcd4121a43413ecd5be246d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:23.851432 24540 raft_consensus.cc:493] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:23.879632 24540 raft_consensus.cc:515] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.880411 24320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59ddbccb19fc424c95e59a4455cea713" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219"
I20260709 13:23:23.881145 24320 raft_consensus.cc:3060] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:23.890563 24540 leader_election.cc:290] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:23.895805 24320 raft_consensus.cc:2468] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 1.
I20260709 13:23:23.855690 24529 raft_consensus.cc:515] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.857955 24561 raft_consensus.cc:493] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:23.900519 24561 raft_consensus.cc:515] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:23.902354 24529 leader_election.cc:290] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 election: Requested vote from peers d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:23.895695 24395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "192a3efb7b804c38bca2ad2df1ed1188" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d" is_pre_election: true
I20260709 13:23:23.912978 24395 raft_consensus.cc:2468] T 192a3efb7b804c38bca2ad2df1ed1188 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 0.
I20260709 13:23:23.913046 24321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef9538c5bb80422eb571d567b173d003" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219"
I20260709 13:23:23.914022 24321 raft_consensus.cc:3060] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:23.914580 24288 leader_election.cc:304] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:23.918104 24540 raft_consensus.cc:2804] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:23.918658 24540 raft_consensus.cc:493] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:23.919149 24540 raft_consensus.cc:3060] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:23.896790 24247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "192a3efb7b804c38bca2ad2df1ed1188" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50" is_pre_election: true
I20260709 13:23:23.908484 24561 leader_election.cc:290] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237)
I20260709 13:23:23.850605 24577 consensus_queue.cc:1048] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:23.948134 24580 consensus_queue.cc:1048] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:23.953799 24247 raft_consensus.cc:2468] T 192a3efb7b804c38bca2ad2df1ed1188 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 0.
I20260709 13:23:23.910835 24320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48b0cd91c4e04dc7afb4f78ceeebb0b2" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219" is_pre_election: true
I20260709 13:23:23.957988 24320 raft_consensus.cc:2468] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 1.
I20260709 13:23:23.959779 24361 leader_election.cc:304] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:23.961097 24561 raft_consensus.cc:2804] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:23.961711 24561 raft_consensus.cc:493] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:23.962244 24561 raft_consensus.cc:3060] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:23.994225 24393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef9538c5bb80422eb571d567b173d003" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
I20260709 13:23:23.993053 24395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59ddbccb19fc424c95e59a4455cea713" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
I20260709 13:23:23.996392 24395 raft_consensus.cc:3060] T 59ddbccb19fc424c95e59a4455cea713 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:24.008327 24212 leader_election.cc:304] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:23.878718 24552 leader_election.cc:290] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 election: Requested vote from peers d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:24.036314 24543 tablet_bootstrap.cc:492] T 9de1041b009d4b3195afb70e0852c742 P d3ac721977e3467892cbf730c5262219: No bootstrap required, opened a new log
I20260709 13:23:24.036932 24247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48b0cd91c4e04dc7afb4f78ceeebb0b2" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50" is_pre_election: true
I20260709 13:23:23.936698 24540 raft_consensus.cc:515] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.038121 24577 raft_consensus.cc:2804] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:24.038684 24577 raft_consensus.cc:697] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Becoming Leader. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Running, Role: LEADER
I20260709 13:23:24.039610 24577 consensus_queue.cc:237] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.037009 24543 ts_tablet_manager.cc:1403] T 9de1041b009d4b3195afb70e0852c742 P d3ac721977e3467892cbf730c5262219: Time spent bootstrapping tablet: real 0.357s	user 0.027s	sys 0.140s
I20260709 13:23:24.041700 24541 tablet_bootstrap.cc:492] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50: No bootstrap required, opened a new log
I20260709 13:23:24.042241 24541 ts_tablet_manager.cc:1403] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50: Time spent bootstrapping tablet: real 0.333s	user 0.093s	sys 0.135s
I20260709 13:23:24.044757 24543 raft_consensus.cc:359] T 9de1041b009d4b3195afb70e0852c742 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.047142 24552 consensus_queue.cc:1048] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:24.047271 24542 tablet_bootstrap.cc:492] T ef9538c5bb80422eb571d567b173d003 P 3aa42500bbcd4121a43413ecd5be246d: No bootstrap required, opened a new log
I20260709 13:23:24.050238 24542 ts_tablet_manager.cc:1403] T ef9538c5bb80422eb571d567b173d003 P 3aa42500bbcd4121a43413ecd5be246d: Time spent bootstrapping tablet: real 0.417s	user 0.070s	sys 0.127s
W20260709 13:23:24.052743 24213 leader_election.cc:343] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:24.053335 24542 raft_consensus.cc:359] T ef9538c5bb80422eb571d567b173d003 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.053776 24580 consensus_queue.cc:1048] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:24.047672 24543 raft_consensus.cc:385] T 9de1041b009d4b3195afb70e0852c742 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:24.054692 24543 raft_consensus.cc:740] T 9de1041b009d4b3195afb70e0852c742 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3ac721977e3467892cbf730c5262219, State: Initialized, Role: FOLLOWER
I20260709 13:23:24.048437 24540 leader_election.cc:290] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 1 election: Requested vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:24.045414 24541 raft_consensus.cc:359] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.055765 24541 raft_consensus.cc:385] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:24.056054 24541 raft_consensus.cc:740] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Initialized, Role: FOLLOWER
I20260709 13:23:24.056893 24543 consensus_queue.cc:260] T 9de1041b009d4b3195afb70e0852c742 P d3ac721977e3467892cbf730c5262219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.048879 24396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "192a3efb7b804c38bca2ad2df1ed1188" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
I20260709 13:23:24.059213 24396 raft_consensus.cc:3060] T 192a3efb7b804c38bca2ad2df1ed1188 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:24.034574 24395 raft_consensus.cc:2468] T 59ddbccb19fc424c95e59a4455cea713 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 1.
I20260709 13:23:24.046478 24321 raft_consensus.cc:2468] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 1.
I20260709 13:23:24.065860 24541 consensus_queue.cc:260] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.068781 24543 ts_tablet_manager.cc:1434] T 9de1041b009d4b3195afb70e0852c742 P d3ac721977e3467892cbf730c5262219: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:23:24.070253 24541 ts_tablet_manager.cc:1434] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.001s
I20260709 13:23:24.054185 24542 raft_consensus.cc:385] T ef9538c5bb80422eb571d567b173d003 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:24.071051 24542 raft_consensus.cc:740] T ef9538c5bb80422eb571d567b173d003 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Initialized, Role: FOLLOWER
I20260709 13:23:24.071875 24542 consensus_queue.cc:260] T ef9538c5bb80422eb571d567b173d003 P 3aa42500bbcd4121a43413ecd5be246d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.073314 24562 consensus_queue.cc:1048] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [LEADER]: Connected to new peer: Peer: permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:24.074296 24542 ts_tablet_manager.cc:1434] T ef9538c5bb80422eb571d567b173d003 P 3aa42500bbcd4121a43413ecd5be246d: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:23:24.075373 24542 tablet_bootstrap.cc:492] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d: Bootstrap starting.
I20260709 13:23:24.076690 24245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "192a3efb7b804c38bca2ad2df1ed1188" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
I20260709 13:23:24.078177 24396 raft_consensus.cc:2468] T 192a3efb7b804c38bca2ad2df1ed1188 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 1.
I20260709 13:23:24.078789 24245 raft_consensus.cc:3060] T 192a3efb7b804c38bca2ad2df1ed1188 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:24.090022 24288 leader_election.cc:304] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:24.094996 24476 raft_consensus.cc:2804] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:24.095785 24476 raft_consensus.cc:697] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219 [term 1 LEADER]: Becoming Leader. State: Replica: d3ac721977e3467892cbf730c5262219, State: Running, Role: LEADER
I20260709 13:23:24.097041 24476 consensus_queue.cc:237] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.138283 24525 consensus_queue.cc:1048] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:24.139868 24145 catalog_manager.cc:5697] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "66e4fe095997420ba9949bad99bf4d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:24.146613 24577 consensus_queue.cc:1048] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:24.128724 24561 raft_consensus.cc:515] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.146771 24212 leader_election.cc:304] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 3aa42500bbcd4121a43413ecd5be246d
I20260709 13:23:24.156527 24395 raft_consensus.cc:1240] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Rejecting Update request from peer 66e4fe095997420ba9949bad99bf4d50 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:23:24.159336 24542 tablet_bootstrap.cc:654] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d: Neither blocks nor log segments found. Creating new log.
I20260709 13:23:24.161423 24577 raft_consensus.cc:2804] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:24.161749 24549 raft_consensus.cc:493] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:24.162307 24549 raft_consensus.cc:515] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.172570 24561 leader_election.cc:290] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 2 election: Requested vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237)
I20260709 13:23:24.177970 24321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48b0cd91c4e04dc7afb4f78ceeebb0b2" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219"
I20260709 13:23:24.164958 24581 consensus_queue.cc:1059] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:23:24.164173 24245 raft_consensus.cc:2468] T 192a3efb7b804c38bca2ad2df1ed1188 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 1.
I20260709 13:23:24.172672 24549 leader_election.cc:290] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:24.184504 24529 raft_consensus.cc:3055] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:24.185164 24529 raft_consensus.cc:740] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Running, Role: LEADER
I20260709 13:23:24.186030 24529 consensus_queue.cc:260] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.188427 24247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48b0cd91c4e04dc7afb4f78ceeebb0b2" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
I20260709 13:23:24.166986 24577 raft_consensus.cc:697] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Becoming Leader. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Running, Role: LEADER
I20260709 13:23:24.191043 24395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9de1041b009d4b3195afb70e0852c742" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d" is_pre_election: true
I20260709 13:23:24.198860 24318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9de1041b009d4b3195afb70e0852c742" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219" is_pre_election: true
I20260709 13:23:24.199719 24318 raft_consensus.cc:2468] T 9de1041b009d4b3195afb70e0852c742 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 0.
W20260709 13:23:24.202401 24213 leader_election.cc:343] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:23:24.202992 24577 consensus_queue.cc:237] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.216835 24212 leader_election.cc:304] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 3aa42500bbcd4121a43413ecd5be246d
I20260709 13:23:24.218295 24561 consensus_queue.cc:1048] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:24.218858 24581 raft_consensus.cc:2804] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:23:24.219527 24581 raft_consensus.cc:493] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:23:24.219954 24581 raft_consensus.cc:3060] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:24.218902 24145 catalog_manager.cc:5697] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3ac721977e3467892cbf730c5262219 (127.11.63.66). New cstate: current_term: 1 leader_uuid: "d3ac721977e3467892cbf730c5262219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:24.223275 24529 raft_consensus.cc:3060] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:24.262697 24561 consensus_queue.cc:1048] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [LEADER]: Connected to new peer: Peer: permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:24.273104 24247 raft_consensus.cc:2410] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3aa42500bbcd4121a43413ecd5be246d for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:23:24.276518 24361 leader_election.cc:304] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d; no voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219
I20260709 13:23:24.269213 24542 tablet_bootstrap.cc:492] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d: No bootstrap required, opened a new log
I20260709 13:23:24.278028 24562 raft_consensus.cc:2749] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:23:24.278493 24542 ts_tablet_manager.cc:1403] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d: Time spent bootstrapping tablet: real 0.203s	user 0.010s	sys 0.016s
I20260709 13:23:24.288348 24581 raft_consensus.cc:515] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.292583 24143 catalog_manager.cc:5697] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "66e4fe095997420ba9949bad99bf4d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:24.302505 24542 raft_consensus.cc:359] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.303366 24542 raft_consensus.cc:385] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:23:24.303644 24542 raft_consensus.cc:740] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Initialized, Role: FOLLOWER
I20260709 13:23:24.307313 24319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9de1041b009d4b3195afb70e0852c742" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219"
I20260709 13:23:24.307566 24394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9de1041b009d4b3195afb70e0852c742" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
I20260709 13:23:24.308207 24319 raft_consensus.cc:3060] T 9de1041b009d4b3195afb70e0852c742 P d3ac721977e3467892cbf730c5262219 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:24.309674 24581 leader_election.cc:290] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 election: Requested vote from peers d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:24.310624 24542 consensus_queue.cc:260] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.313094 24394 raft_consensus.cc:3060] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:24.313252 24542 ts_tablet_manager.cc:1434] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 13:23:24.317066 24319 raft_consensus.cc:2468] T 9de1041b009d4b3195afb70e0852c742 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 1.
I20260709 13:23:24.318395 24212 leader_election.cc:304] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:24.319363 24581 raft_consensus.cc:2804] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:23:24.319876 24581 raft_consensus.cc:697] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Becoming Leader. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Running, Role: LEADER
I20260709 13:23:24.320681 24581 consensus_queue.cc:237] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.327373 24394 raft_consensus.cc:2468] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 1.
I20260709 13:23:24.359760 24142 catalog_manager.cc:5697] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65). New cstate: current_term: 1 leader_uuid: "66e4fe095997420ba9949bad99bf4d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:24.490494 24476 consensus_queue.cc:1048] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:24.500193 24562 raft_consensus.cc:493] T ef9538c5bb80422eb571d567b173d003 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:24.500790 24562 raft_consensus.cc:515] T ef9538c5bb80422eb571d567b173d003 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.503073 24562 leader_election.cc:290] T ef9538c5bb80422eb571d567b173d003 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237)
I20260709 13:23:24.508275 24319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef9538c5bb80422eb571d567b173d003" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219" is_pre_election: true
I20260709 13:23:24.509397 24319 raft_consensus.cc:2393] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3aa42500bbcd4121a43413ecd5be246d in current term 1: Already voted for candidate 66e4fe095997420ba9949bad99bf4d50 in this term.
I20260709 13:23:24.513690 24245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef9538c5bb80422eb571d567b173d003" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50" is_pre_election: true
I20260709 13:23:24.515743 24361 leader_election.cc:304] T ef9538c5bb80422eb571d567b173d003 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d; no voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219
I20260709 13:23:24.517637 24562 raft_consensus.cc:3060] T ef9538c5bb80422eb571d567b173d003 P 3aa42500bbcd4121a43413ecd5be246d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:23:24.531697 24540 consensus_queue.cc:1048] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:24.556257 24562 raft_consensus.cc:2749] T ef9538c5bb80422eb571d567b173d003 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:23:24.571837 24581 consensus_queue.cc:1048] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:24.591887 24549 consensus_queue.cc:1048] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:24.693172 24394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a38a604d77f4400b910e48e3e9adc54"
dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
mode: GRACEFUL
new_leader_uuid: "66e4fe095997420ba9949bad99bf4d50"
 from {username='slave'} at 127.0.0.1:56354
I20260709 13:23:24.694098 24394 raft_consensus.cc:606] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [term 1 LEADER]: Received request to transfer leadership to TS 66e4fe095997420ba9949bad99bf4d50
I20260709 13:23:24.700091 24394 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32ccadb88b4348f7ab9d7a50b967c3d6"
dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
mode: GRACEFUL
new_leader_uuid: "d3ac721977e3467892cbf730c5262219"
 from {username='slave'} at 127.0.0.1:56354
I20260709 13:23:24.700702 24394 raft_consensus.cc:606] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [term 1 LEADER]: Received request to transfer leadership to TS d3ac721977e3467892cbf730c5262219
I20260709 13:23:24.705912 24245 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59ddbccb19fc424c95e59a4455cea713"
dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
mode: GRACEFUL
new_leader_uuid: "d3ac721977e3467892cbf730c5262219"
 from {username='slave'} at 127.0.0.1:56902
I20260709 13:23:24.706647 24245 raft_consensus.cc:606] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Received request to transfer leadership to TS d3ac721977e3467892cbf730c5262219
I20260709 13:23:24.710350 24525 raft_consensus.cc:993] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d: : Instructing follower 66e4fe095997420ba9949bad99bf4d50 to start an election
I20260709 13:23:24.710834 24562 raft_consensus.cc:1081] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [term 1 LEADER]: Signalling peer 66e4fe095997420ba9949bad99bf4d50 to start an election
I20260709 13:23:24.712442 24245 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a38a604d77f4400b910e48e3e9adc54"
dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
 from {username='slave'} at 127.0.0.1:56886
I20260709 13:23:24.713014 24245 raft_consensus.cc:493] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:24.713387 24245 raft_consensus.cc:3060] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:24.718701 24247 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef9538c5bb80422eb571d567b173d003"
dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
mode: GRACEFUL
new_leader_uuid: "d3ac721977e3467892cbf730c5262219"
 from {username='slave'} at 127.0.0.1:56902
I20260709 13:23:24.719415 24247 raft_consensus.cc:606] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Received request to transfer leadership to TS d3ac721977e3467892cbf730c5262219
I20260709 13:23:24.721652 24245 raft_consensus.cc:515] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.724720 24245 leader_election.cc:290] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 2 election: Requested vote from peers d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:24.724817 24319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a38a604d77f4400b910e48e3e9adc54" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3ac721977e3467892cbf730c5262219"
I20260709 13:23:24.724977 24394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a38a604d77f4400b910e48e3e9adc54" candidate_uuid: "66e4fe095997420ba9949bad99bf4d50" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
I20260709 13:23:24.725461 24319 raft_consensus.cc:3060] T 3a38a604d77f4400b910e48e3e9adc54 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:24.725795 24394 raft_consensus.cc:3055] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:24.726230 24394 raft_consensus.cc:740] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Running, Role: LEADER
I20260709 13:23:24.727181 24394 consensus_queue.cc:260] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.728709 24394 raft_consensus.cc:3060] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:24.732478 24319 raft_consensus.cc:2468] T 3a38a604d77f4400b910e48e3e9adc54 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 2.
I20260709 13:23:24.735401 24245 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9de1041b009d4b3195afb70e0852c742"
dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
mode: GRACEFUL
new_leader_uuid: "3aa42500bbcd4121a43413ecd5be246d"
 from {username='slave'} at 127.0.0.1:56902
I20260709 13:23:24.735692 24212 leader_election.cc:304] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:24.736713 24581 raft_consensus.cc:2804] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:24.737213 24581 raft_consensus.cc:697] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50 [term 2 LEADER]: Becoming Leader. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Running, Role: LEADER
I20260709 13:23:24.738143 24581 consensus_queue.cc:237] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.740041 24394 raft_consensus.cc:2468] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66e4fe095997420ba9949bad99bf4d50 in term 2.
I20260709 13:23:24.736055 24245 raft_consensus.cc:606] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Received request to transfer leadership to TS 3aa42500bbcd4121a43413ecd5be246d
I20260709 13:23:24.746006 24143 catalog_manager.cc:5697] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50 reported cstate change: term changed from 1 to 2, leader changed from 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67) to 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65). New cstate: current_term: 2 leader_uuid: "66e4fe095997420ba9949bad99bf4d50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:24.766004 24562 raft_consensus.cc:993] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d: : Instructing follower d3ac721977e3467892cbf730c5262219 to start an election
I20260709 13:23:24.766513 24561 raft_consensus.cc:1081] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [term 1 LEADER]: Signalling peer d3ac721977e3467892cbf730c5262219 to start an election
I20260709 13:23:24.768573 24319 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32ccadb88b4348f7ab9d7a50b967c3d6"
dest_uuid: "d3ac721977e3467892cbf730c5262219"
 from {username='slave'} at 127.0.0.1:48130
I20260709 13:23:24.769243 24319 raft_consensus.cc:493] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:24.769685 24319 raft_consensus.cc:3060] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:24.779268 24319 raft_consensus.cc:515] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.782533 24319 leader_election.cc:290] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 2 election: Requested vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:24.782826 24245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32ccadb88b4348f7ab9d7a50b967c3d6" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
I20260709 13:23:24.783223 24394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32ccadb88b4348f7ab9d7a50b967c3d6" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
I20260709 13:23:24.783666 24245 raft_consensus.cc:3060] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:24.783805 24394 raft_consensus.cc:3055] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:24.784198 24394 raft_consensus.cc:740] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Running, Role: LEADER
I20260709 13:23:24.785051 24394 consensus_queue.cc:260] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.786303 24394 raft_consensus.cc:3060] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:24.791337 24245 raft_consensus.cc:2468] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 2.
I20260709 13:23:24.792876 24287 leader_election.cc:304] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:24.793761 24602 raft_consensus.cc:2804] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:24.794201 24602 raft_consensus.cc:697] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219 [term 2 LEADER]: Becoming Leader. State: Replica: d3ac721977e3467892cbf730c5262219, State: Running, Role: LEADER
I20260709 13:23:24.794857 24602 consensus_queue.cc:237] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.795126 24394 raft_consensus.cc:2468] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 2.
I20260709 13:23:24.798699 24581 consensus_queue.cc:1048] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:24.803707 24143 catalog_manager.cc:5697] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219 reported cstate change: term changed from 1 to 2, leader changed from 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67) to d3ac721977e3467892cbf730c5262219 (127.11.63.66). New cstate: current_term: 2 leader_uuid: "d3ac721977e3467892cbf730c5262219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:24.806869 24577 raft_consensus.cc:993] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50: : Instructing follower d3ac721977e3467892cbf730c5262219 to start an election
I20260709 13:23:24.807615 24577 raft_consensus.cc:1081] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Signalling peer d3ac721977e3467892cbf730c5262219 to start an election
I20260709 13:23:24.810806 24321 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef9538c5bb80422eb571d567b173d003"
dest_uuid: "d3ac721977e3467892cbf730c5262219"
 from {username='slave'} at 127.0.0.1:48128
I20260709 13:23:24.812186 24321 raft_consensus.cc:493] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:24.812669 24321 raft_consensus.cc:3060] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:24.817615 24529 consensus_queue.cc:1048] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:24.830940 24321 raft_consensus.cc:515] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.834106 24321 leader_election.cc:290] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 2 election: Requested vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:24.839704 24245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef9538c5bb80422eb571d567b173d003" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
I20260709 13:23:24.840502 24245 raft_consensus.cc:3055] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:24.840901 24245 raft_consensus.cc:740] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Running, Role: LEADER
I20260709 13:23:24.842020 24394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef9538c5bb80422eb571d567b173d003" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
I20260709 13:23:24.841868 24245 consensus_queue.cc:260] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.842778 24394 raft_consensus.cc:3060] T ef9538c5bb80422eb571d567b173d003 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:24.843472 24245 raft_consensus.cc:3060] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:24.852105 24394 raft_consensus.cc:2468] T ef9538c5bb80422eb571d567b173d003 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 2.
I20260709 13:23:24.853267 24288 leader_election.cc:304] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:24.854020 24602 raft_consensus.cc:2804] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:24.854491 24602 raft_consensus.cc:697] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219 [term 2 LEADER]: Becoming Leader. State: Replica: d3ac721977e3467892cbf730c5262219, State: Running, Role: LEADER
I20260709 13:23:24.855571 24602 consensus_queue.cc:237] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.855600 24245 raft_consensus.cc:2468] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 2.
I20260709 13:23:24.863653 24143 catalog_manager.cc:5697] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219 reported cstate change: term changed from 1 to 2, leader changed from 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65) to d3ac721977e3467892cbf730c5262219 (127.11.63.66). New cstate: current_term: 2 leader_uuid: "d3ac721977e3467892cbf730c5262219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:24.895829 24577 consensus_queue.cc:1048] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:24.902443 24529 raft_consensus.cc:993] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50: : Instructing follower 3aa42500bbcd4121a43413ecd5be246d to start an election
I20260709 13:23:24.902916 24529 raft_consensus.cc:1081] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Signalling peer 3aa42500bbcd4121a43413ecd5be246d to start an election
I20260709 13:23:24.904896 24395 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9de1041b009d4b3195afb70e0852c742"
dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
 from {username='slave'} at 127.0.0.1:56328
I20260709 13:23:24.905402 24395 raft_consensus.cc:493] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:24.905771 24395 raft_consensus.cc:3060] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:24.910501 24529 consensus_queue.cc:1048] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:23:24.913390 24395 raft_consensus.cc:515] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.917212 24245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9de1041b009d4b3195afb70e0852c742" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
I20260709 13:23:24.917475 24319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9de1041b009d4b3195afb70e0852c742" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3ac721977e3467892cbf730c5262219"
I20260709 13:23:24.917265 24394 raft_consensus.cc:1240] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Rejecting Update request from peer 66e4fe095997420ba9949bad99bf4d50 for earlier term 1. Current term is 2. Ops: []
I20260709 13:23:24.918640 24395 leader_election.cc:290] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 2 election: Requested vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237)
I20260709 13:23:24.918274 24245 raft_consensus.cc:3055] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:24.919308 24245 raft_consensus.cc:740] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Running, Role: LEADER
I20260709 13:23:24.920280 24245 consensus_queue.cc:260] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.921365 24245 raft_consensus.cc:3060] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:24.932475 24245 raft_consensus.cc:2468] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 2.
I20260709 13:23:24.933693 24361 leader_election.cc:304] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, 66e4fe095997420ba9949bad99bf4d50; no voters: d3ac721977e3467892cbf730c5262219
I20260709 13:23:24.934568 24561 raft_consensus.cc:2804] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:24.935163 24561 raft_consensus.cc:697] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d [term 2 LEADER]: Becoming Leader. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Running, Role: LEADER
I20260709 13:23:24.936133 24561 consensus_queue.cc:237] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:24.945086 24143 catalog_manager.cc:5697] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d reported cstate change: term changed from 1 to 2, leader changed from 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65) to 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67). New cstate: current_term: 2 leader_uuid: "3aa42500bbcd4121a43413ecd5be246d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:25.003927 24549 raft_consensus.cc:993] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50: : Instructing follower d3ac721977e3467892cbf730c5262219 to start an election
I20260709 13:23:25.004484 24529 raft_consensus.cc:1081] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Signalling peer d3ac721977e3467892cbf730c5262219 to start an election
I20260709 13:23:25.006335 24321 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59ddbccb19fc424c95e59a4455cea713"
dest_uuid: "d3ac721977e3467892cbf730c5262219"
 from {username='slave'} at 127.0.0.1:48128
I20260709 13:23:25.006915 24321 raft_consensus.cc:493] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:25.007251 24321 raft_consensus.cc:3060] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:25.013252 24321 raft_consensus.cc:515] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:25.015074 24321 leader_election.cc:290] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 2 election: Requested vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:25.016050 24395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59ddbccb19fc424c95e59a4455cea713" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
I20260709 13:23:25.015937 24245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59ddbccb19fc424c95e59a4455cea713" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
I20260709 13:23:25.016810 24395 raft_consensus.cc:3060] T 59ddbccb19fc424c95e59a4455cea713 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:25.016952 24245 raft_consensus.cc:3055] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:25.017760 24245 raft_consensus.cc:740] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Running, Role: LEADER
I20260709 13:23:25.018605 24245 consensus_queue.cc:260] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:25.019858 24245 raft_consensus.cc:3060] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:25.023785 24395 raft_consensus.cc:2468] T 59ddbccb19fc424c95e59a4455cea713 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 2.
I20260709 13:23:25.025130 24288 leader_election.cc:304] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, d3ac721977e3467892cbf730c5262219; no voters: 
I20260709 13:23:25.026023 24603 raft_consensus.cc:2804] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:25.026604 24603 raft_consensus.cc:697] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219 [term 2 LEADER]: Becoming Leader. State: Replica: d3ac721977e3467892cbf730c5262219, State: Running, Role: LEADER
I20260709 13:23:25.027542 24245 raft_consensus.cc:2468] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 2.
I20260709 13:23:25.027585 24603 consensus_queue.cc:237] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:25.038784 24143 catalog_manager.cc:5697] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219 reported cstate change: term changed from 1 to 2, leader changed from 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65) to d3ac721977e3467892cbf730c5262219 (127.11.63.66). New cstate: current_term: 2 leader_uuid: "d3ac721977e3467892cbf730c5262219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:25.152755 24321 raft_consensus.cc:1275] T 3a38a604d77f4400b910e48e3e9adc54 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Refusing update from remote peer 66e4fe095997420ba9949bad99bf4d50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:25.154284 24549 consensus_queue.cc:1048] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:25.166608 24395 raft_consensus.cc:1275] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Refusing update from remote peer 66e4fe095997420ba9949bad99bf4d50: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:25.168857 24581 consensus_queue.cc:1048] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:25.239825 24245 raft_consensus.cc:1275] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Refusing update from remote peer d3ac721977e3467892cbf730c5262219: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:25.241206 24603 consensus_queue.cc:1048] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:25.251296 24247 raft_consensus.cc:1275] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Refusing update from remote peer d3ac721977e3467892cbf730c5262219: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:25.252859 24611 consensus_queue.cc:1048] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:25.259390 24395 raft_consensus.cc:1275] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Refusing update from remote peer d3ac721977e3467892cbf730c5262219: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:25.268227 24395 raft_consensus.cc:1275] T ef9538c5bb80422eb571d567b173d003 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Refusing update from remote peer d3ac721977e3467892cbf730c5262219: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:25.271231 24602 consensus_queue.cc:1048] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:25.271510 24611 consensus_queue.cc:1048] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:23:25.320060 24245 raft_consensus.cc:1275] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Refusing update from remote peer 3aa42500bbcd4121a43413ecd5be246d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:25.321704 24561 consensus_queue.cc:1048] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d [LEADER]: Connected to new peer: Peer: permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:25.342456 24321 raft_consensus.cc:3060] T 9de1041b009d4b3195afb70e0852c742 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:25.349071 24321 raft_consensus.cc:1275] T 9de1041b009d4b3195afb70e0852c742 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Refusing update from remote peer 3aa42500bbcd4121a43413ecd5be246d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:25.350425 24613 consensus_queue.cc:1048] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:25.421271 24395 raft_consensus.cc:1275] T 59ddbccb19fc424c95e59a4455cea713 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Refusing update from remote peer d3ac721977e3467892cbf730c5262219: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:25.422927 24611 consensus_queue.cc:1048] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:25.432753 24245 raft_consensus.cc:1275] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Refusing update from remote peer d3ac721977e3467892cbf730c5262219: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:25.434685 24611 consensus_queue.cc:1048] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:25.550532 24611 raft_consensus.cc:493] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 66e4fe095997420ba9949bad99bf4d50)
I20260709 13:23:25.551129 24611 raft_consensus.cc:515] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:25.552795 24611 leader_election.cc:290] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:25.553785 24395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48b0cd91c4e04dc7afb4f78ceeebb0b2" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d" is_pre_election: true
I20260709 13:23:25.553910 24245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48b0cd91c4e04dc7afb4f78ceeebb0b2" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50" is_pre_election: true
I20260709 13:23:25.554548 24395 raft_consensus.cc:2393] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3ac721977e3467892cbf730c5262219 in current term 2: Already voted for candidate 3aa42500bbcd4121a43413ecd5be246d in this term.
I20260709 13:23:25.554705 24245 raft_consensus.cc:2468] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 2.
I20260709 13:23:25.555960 24287 leader_election.cc:304] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 3aa42500bbcd4121a43413ecd5be246d
I20260709 13:23:25.556999 24611 raft_consensus.cc:2804] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:23:25.557328 24611 raft_consensus.cc:493] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 66e4fe095997420ba9949bad99bf4d50)
I20260709 13:23:25.557633 24611 raft_consensus.cc:3060] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:25.563477 24611 raft_consensus.cc:515] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:25.565372 24611 leader_election.cc:290] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 2 election: Requested vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67:41619)
I20260709 13:23:25.566398 24395 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48b0cd91c4e04dc7afb4f78ceeebb0b2" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
I20260709 13:23:25.566239 24245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48b0cd91c4e04dc7afb4f78ceeebb0b2" candidate_uuid: "d3ac721977e3467892cbf730c5262219" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
I20260709 13:23:25.567178 24395 raft_consensus.cc:2393] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d3ac721977e3467892cbf730c5262219 in current term 2: Already voted for candidate 3aa42500bbcd4121a43413ecd5be246d in this term.
I20260709 13:23:25.573812 24245 raft_consensus.cc:2468] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ac721977e3467892cbf730c5262219 in term 2.
I20260709 13:23:25.575022 24287 leader_election.cc:304] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219; no voters: 3aa42500bbcd4121a43413ecd5be246d
I20260709 13:23:25.576143 24611 raft_consensus.cc:2804] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:25.576612 24611 raft_consensus.cc:697] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [term 2 LEADER]: Becoming Leader. State: Replica: d3ac721977e3467892cbf730c5262219, State: Running, Role: LEADER
I20260709 13:23:25.577335 24611 consensus_queue.cc:237] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:25.587096 24142 catalog_manager.cc:5697] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 reported cstate change: term changed from 1 to 2, leader changed from 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65) to d3ac721977e3467892cbf730c5262219 (127.11.63.66). New cstate: current_term: 2 leader_uuid: "d3ac721977e3467892cbf730c5262219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: UNKNOWN } } }
I20260709 13:23:25.915387 24613 raft_consensus.cc:493] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:23:25.916102 24613 raft_consensus.cc:515] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:25.918294 24613 leader_election.cc:290] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237)
I20260709 13:23:25.919293 24245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48b0cd91c4e04dc7afb4f78ceeebb0b2" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66e4fe095997420ba9949bad99bf4d50" is_pre_election: true
I20260709 13:23:25.920112 24321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48b0cd91c4e04dc7afb4f78ceeebb0b2" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3ac721977e3467892cbf730c5262219" is_pre_election: true
I20260709 13:23:25.920385 24245 raft_consensus.cc:2410] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3aa42500bbcd4121a43413ecd5be246d for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:23:25.922324 24361 leader_election.cc:304] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d; no voters: 66e4fe095997420ba9949bad99bf4d50, d3ac721977e3467892cbf730c5262219
I20260709 13:23:25.923568 24613 raft_consensus.cc:2749] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:23:26.069015 24245 raft_consensus.cc:1275] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Refusing update from remote peer d3ac721977e3467892cbf730c5262219: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:26.070598 24611 consensus_queue.cc:1048] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:26.090726 24395 raft_consensus.cc:1275] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Refusing update from remote peer d3ac721977e3467892cbf730c5262219: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:26.094297 24611 consensus_queue.cc:1048] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:23:27.751750 24142 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49776:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:23:27.772891 24142 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:49776:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:23:27.853730 11517 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:23:27.944602 24671 tablet.cc:1724] T 59ddbccb19fc424c95e59a4455cea713 P 3aa42500bbcd4121a43413ecd5be246d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:27.909761 24664 tablet.cc:1724] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:23:27.978200 24213 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41619, user_credentials={real_user=slave}} blocked reactor thread for 78759.5us
I20260709 13:23:27.982436 24685 tablet.cc:1724] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:27.984110 24683 tablet.cc:1724] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:27.992895 24687 tablet.cc:1724] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:28.009692 24689 tablet.cc:1724] T 337bdcde8c194dccb8aee8a99e36a997 P 3aa42500bbcd4121a43413ecd5be246d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:23:28.023463 24361 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:36043, user_credentials={real_user=slave}} blocked reactor thread for 43955.7us
W20260709 13:23:28.024263 24287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.65:36043, user_credentials={real_user=slave}} blocked reactor thread for 70132.7us
I20260709 13:23:28.029208 24670 tablet.cc:1724] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:28.057232 24322 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "33f74f7b25d0436fa15cc6053960e3a3"
dest_uuid: "d3ac721977e3467892cbf730c5262219"
mode: GRACEFUL
new_leader_uuid: "3aa42500bbcd4121a43413ecd5be246d"
 from {username='slave'} at 127.0.0.1:34490
I20260709 13:23:28.061913 24322 raft_consensus.cc:606] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [term 2 LEADER]: Received request to transfer leadership to TS 3aa42500bbcd4121a43413ecd5be246d
I20260709 13:23:28.073766 24663 tablet.cc:1724] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:28.074272 24690 tablet.cc:1724] T 3a38a604d77f4400b910e48e3e9adc54 P d3ac721977e3467892cbf730c5262219: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:28.129808 24692 tablet.cc:1724] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:28.164351 24685 tablet.cc:1724] T 192a3efb7b804c38bca2ad2df1ed1188 P 3aa42500bbcd4121a43413ecd5be246d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:23:28.166343 24691 log.cc:927] Time spent T 5a467f6687c5464e8c8ea4429a2f0e4b P d3ac721977e3467892cbf730c5262219: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:23:28.171747 24694 tablet.cc:1724] T 337bdcde8c194dccb8aee8a99e36a997 P d3ac721977e3467892cbf730c5262219: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:28.166622 24683 tablet.cc:1724] T 9de1041b009d4b3195afb70e0852c742 P d3ac721977e3467892cbf730c5262219: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:28.179670 24687 tablet.cc:1724] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:28.093019 24664 tablet.cc:1724] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:28.190552 24671 tablet.cc:1724] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:28.194871 24696 tablet.cc:1724] T 192a3efb7b804c38bca2ad2df1ed1188 P 66e4fe095997420ba9949bad99bf4d50: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:28.169611 24690 tablet.cc:1724] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:28.203380 24663 tablet.cc:1724] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:28.238323 24245 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a91cd6890c73435e9b43afc4262d4f01"
dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
mode: GRACEFUL
new_leader_uuid: "3aa42500bbcd4121a43413ecd5be246d"
 from {username='slave'} at 127.0.0.1:60804
I20260709 13:23:28.239045 24245 raft_consensus.cc:606] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Received request to transfer leadership to TS 3aa42500bbcd4121a43413ecd5be246d
I20260709 13:23:28.240576 11517 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:23:28.241211 11517 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:23:28.240959 24687 tablet.cc:1724] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:23:28.253907 24288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.63.67:41619, user_credentials={real_user=slave}} blocked reactor thread for 125547us
I20260709 13:23:28.266139 24694 tablet.cc:1724] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:28.269083 24672 raft_consensus.cc:993] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219: : Instructing follower 3aa42500bbcd4121a43413ecd5be246d to start an election
I20260709 13:23:28.266974 24143 catalog_manager.cc:5789] TS d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237) does not have the latest schema for tablet 48b0cd91c4e04dc7afb4f78ceeebb0b2 (table soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]). Expected version 1 got 0
I20260709 13:23:28.270004 24672 raft_consensus.cc:1081] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [term 2 LEADER]: Signalling peer 3aa42500bbcd4121a43413ecd5be246d to start an election
I20260709 13:23:28.270125 24143 catalog_manager.cc:5789] TS d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237) does not have the latest schema for tablet ef9538c5bb80422eb571d567b173d003 (table soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]). Expected version 1 got 0
I20260709 13:23:28.278525 24692 tablet.cc:1724] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:28.280834 24143 catalog_manager.cc:5789] TS d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237) does not have the latest schema for tablet 32ccadb88b4348f7ab9d7a50b967c3d6 (table soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5]). Expected version 1 got 0
I20260709 13:23:28.273245 24670 tablet.cc:1724] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:28.285897 24687 tablet.cc:1724] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 66e4fe095997420ba9949bad99bf4d50: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:28.290092 24688 raft_consensus.cc:993] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50: : Instructing follower 3aa42500bbcd4121a43413ecd5be246d to start an election
I20260709 13:23:28.291766 24688 raft_consensus.cc:1081] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Signalling peer 3aa42500bbcd4121a43413ecd5be246d to start an election
I20260709 13:23:28.292075 24391 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "33f74f7b25d0436fa15cc6053960e3a3"
dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
 from {username='slave'} at 127.0.0.1:56340
I20260709 13:23:28.292691 24391 raft_consensus.cc:493] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:28.293048 24391 raft_consensus.cc:3060] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:28.293535 24393 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a91cd6890c73435e9b43afc4262d4f01"
dest_uuid: "3aa42500bbcd4121a43413ecd5be246d"
 from {username='slave'} at 127.0.0.1:56328
I20260709 13:23:28.294170 24393 raft_consensus.cc:493] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:23:28.294516 24393 raft_consensus.cc:3060] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:28.303774 24393 raft_consensus.cc:515] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:28.304981 24671 tablet.cc:1724] T ef9538c5bb80422eb571d567b173d003 P 3aa42500bbcd4121a43413ecd5be246d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:28.305473 24391 raft_consensus.cc:515] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:28.306767 24664 tablet.cc:1724] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:28.322489 24391 leader_election.cc:290] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 3 election: Requested vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237)
I20260709 13:23:28.322944 24393 leader_election.cc:290] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 2 election: Requested vote from peers 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043), d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237)
I20260709 13:23:28.308508 24690 tablet.cc:1724] T 5a467f6687c5464e8c8ea4429a2f0e4b P d3ac721977e3467892cbf730c5262219: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:23:28.328158 24319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33f74f7b25d0436fa15cc6053960e3a3" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d3ac721977e3467892cbf730c5262219"
I20260709 13:23:28.328758 24319 raft_consensus.cc:3055] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:23:28.329069 24319 raft_consensus.cc:740] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d3ac721977e3467892cbf730c5262219, State: Running, Role: LEADER
I20260709 13:23:28.329892 24319 consensus_queue.cc:260] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:28.331153 24319 raft_consensus.cc:3060] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:28.333951 24322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a91cd6890c73435e9b43afc4262d4f01" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d3ac721977e3467892cbf730c5262219"
I20260709 13:23:28.334619 24322 raft_consensus.cc:3060] T a91cd6890c73435e9b43afc4262d4f01 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:28.335347 24247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33f74f7b25d0436fa15cc6053960e3a3" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
I20260709 13:23:28.335886 24247 raft_consensus.cc:3060] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:23:28.338109 24244 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a91cd6890c73435e9b43afc4262d4f01" candidate_uuid: "3aa42500bbcd4121a43413ecd5be246d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "66e4fe095997420ba9949bad99bf4d50"
I20260709 13:23:28.338721 24244 raft_consensus.cc:3055] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:23:28.339103 24244 raft_consensus.cc:740] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 66e4fe095997420ba9949bad99bf4d50, State: Running, Role: LEADER
I20260709 13:23:28.339955 24244 consensus_queue.cc:260] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:28.341228 24244 raft_consensus.cc:3060] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:23:28.345866 24247 raft_consensus.cc:2468] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 3.
I20260709 13:23:28.347184 24361 leader_election.cc:304] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, 66e4fe095997420ba9949bad99bf4d50; no voters: 
I20260709 13:23:28.348647 24686 raft_consensus.cc:2804] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:23:28.349169 24686 raft_consensus.cc:697] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [term 3 LEADER]: Becoming Leader. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Running, Role: LEADER
I20260709 13:23:28.350570 24686 consensus_queue.cc:237] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:28.351361 24244 raft_consensus.cc:2468] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 2.
I20260709 13:23:28.352723 24361 leader_election.cc:304] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aa42500bbcd4121a43413ecd5be246d, 66e4fe095997420ba9949bad99bf4d50; no voters: 
I20260709 13:23:28.353616 24322 raft_consensus.cc:2468] T a91cd6890c73435e9b43afc4262d4f01 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 2.
I20260709 13:23:28.353703 24682 raft_consensus.cc:2804] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:23:28.354353 24682 raft_consensus.cc:697] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d [term 2 LEADER]: Becoming Leader. State: Replica: 3aa42500bbcd4121a43413ecd5be246d, State: Running, Role: LEADER
I20260709 13:23:28.355219 24682 consensus_queue.cc:237] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } }
I20260709 13:23:28.357761 24319 raft_consensus.cc:2468] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aa42500bbcd4121a43413ecd5be246d in term 3.
I20260709 13:23:28.366608 24143 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261: soft_deleted_table [id=b7d88905a72a426fb7b66c8fcdb3b9e5] alter complete (version 1)
I20260709 13:23:28.375716 24143 catalog_manager.cc:5697] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d reported cstate change: term changed from 2 to 3, leader changed from d3ac721977e3467892cbf730c5262219 (127.11.63.66) to 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67). New cstate: current_term: 3 leader_uuid: "3aa42500bbcd4121a43413ecd5be246d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:28.377722 24143 catalog_manager.cc:5697] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d reported cstate change: term changed from 1 to 2, leader changed from 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65) to 3aa42500bbcd4121a43413ecd5be246d (127.11.63.67). New cstate: current_term: 2 leader_uuid: "3aa42500bbcd4121a43413ecd5be246d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aa42500bbcd4121a43413ecd5be246d" member_type: VOTER last_known_addr { host: "127.11.63.67" port: 41619 } health_report { overall_health: HEALTHY } } }
I20260709 13:23:28.735723 24247 raft_consensus.cc:1275] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [term 3 FOLLOWER]: Refusing update from remote peer 3aa42500bbcd4121a43413ecd5be246d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:23:28.737727 24682 consensus_queue.cc:1048] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [LEADER]: Connected to new peer: Peer: permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:23:28.756175 24319 raft_consensus.cc:1275] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [term 3 FOLLOWER]: Refusing update from remote peer 3aa42500bbcd4121a43413ecd5be246d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:23:28.757925 24682 consensus_queue.cc:1048] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:23:28.869119 24247 raft_consensus.cc:1275] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Refusing update from remote peer 3aa42500bbcd4121a43413ecd5be246d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:28.870620 24682 consensus_queue.cc:1048] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d [LEADER]: Connected to new peer: Peer: permanent_uuid: "66e4fe095997420ba9949bad99bf4d50" member_type: VOTER last_known_addr { host: "127.11.63.65" port: 36043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:28.881505 24319 raft_consensus.cc:1275] T a91cd6890c73435e9b43afc4262d4f01 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Refusing update from remote peer 3aa42500bbcd4121a43413ecd5be246d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:23:28.883034 24682 consensus_queue.cc:1048] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3ac721977e3467892cbf730c5262219" member_type: VOTER last_known_addr { host: "127.11.63.66" port: 40237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:23:29.312490 11517 tablet_server.cc:179] TabletServer@127.11.63.65:0 shutting down...
I20260709 13:23:29.338783 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:23:29.339922 11517 tablet_replica.cc:333] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50: stopping tablet replica
I20260709 13:23:29.340864 11517 raft_consensus.cc:2243] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.341506 11517 raft_consensus.cc:2272] T 9de1041b009d4b3195afb70e0852c742 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.343497 11517 tablet_replica.cc:333] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 66e4fe095997420ba9949bad99bf4d50: stopping tablet replica
I20260709 13:23:29.344111 11517 raft_consensus.cc:2243] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.344655 11517 raft_consensus.cc:2272] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:29.346874 24361 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 7 similar messages]
I20260709 13:23:29.347579 11517 tablet_replica.cc:333] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50: stopping tablet replica
I20260709 13:23:29.348453 11517 raft_consensus.cc:2243] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.349112 11517 raft_consensus.cc:2272] T ef9538c5bb80422eb571d567b173d003 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:29.350245 24361 consensus_peers.cc:597] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d -> Peer 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043): Couldn't send request to peer 66e4fe095997420ba9949bad99bf4d50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:29.351985 11517 tablet_replica.cc:333] T 192a3efb7b804c38bca2ad2df1ed1188 P 66e4fe095997420ba9949bad99bf4d50: stopping tablet replica
I20260709 13:23:29.352998 11517 raft_consensus.cc:2243] T 192a3efb7b804c38bca2ad2df1ed1188 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.353547 11517 raft_consensus.cc:2272] T 192a3efb7b804c38bca2ad2df1ed1188 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.355376 11517 tablet_replica.cc:333] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50: stopping tablet replica
I20260709 13:23:29.355898 11517 raft_consensus.cc:2243] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:29.356577 11517 raft_consensus.cc:2272] T 337bdcde8c194dccb8aee8a99e36a997 P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.358376 11517 tablet_replica.cc:333] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50: stopping tablet replica
I20260709 13:23:29.358901 11517 raft_consensus.cc:2243] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.359293 11517 raft_consensus.cc:2272] T b027fa0920c8440fab0837dd1142fd57 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.361250 11517 tablet_replica.cc:333] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50: stopping tablet replica
I20260709 13:23:29.361836 11517 raft_consensus.cc:2243] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:29.362594 11517 raft_consensus.cc:2272] T 0f82955421054b95ad8df878b1827aa4 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.364619 11517 tablet_replica.cc:333] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50: stopping tablet replica
I20260709 13:23:29.365134 11517 raft_consensus.cc:2243] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.365538 11517 raft_consensus.cc:2272] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.367794 11517 tablet_replica.cc:333] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50: stopping tablet replica
I20260709 13:23:29.368397 11517 raft_consensus.cc:2243] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.368891 11517 raft_consensus.cc:2272] T 59ddbccb19fc424c95e59a4455cea713 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.370850 11517 tablet_replica.cc:333] T 05cc80ee32e0409c959205fbbcf09e58 P 66e4fe095997420ba9949bad99bf4d50: stopping tablet replica
I20260709 13:23:29.371491 11517 raft_consensus.cc:2243] T 05cc80ee32e0409c959205fbbcf09e58 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.371951 11517 raft_consensus.cc:2272] T 05cc80ee32e0409c959205fbbcf09e58 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.374516 11517 tablet_replica.cc:333] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50: stopping tablet replica
I20260709 13:23:29.375109 11517 raft_consensus.cc:2243] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.375905 11517 raft_consensus.cc:2272] T 33f74f7b25d0436fa15cc6053960e3a3 P 66e4fe095997420ba9949bad99bf4d50 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:29.378116 24361 consensus_peers.cc:597] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d -> Peer 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043): Couldn't send request to peer 66e4fe095997420ba9949bad99bf4d50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:29.379235 11517 tablet_replica.cc:333] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50: stopping tablet replica
I20260709 13:23:29.379854 11517 raft_consensus.cc:2243] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:29.380640 11517 raft_consensus.cc:2272] T 7f2735d0953445d8b941fd6ad09e76fc P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.383229 11517 tablet_replica.cc:333] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50: stopping tablet replica
I20260709 13:23:29.383785 11517 raft_consensus.cc:2243] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.384172 11517 raft_consensus.cc:2272] T a91cd6890c73435e9b43afc4262d4f01 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.387135 11517 tablet_replica.cc:333] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50: stopping tablet replica
I20260709 13:23:29.387660 11517 raft_consensus.cc:2243] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:29.388527 11517 raft_consensus.cc:2272] T 5a467f6687c5464e8c8ea4429a2f0e4b P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.391080 11517 tablet_replica.cc:333] T 32b70a25b0264a8a96170bbecee4837d P 66e4fe095997420ba9949bad99bf4d50: stopping tablet replica
I20260709 13:23:29.391912 11517 raft_consensus.cc:2243] T 32b70a25b0264a8a96170bbecee4837d P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.392575 11517 raft_consensus.cc:2272] T 32b70a25b0264a8a96170bbecee4837d P 66e4fe095997420ba9949bad99bf4d50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.395725 11517 tablet_replica.cc:333] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50: stopping tablet replica
W20260709 13:23:29.395685 24287 consensus_peers.cc:597] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219 -> Peer 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043): Couldn't send request to peer 66e4fe095997420ba9949bad99bf4d50. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:29.396821 11517 raft_consensus.cc:2243] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:29.397822 11517 raft_consensus.cc:2272] T 0689d527efd74dc1a43383e7f3034595 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.400022 11517 tablet_replica.cc:333] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50: stopping tablet replica
I20260709 13:23:29.400543 11517 raft_consensus.cc:2243] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:29.401288 11517 raft_consensus.cc:2272] T 3a38a604d77f4400b910e48e3e9adc54 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.403121 11517 tablet_replica.cc:333] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50: stopping tablet replica
I20260709 13:23:29.403618 11517 raft_consensus.cc:2243] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.404067 11517 raft_consensus.cc:2272] T 0713792c0cb2436b932fadc8d1f428a2 P 66e4fe095997420ba9949bad99bf4d50 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:23:29.452517 24287 consensus_peers.cc:597] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219 -> Peer 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043): Couldn't send request to peer 66e4fe095997420ba9949bad99bf4d50. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36043: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:29.464692 11517 tablet_server.cc:196] TabletServer@127.11.63.65:0 shutdown complete.
I20260709 13:23:29.498364 11517 tablet_server.cc:179] TabletServer@127.11.63.66:0 shutting down...
W20260709 13:23:29.504469 24287 consensus_peers.cc:597] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219 -> Peer 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043): Couldn't send request to peer 66e4fe095997420ba9949bad99bf4d50. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36043: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:23:29.523653 24361 consensus_peers.cc:597] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d -> Peer d3ac721977e3467892cbf730c5262219 (127.11.63.66:40237): Couldn't send request to peer d3ac721977e3467892cbf730c5262219. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:29.537902 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:23:29.538880 11517 tablet_replica.cc:333] T 9de1041b009d4b3195afb70e0852c742 P d3ac721977e3467892cbf730c5262219: stopping tablet replica
W20260709 13:23:29.539122 24287 consensus_peers.cc:597] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219 -> Peer 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043): Couldn't send request to peer 66e4fe095997420ba9949bad99bf4d50. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36043: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:29.540035 11517 raft_consensus.cc:2243] T 9de1041b009d4b3195afb70e0852c742 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.540800 11517 raft_consensus.cc:2272] T 9de1041b009d4b3195afb70e0852c742 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.543366 11517 tablet_replica.cc:333] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219: stopping tablet replica
I20260709 13:23:29.543931 11517 raft_consensus.cc:2243] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:29.544654 11517 raft_consensus.cc:2272] T ef9538c5bb80422eb571d567b173d003 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.546551 11517 tablet_replica.cc:333] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219: stopping tablet replica
I20260709 13:23:29.547051 11517 raft_consensus.cc:2243] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:29.547793 11517 raft_consensus.cc:2272] T 32ccadb88b4348f7ab9d7a50b967c3d6 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.549788 11517 tablet_replica.cc:333] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219: stopping tablet replica
I20260709 13:23:29.550348 11517 raft_consensus.cc:2243] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:29.551096 11517 raft_consensus.cc:2272] T 192a3efb7b804c38bca2ad2df1ed1188 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.553318 11517 tablet_replica.cc:333] T 337bdcde8c194dccb8aee8a99e36a997 P d3ac721977e3467892cbf730c5262219: stopping tablet replica
I20260709 13:23:29.554041 11517 raft_consensus.cc:2243] T 337bdcde8c194dccb8aee8a99e36a997 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.554553 11517 raft_consensus.cc:2272] T 337bdcde8c194dccb8aee8a99e36a997 P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.556952 11517 tablet_replica.cc:333] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219: stopping tablet replica
I20260709 13:23:29.557718 11517 raft_consensus.cc:2243] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:29.558513 11517 raft_consensus.cc:2272] T b027fa0920c8440fab0837dd1142fd57 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.560840 11517 tablet_replica.cc:333] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219: stopping tablet replica
I20260709 13:23:29.561385 11517 raft_consensus.cc:2243] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.561918 11517 raft_consensus.cc:2272] T 0f82955421054b95ad8df878b1827aa4 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.564332 11517 tablet_replica.cc:333] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219: stopping tablet replica
I20260709 13:23:29.564954 11517 raft_consensus.cc:2243] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:29.565795 11517 raft_consensus.cc:2272] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.567946 11517 tablet_replica.cc:333] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219: stopping tablet replica
I20260709 13:23:29.568569 11517 raft_consensus.cc:2243] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:29.569387 11517 raft_consensus.cc:2272] T 59ddbccb19fc424c95e59a4455cea713 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.571343 11517 tablet_replica.cc:333] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219: stopping tablet replica
I20260709 13:23:29.571920 11517 raft_consensus.cc:2243] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:29.572708 11517 raft_consensus.cc:2272] T 05cc80ee32e0409c959205fbbcf09e58 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.574761 11517 tablet_replica.cc:333] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219: stopping tablet replica
I20260709 13:23:29.575336 11517 raft_consensus.cc:2243] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.575776 11517 raft_consensus.cc:2272] T 33f74f7b25d0436fa15cc6053960e3a3 P d3ac721977e3467892cbf730c5262219 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.580054 11517 tablet_replica.cc:333] T 7f2735d0953445d8b941fd6ad09e76fc P d3ac721977e3467892cbf730c5262219: stopping tablet replica
I20260709 13:23:29.581141 11517 raft_consensus.cc:2243] T 7f2735d0953445d8b941fd6ad09e76fc P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.581883 11517 raft_consensus.cc:2272] T 7f2735d0953445d8b941fd6ad09e76fc P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.584223 11517 tablet_replica.cc:333] T a91cd6890c73435e9b43afc4262d4f01 P d3ac721977e3467892cbf730c5262219: stopping tablet replica
I20260709 13:23:29.584782 11517 raft_consensus.cc:2243] T a91cd6890c73435e9b43afc4262d4f01 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.585214 11517 raft_consensus.cc:2272] T a91cd6890c73435e9b43afc4262d4f01 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.587981 11517 tablet_replica.cc:333] T 5a467f6687c5464e8c8ea4429a2f0e4b P d3ac721977e3467892cbf730c5262219: stopping tablet replica
I20260709 13:23:29.588640 11517 raft_consensus.cc:2243] T 5a467f6687c5464e8c8ea4429a2f0e4b P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.589159 11517 raft_consensus.cc:2272] T 5a467f6687c5464e8c8ea4429a2f0e4b P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.591179 11517 tablet_replica.cc:333] T 32b70a25b0264a8a96170bbecee4837d P d3ac721977e3467892cbf730c5262219: stopping tablet replica
I20260709 13:23:29.591848 11517 raft_consensus.cc:2243] T 32b70a25b0264a8a96170bbecee4837d P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.592375 11517 raft_consensus.cc:2272] T 32b70a25b0264a8a96170bbecee4837d P d3ac721977e3467892cbf730c5262219 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.594874 11517 tablet_replica.cc:333] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219: stopping tablet replica
I20260709 13:23:29.595408 11517 raft_consensus.cc:2243] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.595857 11517 raft_consensus.cc:2272] T 0689d527efd74dc1a43383e7f3034595 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.597959 11517 tablet_replica.cc:333] T 3a38a604d77f4400b910e48e3e9adc54 P d3ac721977e3467892cbf730c5262219: stopping tablet replica
I20260709 13:23:29.598448 11517 raft_consensus.cc:2243] T 3a38a604d77f4400b910e48e3e9adc54 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.598879 11517 raft_consensus.cc:2272] T 3a38a604d77f4400b910e48e3e9adc54 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.600616 11517 tablet_replica.cc:333] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219: stopping tablet replica
I20260709 13:23:29.601109 11517 raft_consensus.cc:2243] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:29.601905 11517 raft_consensus.cc:2272] T 0713792c0cb2436b932fadc8d1f428a2 P d3ac721977e3467892cbf730c5262219 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.642128 11517 tablet_server.cc:196] TabletServer@127.11.63.66:0 shutdown complete.
I20260709 13:23:29.674633 11517 tablet_server.cc:179] TabletServer@127.11.63.67:0 shutting down...
W20260709 13:23:29.680070 24361 consensus_peers.cc:597] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d -> Peer 66e4fe095997420ba9949bad99bf4d50 (127.11.63.65:36043): Couldn't send request to peer 66e4fe095997420ba9949bad99bf4d50. Status: Network error: Client connection negotiation failed: client connection to 127.11.63.65:36043: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:23:29.704535 11517 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:23:29.705343 11517 tablet_replica.cc:333] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d: stopping tablet replica
I20260709 13:23:29.706009 11517 raft_consensus.cc:2243] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:29.706885 11517 raft_consensus.cc:2272] T 9de1041b009d4b3195afb70e0852c742 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.708813 11517 tablet_replica.cc:333] T ef9538c5bb80422eb571d567b173d003 P 3aa42500bbcd4121a43413ecd5be246d: stopping tablet replica
I20260709 13:23:29.709378 11517 raft_consensus.cc:2243] T ef9538c5bb80422eb571d567b173d003 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.709879 11517 raft_consensus.cc:2272] T ef9538c5bb80422eb571d567b173d003 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.711690 11517 tablet_replica.cc:333] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d: stopping tablet replica
I20260709 13:23:29.712258 11517 raft_consensus.cc:2243] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.712692 11517 raft_consensus.cc:2272] T 32ccadb88b4348f7ab9d7a50b967c3d6 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.714638 11517 tablet_replica.cc:333] T 192a3efb7b804c38bca2ad2df1ed1188 P 3aa42500bbcd4121a43413ecd5be246d: stopping tablet replica
I20260709 13:23:29.715204 11517 raft_consensus.cc:2243] T 192a3efb7b804c38bca2ad2df1ed1188 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.715629 11517 raft_consensus.cc:2272] T 192a3efb7b804c38bca2ad2df1ed1188 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.717443 11517 tablet_replica.cc:333] T 337bdcde8c194dccb8aee8a99e36a997 P 3aa42500bbcd4121a43413ecd5be246d: stopping tablet replica
I20260709 13:23:29.718048 11517 raft_consensus.cc:2243] T 337bdcde8c194dccb8aee8a99e36a997 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.718482 11517 raft_consensus.cc:2272] T 337bdcde8c194dccb8aee8a99e36a997 P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.720403 11517 tablet_replica.cc:333] T b027fa0920c8440fab0837dd1142fd57 P 3aa42500bbcd4121a43413ecd5be246d: stopping tablet replica
I20260709 13:23:29.721062 11517 raft_consensus.cc:2243] T b027fa0920c8440fab0837dd1142fd57 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.721540 11517 raft_consensus.cc:2272] T b027fa0920c8440fab0837dd1142fd57 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.723543 11517 tablet_replica.cc:333] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d: stopping tablet replica
I20260709 13:23:29.724105 11517 raft_consensus.cc:2243] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.724735 11517 raft_consensus.cc:2272] T 7f2735d0953445d8b941fd6ad09e76fc P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.726886 11517 tablet_replica.cc:333] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d: stopping tablet replica
I20260709 13:23:29.727451 11517 raft_consensus.cc:2243] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:23:29.728395 11517 raft_consensus.cc:2272] T 33f74f7b25d0436fa15cc6053960e3a3 P 3aa42500bbcd4121a43413ecd5be246d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.731189 11517 tablet_replica.cc:333] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d: stopping tablet replica
I20260709 13:23:29.731798 11517 raft_consensus.cc:2243] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.732249 11517 raft_consensus.cc:2272] T 5a467f6687c5464e8c8ea4429a2f0e4b P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.734133 11517 tablet_replica.cc:333] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d: stopping tablet replica
I20260709 13:23:29.734706 11517 raft_consensus.cc:2243] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:29.735510 11517 raft_consensus.cc:2272] T 32b70a25b0264a8a96170bbecee4837d P 3aa42500bbcd4121a43413ecd5be246d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.737825 11517 tablet_replica.cc:333] T 0689d527efd74dc1a43383e7f3034595 P 3aa42500bbcd4121a43413ecd5be246d: stopping tablet replica
I20260709 13:23:29.738422 11517 raft_consensus.cc:2243] T 0689d527efd74dc1a43383e7f3034595 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.738933 11517 raft_consensus.cc:2272] T 0689d527efd74dc1a43383e7f3034595 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.741163 11517 tablet_replica.cc:333] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d: stopping tablet replica
I20260709 13:23:29.741766 11517 raft_consensus.cc:2243] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.742213 11517 raft_consensus.cc:2272] T 48b0cd91c4e04dc7afb4f78ceeebb0b2 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.744095 11517 tablet_replica.cc:333] T 59ddbccb19fc424c95e59a4455cea713 P 3aa42500bbcd4121a43413ecd5be246d: stopping tablet replica
I20260709 13:23:29.744627 11517 raft_consensus.cc:2243] T 59ddbccb19fc424c95e59a4455cea713 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.745071 11517 raft_consensus.cc:2272] T 59ddbccb19fc424c95e59a4455cea713 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.746946 11517 tablet_replica.cc:333] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d: stopping tablet replica
I20260709 13:23:29.747478 11517 raft_consensus.cc:2243] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.747965 11517 raft_consensus.cc:2272] T 05cc80ee32e0409c959205fbbcf09e58 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.750108 11517 tablet_replica.cc:333] T 0f82955421054b95ad8df878b1827aa4 P 3aa42500bbcd4121a43413ecd5be246d: stopping tablet replica
I20260709 13:23:29.750609 11517 raft_consensus.cc:2243] T 0f82955421054b95ad8df878b1827aa4 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.751056 11517 raft_consensus.cc:2272] T 0f82955421054b95ad8df878b1827aa4 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.753146 11517 tablet_replica.cc:333] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d: stopping tablet replica
I20260709 13:23:29.753720 11517 raft_consensus.cc:2243] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:23:29.754513 11517 raft_consensus.cc:2272] T a91cd6890c73435e9b43afc4262d4f01 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.756793 11517 tablet_replica.cc:333] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d: stopping tablet replica
I20260709 13:23:29.757305 11517 raft_consensus.cc:2243] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.757853 11517 raft_consensus.cc:2272] T 3a38a604d77f4400b910e48e3e9adc54 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.759601 11517 tablet_replica.cc:333] T 0713792c0cb2436b932fadc8d1f428a2 P 3aa42500bbcd4121a43413ecd5be246d: stopping tablet replica
I20260709 13:23:29.760073 11517 raft_consensus.cc:2243] T 0713792c0cb2436b932fadc8d1f428a2 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:23:29.760496 11517 raft_consensus.cc:2272] T 0713792c0cb2436b932fadc8d1f428a2 P 3aa42500bbcd4121a43413ecd5be246d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.800702 11517 tablet_server.cc:196] TabletServer@127.11.63.67:0 shutdown complete.
I20260709 13:23:29.830606 11517 master.cc:562] Master@127.11.63.126:35149 shutting down...
I20260709 13:23:29.854357 11517 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:23:29.854931 11517 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:23:29.855268 11517 tablet_replica.cc:333] T 00000000000000000000000000000000 P 62cb1a883353423da73ff1a94b77b261: stopping tablet replica
I20260709 13:23:29.876488 11517 master.cc:584] Master@127.11.63.126:35149 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (12514 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (27084 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (390454 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 13:23:29.931659 11517 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 413 messages since previous log ~15 seconds ago
I20260709 13:23:29.931923 11517 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 412 messages since previous log ~15 seconds ago
I20260709 13:23:29.932125 11517 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 407 messages since previous log ~15 seconds ago
I20260709 13:23:29.932336 11517 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~35 seconds ago
I20260709 13:23:29.932542 11517 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~35 seconds ago
I20260709 13:23:29.932765 11517 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 6 messages since previous log ~218 seconds ago
I20260709 13:23:29.933035 11517 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 7 messages since previous log ~0 seconds ago
I20260709 13:23:29.933238 11517 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~11 seconds ago
